How to evaluate a fine-tuning dataset before you buy it

The purchase question the test must answer

A pre-purchase test answers one commercial question: does this supplier's data improve your model on your tasks by more than an equal amount of other data would, without breaking behavior you already ship? That framing defines the target metric, the comparison arm and the regression budget before any training run. Sign a one-page test plan fixing them before the sample arrives; once results exist, it is easy to pick the metric, seed or checkpoint that favors the data you already want.

This page, part of the fine-tuning and post-training data guide, covers the protocol for supervised fine-tuning (SFT) and preference data. Pilot logistics are in SourceX's guide to running a data pilot with a supplier, and valuation methods for any data type are in estimating a dataset's value before purchase.

Get evaluation rights that cover training a test model

Confirm in writing that the evaluation grant permits training test models on the sample, because a sample you may only inspect is not one you may fine-tune on. Published evaluation licenses can be narrow: one vendor's sample-data license grants limited, non-exclusive, revocable, non-transferable rights solely to evaluate and test the licensor's own technologies, and bars other uses [1]. Wording like that does not clearly cover fine-tuning your model.

Ask for terms that name the following; clause-level detail is in evaluation licenses and NDAs for dataset samples and evaluation-only license terms.

• Permitted activities: internal training and evaluation of test checkpoints, including full fine-tunes, LoRA adapters and DPO runs.
• Disposal: deletion of the sample, checkpoints, adapters and derived artifacts such as filtered subsets if you do not buy, certified in writing.
• Results: your scores and failure analysis stay confidential; the supplier learns only the decision unless you agree otherwise.
• Sampling method: how the sample was drawn, stated as a term.

Draw the sample the way the full delivery will be built

Results transfer from sample to full dataset only if the sample comes out of the same pipeline: same source systems, date range, filters, de-identification and formatting, chosen by random or stratified random draw.

Specify the sampling frame (for example, stratified by task category and month), seed and record IDs so the delivery can be checked against the sample. Require identical preprocessing: if names in the full set become typed placeholders such as [CUSTOMER_NAME], the sample needs the same placeholders, because the model learns them. Ask for the raw-to-usable ratio after deduplication and filtering. See checking whether a vendor sample is representative and how to request a training data sample.

Inspect the sample before you spend GPU hours

Many weak fine-tuning samples fail cheap checks, so run these before any training and train only on samples that pass.

Lee et al. found near-duplicates common in language-modeling datasets; deduplicated training emitted memorized text about ten times less often, and train-test overlap affected over 4% of validation examples in standard datasets [2]. N-gram overlap is the usual first-pass contamination check, with lab-specific thresholds such as the 13-gram rule associated with GPT-3 [3]. Paraphrased or translated items pass n-gram checks and can evade embedding-similarity filters, and a model trained on rephrased test items can post inflated scores that these checks miss [4].

Scan inputs as well as targets, because personal data in any field enters your training environment, and one study found that fine-tuning on a small set of examples containing personal information can recover personal information a model memorized in pre-training [5]. Size the correctness review with sample sizes for estimating a dataset's error rate, and route generated responses through due diligence for synthetic fine-tuning data.

Design ablation arms that isolate the data

An ablation isolates the dataset's effect by changing only the training data between runs: same starting checkpoint, hyperparameters, training budget and seeds in every arm.

Match arms B and C on supervised tokens (the response tokens that receive loss), not only on example count, or the arm with longer responses gets more training signal. Fix hyperparameters on the baseline, run at least two seeds per arm so seed-to-seed spread is visible, and select checkpoints on a validation split, never on test items.

Preference data needs its own arms. DPO trains directly on records of a prompt, a preferred response and a rejected response, without a separate reward model [6]. Start the candidate and control DPO runs from the same SFT checkpoint with the same β, which controls how far the policy may move from the reference model. Record structure is in preference datasets for DPO, and loss masking in chat fine-tuning data format.

Score target tasks on data the supplier never saw

Score on a private held-out set drawn from your own target distribution. A held-out split of the supplier's sample only measures how well the model fits that supplier's records, which is the assumption the test exists to check.

• Target tasks. Build items from your own prompts or records, with gold answers or a rubric, and keep them private, as in private evaluation sets vs public benchmarks. Share failure categories with the supplier, never items, and keep grading independent of it. One 2025 paper argues that when the party curating evaluations also supplies training data, conflicts of interest and annotator preferences can favor models trained on that party's data [7].
• Regression suites. Re-run instruction following, safety and refusal behavior, and output formats you already ship. InstructGPT's authors observed regressions on some public NLP datasets after RLHF, an "alignment tax" they reduced by mixing in pretraining-distribution updates [8]. Public benchmarks can mislead as regression signals once their items leak into training data: in a post dated February 2026, OpenAI said it had stopped reporting SWE-bench Verified because gains increasingly reflected training-time exposure [9]. See regression suites for production LLM applications.
• Grading. Prefer exact-match and programmatic checks. Validate any LLM judge with a judge calibration set, and report response length beside win rate so a gain from verbosity is visible.
• Your own gold labels. Northcutt et al. estimated an average of at least 3.3% label errors across test sets of 10 widely used datasets and showed that such errors can change model rankings [10]. Audit the items behind any small lift.

Decide whether the lift is real

A lift counts when the paired difference between arms exceeds seed-to-seed spread, its confidence interval excludes zero, and its point estimate meets the minimum lift in your plan. Score every arm on the same items and use paired tests: McNemar's test for pass/fail items, or a paired bootstrap for graded scores.

An ICML 2025 position paper argues that standard CLT-based intervals are too narrow below a few hundred datapoints [11]. In one practitioner's worked example, detecting a lift from 82% to 85% accuracy with an unpaired two-proportion test at 80% power and 5% significance needs about 2,400 examples per variant [12]. Paired designs need fewer, but a 200-item set is unlikely to resolve a 3-point gain. Size the set with eval set sizing for statistical power before the sample arrives.

Read the result before extrapolating to full volume

Use arms D and B to judge the slope, not to project a straight line to full volume. If half to full sample adds little, more of the same will add little too, and the case for buying rests on coverage the sample lacks.

Published results show why. LIMA taught response format and style to a 65B model with 1,000 curated examples [13], so format and tone gains can saturate within a sample. AlpaGasus found that Alpaca's 52,000-example set contained many low-quality instances and that about 9,000 filtered examples outperformed the full set [14].

Illustrative pre-registered test plan

A pre-registered plan is the record that procurement, counsel and the post-training team sign.

Illustrative example: invented to show structure; it does not describe an available dataset.

test_id: ft-preeval-support-replies
decision_owner: post-training lead
candidate_sample:
  records: 4000               # SFT pairs: ticket thread -> agent reply
  draw: stratified random by queue and month, seed and record IDs logged
  preprocessing: same pipeline and placeholders as full delivery
evaluation_terms: training test checkpoints permitted; checkpoints deleted with certificate if no purchase
base_checkpoint: internal-8b-instruct-v3
method: LoRA, hyperparameters fixed on arm A, 3 seeds per arm
arms: [A_baseline, B_plus_sample, C_plus_4000_own_records, D_plus_2000_sample]
match_on: supervised tokens
target_eval: 1200 private items from own tickets, programmatic and rubric grading
regression_evals: [instruction_following, refusal_policy, json_reply_format]
pass_rule:
  primary: B - A >= 4.0 points, paired bootstrap 95% CI lower bound > 0
  versus_control: B - C >= 2.0 points
  regressions: no suite drops more than 1.0 point
pre_training_gates: zero eval-set overlap hits; PII scan clean; expert error rate within plan

Carry the test into license and acceptance terms

The sample test protects you only if the full delivery is checked against it, so write the sample's measured properties into acceptance criteria that let you reject a delivery that drifts.

Mirror the inspection table: schema validity, duplicate rate, zero overlap with your held-out set, a clean PII scan, an expert spot-check error rate, and a category mix within an agreed tolerance of the sample. For large deliveries, re-run arm B on a random slice and expect a lift inside the sample's interval. See acceptance criteria for licensed training data and dataset acceptance testing.

Keep the plan, results and decision on file. If the fine-tuned model will form part of a high-risk AI system under the EU AI Act, Article 10(2) expects data governance that includes an assessment of the availability, quantity and suitability of the data sets [15], which an ablation record can help document. Article 10 covers only high-risk systems, and as of October 2026, Regulation (EU) 2026/1744 is reported to have moved Annex III high-risk obligations to 2 December 2027 and Annex I obligations to 2 August 2028 [16].

When SourceX sources a dataset, diligence materials on source, rights, preparation and allowed use are prepared per dataset for the buyer's review. Personal details are removed or replaced before delivery, the method is recorded for each dataset, and a sample is checked after processing, though no de-identification method is perfect. Ask for the preparation record to cover any sample you test as well as the delivery. Because datasets are sourced on request rather than held in stock, you can start a data request with SourceX that names your target tasks and acceptance criteria.

Supplier behavior that invalidates the test

Three supplier behaviors make a clean result meaningless: refusing a random draw, declining to say how responses were produced, and re-cutting the sample after seeing your scores. Repeated "try this slice instead" rounds select data against your test set, so keep a final holdout no one has scored against, as in contamination-resistant evaluation design. For the supplier-side view, see how suppliers share samples with buyers; SourceX's guide to evaluating data supplier quality covers judging the supplier rather than the data.

Describe the fine-tuning data you want to test

On the SourceX buyer page you can submit a data request describing your target tasks, record format and test requirements, or talk to SourceX first. SourceX looks for US companies that hold matching records, checks their licensing permissions, and manages the license and delivery; nothing is contracted until a supplier agrees. Describe your fine-tuning data needs.