Critique and revision data for training generative verifiers

Why generative verifiers need written critiques, not just labels

A generative verifier is trained to write its reasoning about a candidate answer and then emit a verdict, so its training data must contain the reasoning, not only a score. In this design the verifier is an ordinary language model trained with next-token prediction on rationales followed by a verdict token, which also lets you sample several rationales at inference time and aggregate them. A scalar reward model trained on rankings, as in the InstructGPT pipeline, learns a preference without any explanation [1]. A critic that only sees "accept" or "reject" cannot learn which defect justified the decision.

Published work on code critics shows both the payoff and the risk: trained critics can catch bugs that human reviewers miss, but they also report bugs that are not there. NIST AI 600-1 lists confabulation among the core generative AI risks [7], and a critic that invents defects is a confabulating critic. Your data has to teach both recall (find the real problem) and restraint (do not invent one).

Anatomy of a usable critique-revision record

A usable record holds five linked parts: the artifact as the reviewer saw it, the critique text anchored to a location, a severity, the final decision, and the revised artifact. The anchor matters because a comment like "fix this" is only meaningful next to the line, clause or drawing region it refers to. The outcome matters because it tells the model which critiques changed the result. Without the revision you have a judge dataset; with it you also have self-correction training data.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "record_id": "rv-000184",
  "domain": "code_review",
  "artifact_before": {"format": "unified_diff", "path": "billing/proration.py", "content_ref": "blob://a91f"},
  "task_context": "PR description: prorate plan changes mid-cycle",
  "critiques": [
    {
      "critique_id": "c1",
      "anchor": {"file": "billing/proration.py", "line_start": 42, "line_end": 47},
      "text_original": "off by one?",
      "text_expanded": "days_remaining excludes the change date, so a same-day upgrade is billed zero days on the new plan.",
      "category": "correctness",
      "severity": "blocking",
      "reviewer_role": "senior_engineer",
      "resolution": "accepted",
      "resolved_by_revision": "rev-2"
    },
    {
      "critique_id": "c2",
      "anchor": {"file": "billing/proration.py", "line_start": 12, "line_end": 12},
      "text_original": "nit: rename var",
      "category": "style",
      "severity": "nit",
      "resolution": "rejected_by_author",
      "resolution_note": "matches existing naming in module"
    }
  ],
  "final_decision": "approved_after_revision",
  "artifact_after": {"format": "unified_diff", "content_ref": "blob://c03d"},
  "verification": {"tests_passed_after": true, "ci_run_ref": "ci-7731"},
  "deidentification": {"method": "pseudonymized_ids_and_regex_scrub", "sample_checked": true}
}

Three fields do most of the work. resolution separates critiques that mattered from noise. text_expanded keeps the original terse comment for realism while giving the model a self-contained rationale. verification ties the outcome to an external signal, such as a test run, so labels do not rest only on the reviewer's word.

Where real critique data lives in business records

Expert critiques already exist wherever an organization reviews work before release, and each source has a characteristic structure. The table below maps common systems to what they contribute and where they fall short.

Illustrative example: invented to show structure; it does not describe an available dataset.

Public examples confirm the shape but rarely the depth. Seattle publishes reviewer remarks on building permit plan sets, typically many per plan set [8], which is useful for studying comment style but lacks the drawings. For domain specifics, see our guides to plan review comments and code corrections, code review comment resolution data and MLR promotional review comments. If you plan to license QA notes or pull request history, the owner pages for QA scorecards and code reviews and pull requests describe those categories.

Turning terse review comments into training targets

Natural review comments are short and assume shared context, so most need filtering or expansion before they can supervise a critic. Reviewers write "see std," "same as above" or a bare question mark because the author knows what they mean. A model trained on these learns to be vague. Treat expansion as a labeled transformation, not a silent rewrite.

A practical pipeline looks like this:

1. Resolve references. Replace "same as above" and "see earlier" with the comment they point to; drop threads where the reference cannot be resolved.
2. Attach context. Store the anchor span plus a window around it, and the task description, so the critique is checkable.
3. Classify, then filter. Tag each critique (correctness, safety, compliance, style, nit) and decide per use: a verifier for correctness usually drops style nits.
4. Expand with provenance. If an annotator or model expands a terse comment, keep text_original, store text_expanded, and record who or what wrote it. Expansions written by a model are synthetic and should be audited like any synthetic data.
5. Deduplicate. Template comments ("please add tests") can repeat many times across a review history; near-duplicate removal reduces memorization and copying, as shown for pre-training corpora [5].
6. Balance outcomes. Keep rejected critiques and "approved with no comments" cases so the critic learns that some artifacts are fine.

Checklist-style feedback is a useful bridge. RLCF extracts a checklist from each instruction and scores responses item by item [2], and many QA rubrics are already checklists, so scorecard items can become critique prompts.

Linking critiques to outcomes so the verifier learns what mattered

Outcome linkage is what turns a pile of comments into a verifier signal: each critique should be traceable to acceptance, revision or rejection. A critique that the author accepted and fixed, followed by a passing test, is strong positive evidence. A critique rejected by the author and upheld by a second reviewer is a strong negative example for over-flagging, which is exactly the invented-defect failure mode described above.

Watch for three linkage failures. First, squash merges and force pushes destroy the commit that answered a comment, so ask whether the supplier's history preserves intermediate revisions. Second, approvals often happen out of band (a meeting, a chat thread), leaving a "resolved" flag without a reason. Third, revision corpora show that edits mix intents, such as clarity, fluency and meaning changes [3], so a revised version is not proof that the critique was correct; the edit may address something else.

Revision pairs for self-correction training

Self-correction training needs triples of draft, critique and revision, where the revision demonstrably addresses the critique. Before-and-after pairs without the critique teach rewriting, which is covered in our guide to draft-to-final document pairs. The critique in the middle is what lets a model learn to diagnose before it edits.

Check alignment at the span level. A useful test is whether the diff between draft and revision overlaps the critique's anchor; a revision that touches unrelated sections suggests the critique was ignored or the pair is misaligned. Keep partial fixes labeled as partial, since they are realistic and teach the critic to re-check.

Buyer due diligence for critique datasets

Before licensing, test a sample against the questions below; many problems in critique data are visible in a modest sample.

Illustrative example: invented to show structure; it does not describe an available dataset.

Document the result. A data statement describing who wrote the critiques, in what setting and under what review norms [6] helps your team interpret critic behavior later. Open datasets such as HelpSteer2 show how attribute ratings and annotation guidelines can be released alongside responses for reward-model training [4], a useful baseline for what documentation to expect. For general screening steps, see how to evaluate a fine-tuning dataset before you buy it.

How critique data fits with preference and RLVR data

Critique data complements, rather than replaces, preference pairs and verifiable-reward datasets. Preference pairs tell a reward model which output is better; see preference datasets for DPO. RLVR prompts with reference answers give a programmatic check where one exists; see RLVR datasets. Critique records cover the large middle ground of tasks, such as permit sets, contracts and support answers, where no unit test exists and an expert's written reason is the best available verifier. For the full landscape, start at the fine-tuning and post-training data hub or the AI data guides. Judge calibration sets for evaluation are a separate need from critic training data and belong in your eval plan.

Sourcing critique and revision data through SourceX

SourceX sources operational datasets from US companies on request, including engineering records, support histories and document workflows where review comments live; nothing is held in stock and a request does not guarantee a match. You describe the data, such as anchored review comments with outcomes and revisions, and SourceX looks for businesses that hold it. Each dataset is rights-reviewed and delivered under a license, with personal details removed or replaced before delivery. Describe the critique data you need on the buyer page.

Request critique and revision data for your verifier

Describe the artifacts, critique types and outcome signals you need, and SourceX will assess data and licensing permissions with suppliers before anything is agreed. Every release is approved by the supplying company, and terms are agreed per deal. Start a buyer request.