Document Comparison Data: Version Pairs with Labeled Differences, Including Signed and Scanned Copies

Which pair types a comparison model actually needs

Start by deciding which pair types your product must handle, because each one needs different labels and different sources. A redline tool for two Word files is a text-alignment problem; an executed-copy check that compares a counterparty's scan against the approved draft is a cross-modal problem where OCR error, skew and stamps compete with real edits.

• Native vs native (DOCX vs DOCX, PDF vs PDF). Differences can often be derived automatically. WordprocessingML records insertions, deletions, moves and property changes on runs, paragraphs, tables, numbering and sections as revision markup, so a tracked-changes file already contains much of the label. Use these pairs for text-level diff and move detection.
• Native vs scan of the executed copy. This is the high-value pair for signature workflows: the approved draft from the document management system against the image-only PDF that came back with ink signatures, initials, handwritten edits and fax headers. Labels must be created by people or by a verified pipeline; there is no tracked-changes layer in a scan.
• Scan vs scan. Two scans of the same paper at different times, for example a received copy and an archived copy. These test whether the model separates capture noise from content change.
• Unchanged pairs. The same version rendered twice (Word export vs print-to-PDF, 200 vs 300 dpi rescans, a fax of the PDF). These are negatives, and production precision depends on them.

Public document image sets do not cover this. RVL-CDIP is a large scanned-image collection labeled for classification and retrieval [2], and FUNSD contains 199 noisy scanned forms annotated for form understanding [1]; neither ships version pairs with difference labels. For a wider view of what open sets allow, see public document AI datasets and commercial use.

A difference taxonomy that survives disagreement between annotators

Define difference types narrowly enough that two annotators label the same change the same way. Most label disputes in comparison projects come from moved-versus-deleted-and-inserted text, from whether a reflowed paragraph counts as a change, and from what counts as "material."

A workable taxonomy separates content changes from presentation changes and adds a materiality flag rather than deleting low-stakes changes from the labels:

Keep "formatting-only" and "signature present" as their own classes. A product that checks executed copies needs to know a signature appeared, but it should not report it in the same list as a changed payment term.

Labeled record schema for a version pair

Ask for one record per pair, with page-level and region-level differences, alignment and provenance in a sidecar file next to the original files. The schema below is a request template you can hand to a supplier or an internal annotation team.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "pair_id": "pair-000412",
  "pair_type": "native_vs_scan",
  "doc_family": "master_services_agreement",
  "left":  {"file": "v7_approved.docx", "format": "docx", "source": "dms_version_history", "version_label": "7", "author_role": "counsel"},
  "right": {"file": "executed_scan.pdf", "format": "pdf_image_only", "dpi": 300, "capture": "flatbed", "pages": 14},
  "page_alignment": [[1,1],[2,2],[3,3],[12,null]],
  "differences": [
    {"type": "numeric_change", "left_span": {"page": 4, "text": "thirty (30) days"}, "right_region": {"page": 4, "bbox": [412,880,690,912]}, "right_text": "forty-five (45) days", "material": true},
    {"type": "handwritten_edit", "right_region": {"page": 6, "bbox": [120,1010,640,1060]}, "initialed": true, "material": true},
    {"type": "page_missing", "left_span": {"page": 12}, "material": true},
    {"type": "signature_block", "right_region": {"page": 14, "bbox": [100,1500,700,1620]}, "material": false}
  ],
  "is_unchanged_control": false,
  "label_method": "two_annotators_plus_adjudication",
  "ocr_engine_for_right": "recorded_in_manifest",
  "redaction": {"method": "replace_names_accounts", "sample_checked": true}
}

Fields worth insisting on: pair_type, page alignment that allows null for missing pages, region coordinates on the image side, text spans on the native side, the materiality flag, and the label method. Record author_role, not author names. For the delivery format of originals plus sidecars, see delivering original files with sidecar metadata.

Where real version pairs come from inside a business

Real pairs come from three places, and each produces a different label quality. Ask suppliers which of the three they draw from and in what proportions.

1. Document management version history. Systems such as SharePoint, iManage or NetDocuments keep numbered versions. Consecutive versions give native-to-native pairs; the gap between the last approved version and the filed executed copy gives the cross-modal pair.
2. DOCX files with tracked changes. Accepting or rejecting all revisions produces two clean versions plus an exact change list. Watch the failure mode: deleted text stored in w:t instead of w:delText, or revisions without author and date attributes, which breaks automatic derivation.
3. Executed final copies. Signed scans, e-signature completion certificates and countersigned PDFs, typically stored in contract repositories or ERP attachments. Under the federal E-SIGN Act, an electronic record can satisfy a retention requirement if it accurately reflects the information and remains accessible and reproducible, which is one reason companies keep the executed PDF as the record of truth and why comparing it against drafts matters.

Negotiation redlines between counterparties are a separate asset covered on the contract redline datasets page; draft-to-final edits used as a quality signal sit on human feedback QA scores and corrections; and broad saved revisions are listed under enterprise document archives. If your use is rewriting rather than detection, draft-to-final pairs for rewriting models is the closer guide.

Negatives and noise: teaching the model what not to flag

A comparison model is judged in production by its false alarms, so budget a large share of pairs for unchanged documents and formatting-only noise. Without them, models flag every rescan, every PDF renderer difference and every fax header as a change.

Noise sources to include deliberately:

• Different renderers of the same DOCX (Word "Save as PDF", LibreOffice export, print drivers) that shift line breaks and hyphenation.
• Rescans at different dpi, skew, color mode and compression, plus fax and photocopy generations; degraded document capture conditions covers these in depth.
• Header, footer and Bates-number differences, DMS footers with version numbers, and e-signature envelope IDs stamped on each page.
• Page reordering from batch scanning, which overlaps with page-stream segmentation data.

Synthetic edits (programmatically changing a number in a clean PDF) are useful for scale but under-represent handwritten strikes, initials and real OCR drift. The trade-offs are covered in synthetic vs real documents.

Evaluation split and metrics for change detection

Split by document family and by originating organization, not by pair, or versions of the same agreement leak between train and test. Report metrics separately per pair type, because native-to-native scores will hide failures on scans.

Useful measures: precision and recall on material differences; localization accuracy (overlap of predicted and labeled regions); type accuracy (did the model call a move a move); page-level missing-page recall; and false-alarm rate on unchanged control pairs. Long agreements with exhibits stress alignment, so include some, as described in multi-page and long document data. For a full request template beyond comparison, use the document dataset requirements spec.

Privacy and confidentiality in tracked changes and comments

Tracked changes and comments carry more sensitive content than the final text, so agree how they are handled before preparation starts. Revision markup stores author names and timestamps, and comment threads often contain candid remarks about counterparties, pricing positions or employees.

Practical rules to put in the request: replace author names with roles; strip or summarize comment bodies unless comments are part of the task; redact names, emails, phone numbers and account numbers consistently across both versions of a pair so the redaction itself is not detected as a difference; and remove pixel content, OCR text layers and PDF metadata together on scans, as described in redacting PII in scanned documents. If pairs include health records, HIPAA de-identification applies [3].

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

How SourceX approaches document version pair requests

SourceX sources operational datasets, including documents and finance and legal workflows, from US companies on request; it does not hold version pairs in stock, and a request does not guarantee a match. Buyers describe the data they need, such as pair types, difference classes and document families, and SourceX looks for US businesses that hold it, with every release approved by the supplying company.

Each dataset is rights-reviewed for ownership and consents and delivered under a license defining records, uses, term and delivery. Personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. You can describe the version pairs you need to SourceX. For the broader cluster, start at the document AI data hub or the AI data overview.

Request document comparison training data

SourceX sources documents and legal and finance workflow records from US companies on request, with rights review and a license that defines records, uses, term and delivery. Nothing is contracted until a supplier agrees. Describe your document version pair requirements.