Checkbox and Selection Mark Data for Form Extraction

Checkbox errors tend to surface after a key-value model is already in production. Text fields look right, but "Smoker: Yes/No" or "Prior claims in 5 years" comes back wrong or empty, and the downstream rule engine acts on it. This page covers how to specify, source and evaluate selection-mark data for that sub-task. For text-field labels, see key-value extraction labels for annotated forms; for the wider task map, see the Document AI data hub.

Why off-the-shelf selection marks are not enough

Commercial layout and OCR APIs typically return a binary state per mark, such as selected or unselected, with a bounding region and a confidence score. That output is the floor your own labels must exceed, not a substitute for them.

That schema has two gaps for production forms. First, it does not say which question the mark answers, so a page with 40 Yes/No pairs still needs a linking step. Second, a binary state hides the marks that cause disputes: a box filled then scratched out, an X that strays into the neighboring box, or a tick written beside the box rather than in it.

Vision-language models used for zero-shot extraction can show the same blind spot: they read the printed option text fluently but may misjudge which option is marked. Public form sets help but are small and skewed: FUNSD, the common form-understanding baseline, has 199 annotated scanned forms and labels entities and links rather than mark states [1].

A mark-state taxonomy that matches real forms

Label at least six states, then collapse them to the binary your product needs at training or scoring time. Collapsing later is cheap; relabeling 50,000 boxes because "partial" was folded into "checked" is not.

Illustrative example: invented to show structure; it does not describe an available dataset.

Radio-button groups need one extra rule: record the group ID and whether the form allows one answer or many. A "select one" group with two marks is a data-quality signal in its own right, and models that reconcile it need examples where the second mark is corrected rather than a second selection.

Linking each box to its question and value

A mark only becomes an extraction result when it is joined to a label and resolved into a field value. Ask for three linked layers per box: the box polygon, the option label text (for example "Yes", "Owner-occupied", "Type 2 diabetes"), and the parent question or field key with its group ID. FUNSD's entity-linking design is a reasonable pattern to copy for those relations [1].

Illustrative example: invented to show structure; it does not describe an available dataset.

{"doc_id": "f-000193", "page": 2, "form_family": "life_app_part_b", "form_version": "rev_2022_03",
 "box_id": "b-17", "bbox": [412, 1180, 438, 1206], "shape": "square",
 "group_id": "q14_tobacco", "group_rule": "single_select",
 "question_text": "Have you used tobacco in the last 12 months?", "option_text": "No",
 "state": "corrected", "binary": "unselected", "resolved_field": {"key": "tobacco_use_12m", "value": "Yes"},
 "annotator_ids": ["a3", "a7"], "agreement": false, "adjudicated_by": "lead_2", "notes": "No box scratched, Yes box ticked and initialed"}

Delivered as JSON Lines, one record per box, this joins cleanly to page images and to any existing key-value labels. The resolved_field column is what your evaluation should score, because a perfect detector that links "Yes" to the wrong question is still a wrong answer. If the forms also exist as system-of-record entries, the captured value can act as a cross-check; see documents paired with system-of-record entries.

Hard cases synthetic marks underrepresent

Rendered checkmarks on clean templates teach the easy part; the failure modes live in capture and handwriting variance. Treat the following as a coverage checklist in your request, with a minimum count or share per item rather than a vague "include edge cases."

• Faint and low-contrast marks: pencil, light ballpoint, second and third carbonless copies.
• Offset marks: ticks that overlap the box border, land between two boxes, or sit next to the option label.
• Over-written and corrected answers: scribble-outs, "X" over a tick, initials beside a change.
• Capture artifacts: fax halftoning, photocopy speckle that mimics a dot, skew, phone-photo perspective. The degraded document images guide covers how to specify these.
• Box design variety: squares, circles, brackets, underscores used as boxes, and boxes printed in drop-out red ink that disappears in some scans.
• Dense layouts: medical history grids and government eligibility sections with 50 or more boxes per page.
• Non-box selections: circled words, struck options and handwritten "N/A" across a group.

Detector choice does not remove the need for these examples. Whether you train a YOLO- or DETR-style object detector, fine-tune a document model, or start from an OpenCV contour heuristic that treats near-square regions as boxes, each fails differently on this list. A shape heuristic in particular breaks on circles, brackets and drop-out ink, which is exactly where real forms vary.

Measuring checkbox accuracy without hiding the hard cases

Report metrics per state, per form version and per capture condition, and score ambiguous marks on their own line. A single "checkbox accuracy" number is dominated by empty boxes: on a medical history page where most boxes are unchecked, a model that predicts unchecked everywhere can look strong while missing every positive answer.

Illustrative example: invented to show structure; it does not describe an available dataset.

Budget for label noise in the test split itself. An audit of ten widely used benchmarks estimated an average test-set label error rate of at least 3.3% [2], and checkbox labels are especially exposed because a faint tick can be missed by a tired annotator. Double-annotate the evaluation split, keep the disagreement log, and freeze the split by form version so a supplier's new batch cannot leak into it.

Sourcing real filled forms with mark-state labels

Real filled forms come from organizations that process them: insurers and agencies (applications, supplements, claim notices), healthcare providers and billers (intake and history forms), and businesses filing permits, benefits or tax paperwork. Each brings different constraints, and a request should state the form families, versions, capture channels and label layers rather than a raw page count.

Nearby guides cover form families with their own field logic: ACORD form data for insurance submission intake, CMS-1500 and UB-04 claim form images, and document extraction correction logs from production IDP, which are a strong source of hard checkbox cases because reviewers fix exactly the marks the model got wrong.

Illustrative example: invented to show structure; it does not describe an available dataset.

Request template: checkbox and selection-mark data

1. Form families and versions: for example life application Part B, patient intake, permit application; list revision dates you must cover.
2. Capture channels and mix: flatbed scan, fax, mobile photo, born-digital with AcroForm fields flattened.
3. Label layers: box polygon, shape, state (six states plus ambiguous), group ID and rule, option text, question key, resolved value.
4. Annotation process: double annotation rate, adjudication rule, agreement metric reported per state.
5. Hard-case minimums: share of partial, crossed-out, corrected and offset marks; share of dense grids.
6. Splits: train and evaluation separated by form version and source, evaluation double-annotated.
7. Personal data handling: which identifiers are removed or replaced in images and text, how redaction is verified, and the method used for any health records.
8. Delivery format: page images (TIFF or PNG at stated DPI) plus JSON Lines per box, with a manifest of checksums.

Health forms deserve particular care. Intake and history pages are protected health information until de-identified, and HHS describes two routes, Safe Harbor and Expert Determination [3]. Image redaction is harder than text redaction: names, dates of birth and member IDs sit in handwriting next to the very boxes you need, so redaction boxes must not obscure the selection marks or their labels.

SourceX sources operational datasets from US companies on request, including documents and finance and legal workflow records. Every dataset is rights-reviewed for ownership and consents and delivered under a license that defines the records, uses, term and delivery. Names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect; health records require HIPAA de-identification by Safe Harbor or Expert Determination. Buyers can describe the checkbox data they need on the SourceX buyers page; requests are matched to US businesses that hold the data, and a request does not guarantee a match.

Where checkbox data fits in a form-extraction program

Selection marks are one layer in a form stack, and they usually ride on the same pages as text fields, signatures and tables. If you are licensing scanned forms anyway, ask for checkbox labels in the same annotation pass so box-to-question links share IDs with your key-value labels. SourceX also has a page on licensing scanned forms and handwritten documents for AI training, and related tasks such as signature and stamp detection data often share the same source documents. The master AI data guide index lists other document and structured-data tasks.

Source real filled forms for checkbox detection

SourceX finds US businesses that hold the filled forms you describe, assesses the data and licensing permissions, and agrees pricing and allowed uses in a license before anything is delivered. Every release is approved by the supplying company and delivered through private, access-controlled workflows. Start by describing the form families, mark states and label layers you need at sourcex.si/buyers.