Schema Matching and Source-to-Target Mapping Data for Data-Integration AI

Why public schema matching benchmarks fall short

Public benchmarks are useful for method comparison but do not reflect the width, noise and transformation logic of production mappings. Many academic matching benchmarks are built by fabricating table pairs, splitting or perturbing one source table so the correct correspondences are known by construction. Fabrication gives clean labels, but the column names, abbreviations and data drift a matcher sees in a 20-year-old ERP extract are hard to synthesize.

Width is the second gap. Spider 2.0, built on databases from real applications, reports enterprise schemas that often exceed 1,000 columns [1][2], and synthetic text-to-SQL pipelines inject distractor columns to imitate that noise [3]. A matcher trained on 15-column toy tables learns little about candidate pruning across a 1,400-column warehouse. The same tension shows up in model design: small embedding or classifier matchers need labeled pairs to learn domain vocabulary, while prompting an LLM with two full schemas runs into context and cost limits at that width.

The third gap is transformation logic. Most benchmarks label only correspondences ("cust_nm matches customer_name"), while real migration work also needs the expression that converts a value: date format changes, code-table lookups, unit conversions, concatenations and splits. If your product generates mapping code, not just match suggestions, you need labels for both.

Which business artifacts contain mapping ground truth

The richest ground truth sits in documents integration teams produce to build and sign off pipelines. Each artifact type carries different labels and different reliability.

• Source-to-target mapping (STTM) workbooks. Spreadsheets with one row per target column: source table, source column, data types, transformation rule in prose or SQL, default values, and status. These are the closest thing to a labeled matching record, but prose rules vary in precision.
• Integration and interface specifications. Documents describing field-level contracts between systems (for example an order system feeding a warehouse management system), often with examples and code lists.
• iPaaS and ETL tool mapping configs. Exported mapping definitions from integration platforms and ETL tools, usually JSON or XML. They are executable, so the rule is exact, but they omit the reasoning for skipped fields.
• dbt models and SQL transformation code. Staging and mart models encode column lineage and renames directly in SELECT statements; paired with source DDL they yield correspondences plus transformation expressions.
• Migration cutover and reconciliation records. Validation reports from tools such as Google's Data Validation Tool, which compares source and target with row counts, column aggregates and row-level checksums after each migration step [4]. These show whether a mapping actually held up on loaded data.
• Data dictionaries on both sides. Column descriptions disambiguate cryptic names; see data dictionaries and schema documentation for AI.

Ask suppliers which of these exist for each project, and prefer projects where a workbook, executable config and reconciliation record all survive. That triangulation lets you separate the rules that shipped from the ones that were drafted and abandoned.

What a labeled mapping record should contain

A usable record pairs both schemas with correspondence-level labels, transformation rules, cardinality and explicit non-matches. Without explicit unmapped fields with reasons, you cannot train or measure a model's ability to abstain, which is where production matchers cause the most damage.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "pair_id": "mig-0412",
  "source_schema": {"system_type": "legacy order entry", "dialect": "DB2", "tables": 38, "columns": 611},
  "target_schema": {"system_type": "cloud CRM", "objects": 12, "fields": 284},
  "correspondences": [
    {"target": "Account.BillingPostalCode", "sources": ["CUSTMAST.ZIP5", "CUSTMAST.ZIP4"],
     "cardinality": "n:1", "rule": "ZIP5 || CASE WHEN ZIP4 <> '' THEN '-' || ZIP4 END",
     "rule_language": "SQL", "verified_by": "reconciliation", "confidence": "validated"},
    {"target": "Account.Industry", "sources": ["CUSTMAST.SICCD"],
     "cardinality": "1:1", "rule": "lookup(sic_to_industry_v3, SICCD)", "lookup_table_included": true}
  ],
  "unmapped_source": [
    {"column": "CUSTMAST.FILLER2", "reason": "unused padding field"},
    {"column": "CUSTMAST.CRLIMIT", "reason": "out of scope; stays in ERP"}
  ],
  "unmapped_target": [{"field": "Account.Rating", "reason": "no source; set manually"}],
  "sample_values": "masked, 50 rows per source table",
  "label_provenance": {"author_role": "integration consultant", "signed_off": true, "post_load_validated": true}
}

Keep correspondences and rules as structured fields rather than a free-text cell, and record which labels were validated against loaded data versus only drafted. For dataset-level description, a Croissant JSON-LD file can document resources and record structure so loaders and catalogs read the package consistently [5].

How to evaluate a schema matcher on licensed mapping data

Evaluate at the column-correspondence level on held-out schema pairs, then separately score transformation correctness. Splitting rows from the same pair across train and test leaks the vocabulary of that schema and inflates scores.

Stratify the held-out set by source system family, schema width and domain, because a matcher that performs well on CRM-to-CRM pairs may collapse on mainframe-to-cloud pairs. If your system retrieves candidates cheaply and reranks them with an LLM, score the two stages separately so you can tell retrieval recall failures from reranking precision failures.

Failure modes in real mapping documents

Mapping workbooks are written for humans under deadline, so expect label noise you must detect before training.

• Stale rows. The workbook was last edited before go-live, while the shipped config changed. Compare against the executable config or dbt code.
• Prose rules that are not executable. "Convert to standard format" is not a label. Tag such rules as unverified or drop them from transformation training.
• Silent defaults. Target fields filled with constants appear mapped but have no source; label them as such.
• Duplicate projects. The same vendor schema migrated by many customers can dominate a corpus; deduplicate by target schema version or the model memorizes one SaaS object model.
• Lookup tables missing. A rule referencing a code-crosswalk is unusable if the table is absent.
• Target leakage in features. If sample values were captured after load, the target values can reveal the answer; see target leakage audits for tabular data.

Rights and privacy questions specific to mapping data

Mapping artifacts mix the supplier's own work with material others may control, so rights review has to cover each layer. A mapping workbook may describe a SaaS vendor's object model, be authored by an outside integrator under a services agreement, and include sample rows containing customer personal data. Ask who authored each artifact, whether the services contract assigned work product to the company, and whether any vendor terms restrict sharing schema documentation or exported configs.

Sample values deserve the most care. Names, emails, phone numbers and account numbers in sample rows should be removed or replaced before delivery, ideally with format-preserving substitutes so type and pattern signals survive for the matcher. Column names themselves can occasionally reveal confidential business lines or customer names in table prefixes; decide with the supplier whether to tokenize them. Background on migration-era licensing is in licensing data before a CRM migration, and the ETL glossary entry covers terminology.

How this differs from neighboring structured-data tasks

Schema matching aligns two schemas to each other, which is a different labeling problem from linking a question to columns or matching records. Schema linking data maps a natural-language question onto one schema, and text-to-SQL training data pairs questions with queries. Entity resolution data matches rows that refer to the same real-world entity, while schema matching matches columns. Code conversion between SQL dialects is covered in SQL dialect migration datasets. For the whole cluster, start at the structured data buyer's guide or the AI data hub.

Sourcing request checklist for mapping data

Describe the data you need in terms a supplier can check against its project archive. A precise description lets data holders give a clear yes or no.

Illustrative example: invented to show structure; it does not describe an available dataset.

SourceX sources operational datasets, including engineering records and documents, from US companies on request; categories are not inventory and a request does not guarantee a match. You can describe your mapping data need to SourceX, and the supplying company approves every release.

Request real source-to-target mapping data

SourceX looks for US businesses that hold the mapping artifacts you describe, reviews ownership and consents, and delivers each dataset under a license that defines records, uses, term and delivery. Personal details in sample values are removed or replaced before delivery, the method is recorded and a sample is checked (no method is perfect), and nothing is contracted until a supplier agrees. Start a buyer request.