Privacy, de-identification and sensitive data
Is synthetic data derived from licensed records still personal data?
Quick answer
Often, yes. Synthetic records are not anonymous just because they were generated. Under GDPR they leave the regime only if no one can identify a person from them by any means reasonably likely to be used [2]. The EDPB applies a similar bar to models trained on personal data: extraction and disclosure through queries must be insignificant [1]. Research shows that common generators either leak membership or lose utility [3][4]. Treat synthetic data as personal data until a documented attack-based evaluation shows otherwise.
By SourceX Editorial · Updated
- 01Your data spec
- 02Matched to businesses
- 03Rights and samples reviewed
- 04Licensed delivery
This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.
This page covers privacy status only. Whether your license lets you generate derived data at all is covered in rights to synthetic data from licensed data. The commercial value question is covered in synthetic data vs. real business data.
Why "synthetic" is not a legal category
No major privacy statute treats synthetic data as exempt by definition. The test is identifiability, applied to the output you actually hold. GDPR Recital 26 excludes only information that does not relate to an identified or identifiable person, judged by all means reasonably likely to be used, including singling out [2]. The UK ICO frames the same question through the "motivated intruder": could a reasonably competent person with access to other resources identify someone [5]?
A generator is a model fitted to personal data, so the two-part question in EDPB Opinion 28/2024 maps onto it directly. A model is anonymous only if the likelihood of extracting personal data about training subjects, directly or probabilistically, is insignificant. The same must hold for obtaining that data through queries, and the assessment is made case by case [1]. Synthetic output is in effect a large batch of queries already run and saved to disk. If the generator could not pass this test, its output deserves suspicion. For the model-side analysis, see EDPB Opinion 28/2024 and model anonymity.
US rules point the same way through different wording. A HIPAA covered entity that synthesizes from protected health information has two routes. It can strip the Safe Harbor identifiers, or it can obtain an Expert Determination that re-identification risk is very small [7]. "We used a generative model" is neither route. Unless the output demonstrably meets Safe Harbor, synthetic health data shared outside the covered entity usually needs an expert's written determination that addresses the generator itself.
Four ways synthetic records leak their sources
Synthetic data leaks through memorized records, rare-combination copies, membership signals and attribute inference. A single fidelity score will not reveal any of them.
- Verbatim or near-verbatim copying. A model that has overfit, or that saw a record many times, can reproduce it. In tabular data this looks like a synthetic row matching a real row on every column. In text it looks like a support ticket reproduced with the customer's name and order number intact. LLM-based text generators fine-tuned on transcripts are especially prone to this when a template repeats.
- Outlier reproduction. Generators without differential privacy tend to preserve rare combinations, because that is what fidelity rewards. Picture a 94-year-old account holder in a small ZIP code with a rare diagnosis code. If that pattern appears in the synthetic set, it singles out one real person even though no field was copied verbatim.
- Membership inference. An attacker who holds a candidate record can test whether it was in the training data. Statistical traces in the synthetic output are enough. Stadler, Oprisanu and Troncoso showed that across several generator families, synthetic datasets either failed to block such inference or gave up the utility that justified them [3].
- Attribute inference. Partial knowledge about a person, such as employer, age band and city, can let an attacker predict a sensitive field from the learned correlations. Sometimes this is the generalization the buyer wants. When the population is small, it becomes disclosure.
Synthetic text has its own pitfalls. Meeus and colleagues showed that membership inference works against LLM-generated synthetic text. They also showed that canaries built for model audits are poorly suited to auditing synthetic data and need redesign [4]. A vendor claim that "the text is synthetic, so no PII remains" says nothing about these risks. PII redaction of the source, covered in PII redaction for LLM training data, lowers the stakes but does not remove membership or attribute signals.
What a defensible privacy evaluation contains
A defensible evaluation models a realistic attacker and runs attacks against held-out records. It reports results per risk, not as one blended score. Distance metrics alone are not evidence of anonymity. The tabular synthesis literature notes that privacy metrics vary widely across studies and often lack a principled link to attack success [8]. Tools such as SDMetrics compute fidelity and some privacy-oriented comparisons against the real data. Treat them as one input, not a verdict [9].
Require each of the following and read the method, not just the number.
- Holdout design. Split the real data into a training set and a holdout set before generation. Every comparison should ask whether synthetic records sit closer to training rows than to holdout rows. Without a holdout, a low distance proves nothing.
- Distance to closest record (DCR). Report the full distribution of nearest-neighbor distances, synthetic to training versus synthetic to holdout. Also report exact-match counts. Choose a sensible distance for mixed types, such as Gower distance for tabular data or normalized edit distance and embedding similarity for text.
- Membership inference attack. Run a shadow-model or likelihood-ratio attack. Report true positive rate at a low false positive rate, such as 0.1% or 1%, alongside AUC. Average-case AUC hides the outliers that matter most.
- Canary insertion. Plant unique synthetic secrets in the training set before generation. Repeat them at several frequencies, then measure how often they or their rare tokens reappear [4].
- Outlier audit. Profile the rarest quasi-identifier combinations in the source, such as age, geography, rare codes and job titles. Check whether those combinations survive into the output.
- Formal guarantee, if claimed. If the generator was trained with differential privacy, record epsilon, delta and the accounting method. Confirm whether preprocessing, such as fitting vocabularies or category lists on raw data, sat outside the privacy budget. NIST SP 800-188 explains why formal privacy methods carry guarantees that traditional de-identification does not [6].
Differential privacy changes the conversation because it bounds what any one record can contribute. For text pipelines, see differentially private synthetic text.
A decision table for privacy status
Classify a synthetic dataset by its generation method and the evidence behind it, then attach obligations to that class. Use the table below as a starting rubric with counsel, not as a substitute for the case-by-case assessment regulators expect [1][5].
Illustrative example: invented to show structure; it does not describe an available dataset.
| Generation method and evidence | Working status | Typical obligations |
|---|---|---|
| Non-DP generator, no holdout attacks run | Personal data | Lawful basis, purpose limits, security, transfer rules, license use limits; treat like the source |
| Non-DP generator, DCR and MIA run, no exact matches, MIA near chance at low FPR | Likely still personal data in small or rare populations; argue anonymity only with a written motivated-intruder assessment [5] | As above, plus documented risk assessment; restrict onward sharing |
| DP-trained generator, documented epsilon, preprocessing inside budget, attacks confirm | Strongest case for anonymity or Expert Determination [6][7] | Keep the evaluation report; re-run on regeneration; monitor linkage to new public data |
| Rule-based or simulator output with no fitting to personal data | Usually not personal data, if no real records seed the templates | Confirm no copied seed records; record provenance |
| LLM prompted with real records as few-shot examples | Personal data; the prompt is a direct copy channel | Treat output as derived personal data; redact before prompting |
Status can change after delivery. New public datasets, leaks or auxiliary sources raise the "means reasonably likely" baseline [2]. Re-assess when the threat landscape shifts, not only when you regenerate.
Obligations that follow the data downstream
If the synthetic set is personal data, the source's obligations follow it. That includes the use restrictions in your license and the privacy duties of the original collection. In GDPR terms you need a lawful basis for the generation step itself, which is processing of the source data. You also need one for each later use of the output. Where the synthetic set is pseudonymous rather than anonymous, the recipient analysis in receiving pseudonymised data applies.
Watch three practical traps.
- Eval and sharing sets. Synthetic data built "for sharing with a vendor" is the highest-exposure use. Run the full attack suite before it leaves your environment.
- Sector rules. Synthetic data from protected health information, education records or financial nonpublic personal information carries those regimes until de-identification is established under their own standards [7]. Compare the HIPAA routes in Safe Harbor vs Expert Determination.
- Provenance gaps. Without the generator version, training split, seed data and evaluation report, you cannot defend a status claim later. Keep the records described in provenance records for synthetic training data.
Clauses and questions for counsel to put to a supplier
Ask any supplier of synthetic data generated from personal data for evidence, not adjectives. The request below keeps the conversation concrete.
Illustrative example: invented to show structure; it does not describe an available dataset.
Synthetic data privacy disclosure request
- Source: record types, populations, jurisdictions, volume band
- Source de-identification before generation: method, residual identifiers
- Generator: family (CTGAN, TVAE, Bayesian network, fine-tuned LLM), version
- DP: yes/no; epsilon, delta, accountant; preprocessing inside budget?
- Holdout: size and how it was chosen
- Results: DCR distributions (train vs holdout), exact-match count,
MIA TPR at 0.1% and 1% FPR, canary exposure, outlier survival
- Status opinion: anonymous / pseudonymous / personal; who signed it
- Re-assessment trigger: regeneration, new auxiliary data, annual
A supplier that cannot answer the holdout and membership inference lines has not tested privacy. It has only measured resemblance. The terms "synthetic data" and its variants are defined in the synthetic data glossary entry. For how synthetic sets compare with licensed and scraped sources, see licensed vs synthetic vs scraped data.
When licensed real records are the cleaner path
Sometimes the simpler answer is to license de-identified real records under clear terms instead of synthesizing from them. If every synthetic output must be treated as personal data anyway, the generation step adds risk without removing obligations. Teams that need operational data for training or evaluation can describe the records they need to SourceX. SourceX sources operational datasets from US companies on request; it holds no stock, and a request does not guarantee a match. Each dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Personal details such as names, emails, phone numbers and account numbers are removed or replaced before delivery. The method is recorded and a sample is checked, but no method is perfect. Health records require HIPAA de-identification through Safe Harbor or Expert Determination. The wider privacy cluster is at the privacy, de-identification and sensitive data hub, and the full topic map is at the AI data hub.
Source licensed records for your synthetic data program
SourceX finds US businesses that hold the operational data you describe, from support histories to engineering and finance records. It assesses data and licensing permissions and agrees pricing and allowed uses in a license; nothing is contracted until the supplier agrees. Tell SourceX what your team needs.
Sources
- European Data Protection Board, "Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models" (2024). https://www.edpb.europa.eu/system/files/2024-12/edpb_opinion_202428_ai-models_en.pdf
- European Parliament and Council of the European Union (Official Journal of the EU, via EUR-Lex), "Regulation (EU) 2016/679 (General Data Protection Regulation), Recital 26" (2016). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng
- USENIX Security Symposium (Stadler, Oprisanu, Troncoso), "Synthetic Data – Anonymisation Groundhog Day" (2022). https://usenix.org/conference/usenixsecurity22/presentation/stadler
- arXiv, "The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (arXiv:2410.22971)" (2024). https://arxiv.org/html/2410.22971v1
- Information Commissioner's Office (UK), "How do we ensure anonymisation is effective?". https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/how-do-we-ensure-anonymisation-is-effective/
- National Institute of Standards and Technology, "De-Identifying Government Datasets: Techniques and Governance (NIST SP 800-188)" (2023). https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-188.pdf
- U.S. Department of Health and Human Services, Office for Civil Rights, "Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the HIPAA Privacy Rule" (2012). https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification
- arXiv, "Towards Principled Assessment of Tabular Data Synthesis Algorithms (arXiv:2402.06806)" (2024). https://arxiv.org/html/2402.06806v1
- DataCebo (Synthetic Data Vault), "SDMetrics documentation". https://docs.sdv.dev/sdmetrics
Tell us what your models need
Share scope, volume, language, format, timing and licensing requirements.