When a Supplier Won't Release Raw Samples: Secure Viewing, Redacted and Synthetic Stand-ins

Why cautious holders refuse raw samples

Holders refuse raw samples because a sample is a disclosure, and for support tickets, claims notes or patient-adjacent records it carries the same privacy and confidentiality exposure as a full delivery. A 500-row CSV attached to an email cannot be recalled, and the holder's privacy team often has no basis to approve it before a license exists. Health records add a regulatory layer: under HIPAA, protected health information must be de-identified through Safe Harbor or Expert Determination before it can leave as non-PHI [4]. That work is rarely done for a pre-sales sample.

The practical consequence is that you negotiate the sample route before you negotiate the sample. Start from the standard request playbook in how to request a training data sample, then narrow it with the routes below. SourceX's own overview of sharing samples with buyers covers the supplier side of the same conversation.

The four restricted routes and what each can prove

Each route answers a specific pre-purchase question, and none answers all of them. The table below is the decision tool most data partnerships leads need: pick the question first, then the route.

Illustrative example: invented to show structure; it does not describe an available dataset.

Most serious evaluations combine two routes: a view-only session to judge record quality, plus remote aggregates or a redacted sample to judge scale and coverage.

View-only rooms: qualitative review, not model experiments

A view-only room is the right route when the open question is "are these records any good," and the wrong one when it is "will this move my metric." Reviewers can read 50 to 200 real support threads or engineering tickets, check whether resolution codes match the text, and see whether timestamps, agent IDs and product fields are populated. They cannot run a fine-tuning job, compute embedding-space coverage or test a tokenizer, because nothing leaves the environment.

Plan the session like an audit. Bring a written rubric (label agreement, completeness, language mix, template-versus-free-text ratio, PII density) and score records live so the result survives the session. Evaluation terms in this setting typically forbid screenshots, transcription of records and any attempt to bypass the access controls; published sample-data evaluation licenses already restrict data to evaluation and prohibit circumventing technical protections [1]. Treat your notes as confidential information under the NDA, and see evaluation licenses and NDAs for dataset samples for the clauses to check.

Redacted samples: only useful if redaction matches delivery

A redacted sample is only evidence if it was produced by the same redaction method, configuration and replacement scheme the holder will apply to the licensed delivery. If the sample uses hand redaction and the delivery uses an automated NER pass, the sample tells you nothing about leakage rates, over-redaction or token-level artifacts in the real dataset. Ask for the method, tool and version in writing, and ask that the delivery specification reference them.

Check redaction quality the way your model will experience it. Look for consistent pseudonyms across a conversation thread (the same customer replaced by the same token), preserved formats for dates and amounts where they carry signal, and over-redaction that strips product names or error codes. Under-redaction is the other failure: a street name in a free-text note, an account number inside a pasted log line. For health data, confirm whether the holder used Safe Harbor's identifier list or an Expert Determination, because the two produce different field-level outcomes [4].

Synthetic stand-ins: schema and pipeline tests only

A synthetic stand-in shows you the shape of the data, not its substance, so use it for integration tests rather than value estimates. Fidelity tools such as SDMetrics compare synthetic data against the real table on column shapes and column-pair trends [2], and practitioner guidance frames synthetic quality across fidelity, utility and privacy, checking whether proportions, cardinalities and correlations survive [3]. A holder who can share that report gives you useful evidence about the generator, but a high aggregate score still says little about rare events, long-tail vocabulary or label noise, which is often what makes operational data valuable for SFT and evaluation.

Synthetic does not automatically mean safe. Generative models can memorize rare strings from their training data [5], and EDPB Opinion 28/2024 treats whether a model trained on personal data is anonymous as a question requiring assessment rather than an assumption [6]. If a holder offers an LLM-generated stand-in of support transcripts, ask whether it was tested for verbatim regurgitation of real records. If it was not, the stand-in may carry more exposure than a properly redacted sample.

Remote query and compute: aggregates and held-out evaluation

Remote access proves scale and, when compute is allowed, model signal, which no other restricted route can. At the simplest level the holder runs your queries and returns aggregates: record counts by month, label distribution, median ticket length, share of records with attachments. Sharing protocols such as Delta Sharing expose live tables over REST on Delta Lake and Parquet without copying them to the recipient's system [7], and warehouse-native sharing follows the same pattern; see zero-copy warehouse data sharing for delivery-side detail.

The stronger version is held-out evaluation inside the holder's environment: you submit a model or evaluation harness, the holder runs it on a slice of the real data, and only scores come back. This is how you estimate lift before buying, and it is covered in evaluating models on data you can't take and estimating a dataset's value before you buy. Agree on minimum cell sizes for aggregates and a review step for outputs, or the holder's privacy team will block it late. Full clean-room training is a separate access model, covered in data clean rooms for AI training and the short explainer what is a data clean room.

A request template for a restricted sample

The fastest way to get a cautious holder to yes is to ask for a specific route, a specific question and specific controls in one message. Use this structure in your first request.

Illustrative example: invented to show structure; it does not describe an available dataset.

Restricted sample request
Dataset described: B2B SaaS support tickets, 2022-2025, English, with resolution codes
Decision this sample must support: whether resolution-code labels are reliable enough for SFT
Route requested: (1) 2-hour supervised view-only session, 150 records chosen by stratified
  random draw across product lines; (2) remote aggregates
Aggregates requested: tickets per month; resolution code distribution; % with agent notes;
  median turns per ticket; language mix; null rate per field
Minimum cell size accepted: 20
Redaction: same method and configuration as planned delivery; method named in writing
Our controls: named reviewers only; no screenshots or export; notes held as confidential
  information; deleted on request
Out of scope for this stage: model training, embedding computation, any copy of records

Keep the "decision this sample must support" line. It tells the holder why you need each element and makes a narrower sample defensible to their privacy reviewers.

Scoring evidence from mixed routes

Score restricted-route evidence against the same criteria you would apply to a full sample, and mark each score with the route that produced it. ISO/IEC 5259-2 defines data quality measures for ML datasets [9] that make a useful shared vocabulary with the holder: completeness, accuracy, consistency, representativeness. A completeness score from remote aggregates is strong evidence; a representativeness score from 150 hand-picked records is weak evidence.

Record the gaps explicitly. If you never saw model lift, say so in the approval memo and carry it into the license as acceptance criteria or remedies; see remedies when a data delivery fails. Feed the scores into the data vendor evaluation scorecard and the wider AI training data procurement hub. Gated-access patterns from public model hubs, where access requests collect reviewer identity before files are visible [8], are a reasonable template for logging who saw what.

How SourceX handles sensitive data requests

SourceX sources operational datasets from US companies on request, including support and sales histories, engineering records, documents and finance and legal workflows; nothing is held in stock, and a request does not guarantee a match. Every release is approved by the supplying company, every dataset is rights-reviewed for ownership and consents, and personal details such as names, emails, phones and account numbers are removed or replaced before delivery, with the method recorded and a sample checked; no method is perfect. Health records require HIPAA de-identification by Safe Harbor or Expert Determination. Delivery runs through private, access-controlled workflows only after an executed agreement and supplier approval. If you are planning a restricted evaluation, describe the data you need on the SourceX buyers page.

Evaluate a sensitive dataset without receiving a copy

SourceX finds US businesses that hold the data you describe, assesses data and licensing permissions, and agrees pricing and allowed uses in a license before anything transacts. Nothing is contracted until a supplier agrees. Describe the sensitive dataset you need to evaluate.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.