How much agent training data do you need? Sizing trajectories, tasks and variants

Why trajectory count alone misleads

A trajectory count tells you how many episodes you bought, not how many distinct situations the agent learned from. Ten thousand ticket-resolution episodes that all follow the "password reset, verify identity, close" path teach one behavior ten thousand times. Two thousand episodes spread across forty well-defined variants usually teach more, because agent failures cluster at branches: a missing field, a policy exception, a system timeout, a handoff to a human.

Treat this as a working hypothesis to test on your own data rather than a law. General scaling studies show error falling as a power law in training-set size [9], but they measure size within a fixed distribution. In agent data, the distribution itself is the purchase decision, so the first question is which cells of the task space you cover.

Three units matter when you budget:

• Task family: a goal type with a shared success definition, such as "issue a partial refund" or "reconcile a vendor invoice against a purchase order."
• Variant: a distinct path through that family, defined by system, exception branch, policy clause or input format.
• Trajectory: one recorded episode of observations, actions and an outcome label for one variant instance.

For how to define these units in a request, see writing an agent data specification.

What published agent studies actually used

Published agent studies vary widely in how much human data they use, and the small ones lean heavily on augmentation. PC Agent-E started from 312 human-annotated computer-use trajectories and used a frontier model to add alternative actions at each step, a method the authors call Trajectory Boost [1]. That result says a few hundred high-quality human demonstrations can anchor a computer-use agent when you have a strong model to expand them; it does not say 312 is enough without that step.

Synthetic pipelines push in the other direction. Synatra converts indirect knowledge such as tutorials into direct web-agent demonstrations and reports a synthetic demonstration costs roughly 3% of a human-annotated one [2]. OS-Genesis explores GUIs first and synthesizes tasks backward from the interactions it observes [3]. Explorer reports about $0.28 per successful synthetic web trajectory against about $0.55 for AgentTrek [11]. These figures are research costs, not market prices.

Benchmark construction shows how task diversity is usually counted. Mind2Web collected over 2,000 open-ended tasks from 137 real websites across 31 domains [4]; OSWorld uses 369 execution-evaluated tasks in real desktop and web applications [5]. Both count tasks and environments first, which is the right instinct for a buyer. The supervised-fine-tuning literature points the same way: LIMA fine-tuned a 65B model on 1,000 curated examples and noted that curating such examples is labor-intensive [8].

The practical reading: real enterprise trajectories are scarce and expensive, so buy them where synthetic data is weakest, which is in the exception paths, real system quirks and policy judgments that a generator cannot invent. The tradeoffs are compared in licensed records, commissioned demonstrations or synthetic trajectories.

Sizing by purpose: SFT, RL warm-start and evaluation

Each use of agent data has a different sizing driver, so budget them as separate line items. Mixing them, for example training on episodes you later use for evaluation, is a common way an agent data purchase loses its value.

For SFT, the unit you budget is trajectories per variant. For RL warm-start, the binding constraint is often not trajectories but replayable environments; see seed data and state snapshots for agent sandboxes. For reward models, outcome labels need care: business systems record that a case closed, not whether it was resolved correctly, which is covered in task success labels for agent trajectories.

Sizing the evaluation set for statistical power

An evaluation set needs enough distinct tasks that the difference you care about is larger than the noise. A task-success rate is a binomial proportion, so its standard error is roughly the square root of p(1-p)/n. At a 50% success rate, 100 tasks give a 95% interval of about plus or minus 10 percentage points, 400 tasks about plus or minus 5, and 1,000 tasks about plus or minus 3.

Two corrections matter for agents. First, normal-approximation intervals are unreliable below a few hundred items, so use Wilson score or Bayesian intervals on small sets [7]. Second, rerunning the same task several times does not multiply n: reruns are correlated, so effective sample size sits closer to the number of distinct tasks. Repeated trials are still valuable for a different question, reliability, which tau-bench measures as the probability of succeeding on all k attempts at the same task (pass^k) [6].

Budget evaluation per task family, not only in aggregate. If you will ship or block an agent per workflow, each workflow needs its own powered sample. Also budget for audit: even widely used benchmarks carry label errors, estimated at an average of at least 3.3% across ten test sets [10], so plan a second-annotator pass on a sample of evaluation outcomes. More detail is in how many examples an eval set needs, and service-agent specifics are in policy-following evaluation for customer support agents.

A worked sizing sheet for a domain agent

A sizing sheet turns the task inventory into a purchase quantity you can defend to finance. The example below assumes a back-office agent for accounts-payable exceptions.

Illustrative example: invented to show structure; it does not describe an available dataset.

Read it as a budget structure, not a recommendation. If cell C is much larger than you expected, the right move is usually to cut scope to the families that matter for launch, not to thin trajectories per cell. Put the inputs into the data volume estimator and adjust per family.

Run a learning-curve pilot before scaling

A learning-curve pilot tells you whether the next dollar should buy more trajectories, more variants or better labels. Train on nested subsets of the pilot, for example 25%, 50% and 100% of trajectories, and plot held-out success per task family. Fit the curve per family rather than in aggregate, since a power-law trend [9] can hold overall while individual families have already flattened.

Interpret the curve with three questions:

1. Still rising at 100%? Buy more trajectories for those cells.
2. Flat, but low? More of the same will not help. Buy new variants, fix the observation format (see computer-use trajectory data), or check outcome labels.
3. Flat and high? Stop buying for that family and move budget to weaker ones.

Write the pilot into the deal structure. Ask suppliers whether a smaller first tranche can be followed by targeted top-ups for specific variants, and specify the variant fields (system, exception code, policy clause) you will use to request them. Ongoing supply of the same record types matters more for agents than for static datasets, because workflows and systems change.

Data quality checks that change the count

Quality issues reduce your effective trajectory count, so check them before you finalize volume. A trajectory with missing intermediate steps, collapsed retries or an outcome inferred after the fact may be unusable for SFT even if it is fine for evaluation.

• Step completeness: compare logged actions with field-level audit trails; see reconstructing agent actions from audit trails.
• Variant tagging: confirm each episode can be assigned to a cell without manual reading.
• Outcome reliability: sample outcomes for second review, given baseline label error rates [10].
• Deduplication: templated workflows produce near-identical episodes; dedupe before counting.
• Leakage: keep evaluation tasks from a separate period, team or system so near-duplicates do not inflate scores.
• Redaction: names, emails, phone numbers and account numbers should be removed or replaced before delivery, with the method recorded; verify a sample yourself.

Ticket and case histories are a common real-world source for these episodes; reconstructing trajectories from ticket histories covers what survives conversion.

How SourceX approaches agent trajectory requests

SourceX sources operational datasets from US companies, including support histories, engineering records, finance and legal workflows, and new recordings of hands-on work, and manages licensing and ongoing purchases. Data is sourced on request, not held in stock, so a sizing sheet like the one above is the most useful input: describe the task families, variants and evaluation needs, not the businesses that might hold them. Every dataset is rights-reviewed and delivered under a license that defines records, uses, term and delivery. For the broader category, see AI agent training data, workflow task histories, the general question of how many records AI labs want, and the buyer intake.

Buying agent trajectories sized to your task variants

If your sizing sheet shows which task families and variants you need, SourceX can look for US businesses that hold matching workflow records and handle the assessment, licensing and transaction; nothing is contracted until a supplier agrees, and a request does not guarantee a match. Personal details are removed or replaced before delivery, and delivery runs through private, access-controlled workflows after an executed agreement. Describe the agent trajectories you need.