Why AI pilots in portfolio companies stall, and the data reason behind it

Why do AI pilots in portfolio companies stall?#

AI pilots in portfolio companies stall most often at the point where a working demo meets the company's real records. The demo was built on a handful of clean examples chosen by an enthusiastic team; production has to handle every ticket, job or order the company has, including the ones with no resolution, the duplicates and the free-text notes nobody standardized.

Operating partners tend to read the symptom as a vendor problem, a change management problem or a budget problem. Those are real, but underneath many stalled pilots sits a records problem: the history the model needs either does not exist in usable form or cannot be used.

Published surveys on pilot failure use different definitions and samples, so a single headline rate tells you less than a direct look at the records the pilot depends on.

Five stall causes and the missing record behind each#

Each common stall cause maps to a property the records lack. Diagnosing which one applies tells you whether to fix capture, narrow scope or choose another use case.

Why demos succeed and production fails#

Demos succeed because they are curated. Someone picks examples where the notes are complete and the answer is obvious, and the model performs well on them. Nobody is being dishonest; the selection happens naturally when a team wants to show progress.

Production fails because the archive was never built for a model. Field technicians write in shorthand. A past helpdesk migration moved ticket bodies but not internal notes. Many closed opportunities carry no loss reason. Branches that joined through add-on acquisitions still use their own job codes. These gaps stay invisible until the pilot tries to use every record.

How to test the records before funding a pilot#

A records test takes a small random sample of closed cases from the system the pilot depends on and checks five properties. An analyst and a process owner can run it without moving data out of the company.

Read the results as a routing decision. If the sample fails on outcomes or links, fix capture first. If it fails on rights, narrow scope. If it fails on volume, pick a different use case. If it passes, fund the pilot with more confidence.

• 1Pull a random sample of closed records from across the full date range, not just the latest month.
• 2Check each record for an outcome: resolution, approval, rejection, win, loss or fix.
• 3Follow each from first contact to outcome and note where the trail breaks between systems.
• 4Ask who controls the records and whether customer, vendor or employee terms restrict use.
• 5Count records for the case types the pilot targets, and note mixed languages or local codes.

Which pilots tend to survive#

Pilots survive when they target a workflow whose records already carry outcomes and links. That often means choosing a less exciting use case with better history over a headline use case with weak records.

Notice that each surviving pilot depends on a decision someone recorded: an accepted answer, a callback, a won quote, a coded disposition, a merged change. Where people make those decisions but nobody writes them down, the pilot has nothing to learn from.

What operating partners can fix at portfolio level#

Operating partners can fix the capture problems that stall pilots across several companies at once, because the gaps repeat. Most companies lack the same few fields: a required resolution or outcome code, a reason for change orders or lost deals, and a shared ID that links a request to the work and the invoice.

Write those fields into the portfolio's system standards, so every new add-on and every system migration adds them by default. Ask each company to report how often its closed records carry an outcome, alongside its usual operating metrics. Capture fixed this way improves reporting immediately and gives the next pilot a better chance of reaching daily use.

Illustrative: three pilots in one fund#

Illustrative: a fictional lower-middle-market fund launches AI pilots at three companies at once: a 3PL, a roofing contractor and a B2B software company. All three demos impress the investment committee.

The 3PL's exception-triage pilot stalls first. Exception types are logged in the WMS when a case opens, but resolutions live in email threads with carriers, so the model cannot learn what fixed what. The roofing contractor's estimate-review pilot stalls because many jobs lack a recorded reason for change orders. The software company's support pilot reaches daily use: its Zendesk tickets carry resolution codes and Jira links.

The fund pauses new pilots at the two stalled companies until outcome capture is fixed, and adds a records test as a standard gate in its value creation plan.

The same records have a second use#

A records test also works as a first screen for licensing. AI developers look for similar properties when they license operational records. The SourceX Enterprise Data Value Framework names the drivers: uniqueness, domain expertise, human-generated signal, scale, recency, data cleanliness, rights and AI utility increase value; exclusivity increases price; reproducibility reduces value; preparation cost and privacy burden reduce net value.

A company whose pilot stalled on outcomes will not license well either until capture is fixed. A company whose pilot succeeded may hold records worth a metadata-only fit check, which shares no files.