Searching past studies with AI: knowledge management for insight agencies

What does knowledge management mean for an insight agency?#

Knowledge management for an insight agency means a researcher can quickly find how the firm handled a similar audience, method or category before, and reuse the parts the firm owns. The usual problem is not too little material. It is years of studies spread across SharePoint, Google Drive, survey platforms, transcription tools and personal folders, named by job number and client code.

AI search raises both the payoff and the stakes. A question such as how did we screen for hospital pharmacists can now pull a screener, the incidence notes and the debrief into one answer. The same tool, set up loosely, can surface a brand tracker's results to a team pitching that brand's closest competitor.

So the work comes in four steps: tag studies consistently, separate client-confidential material from firm knowledge, connect search that respects permissions, and enforce retention. The order matters, because search built on untagged folders inherits every access mistake already in them.

Which study records are worth indexing?#

The study records worth indexing are the working documents that show how a project was designed and run, because those are what the next team reuses. Final reports matter to the account team, but they are the most client-specific part of the job and the least reusable outside it.

Field notes and post-project reviews are the most neglected. They record why a quota closed late, which sample source failed, and what the team would do differently, which is exactly what a new project manager wants to know.

Step 1: tag every study the same way#

Consistent tags let search filter on what researchers actually ask about: audience, method, category and time. Free-text folder names do not, and a model guessing from file contents can mistake a study about lapsed buyers of a brand for a study commissioned by that brand.

Keep the tag set short enough that project managers complete it at close-out. Apply it to the study as a whole and let every file inherit it, so a transcript carries the same confidentiality tier as the deck it fed.

• 1Study ID and an internal client code, not the client's name in plain text.
• 2Category and audience, such as IT decision-makers, caregivers or quick-service diners.
• 3Method: in-depth interviews, online community, conjoint, segmentation, tracker wave or usability.
• 4Field dates and markets covered.
• 5Sample source and recruitment route.
• 6Confidentiality tier, plus any client-specific restriction from the contract.
• 7Retention date for respondent data and a separate one for the project file.
• 8A named study owner who can answer questions about it.

Step 2: separate client-confidential results from firm knowledge#

Client-confidential material is anything that reveals a client's identity, strategy, data or findings; firm knowledge is how the agency does research. The line usually follows the contract, and many master services agreements treat findings, data and deliverables as the client's confidential information, so read each one rather than assuming.

In practice that means three tiers rather than one shared library. Anyone on the research staff can search firm methods. Client results stay visible to the people staffed on that account. Respondent-level material is restricted further, because privacy notices and consent language decide who may handle it and for how long.

Step 3: connect AI search that respects permissions#

Permissioned AI search answers a question using only the documents the person asking may already open, and shows which files each answer came from. That single property decides whether an AI layer is safe to put over a study archive.

Ask any vendor or internal team how permissions are enforced: checked live against the source system at query time, or copied once into the index and left to drift. Ask whether the vendor may use your content to train its own models, and read the terms rather than the sales deck. Survey platforms, transcription services and meeting note takers each carry their own terms as well.

Agencies that serve two brands in one category often promise separate teams in their contracts. A shared index that ignores those walls can break the promise without anyone noticing, so test for it before launch.

• 1Index masked versions of screeners and questionnaires, not client-branded originals.
• 2Keep raw data files and recordings out of the general index entirely.
• 3Require citations so researchers open the source before reusing anything.
• 4Log queries against client-confidential tiers so access can be reviewed.
• 5Test with hostile questions, such as asking for a named competitor's results.

Step 4: set retention so the archive shrinks as well as grows#

Retention for a study archive runs on two clocks: respondent personal data, deleted on the schedule set by privacy notices and contracts, and firm knowledge, kept while it stays useful. Mixing them is how agencies end up holding recordings long after panelists were told they would be deleted.

Write the retention date into the study tag at close-out, and make the search index drop files when the source is deleted. Client contracts may also require return or destruction of client materials when an engagement ends, so check those clauses before deciding what the archive keeps.

Industry codes of conduct that many agencies follow also set expectations for how respondent data is protected and used. Check the code your firm has adopted, and record which one applies in your retention policy.

Illustrative: a mid-size agency rebuilds its study library#

Illustrative: a fictional healthcare and B2B insight agency keeps studies on SharePoint, survey data in its survey platform and interview recordings in a transcription tool. Researchers keep rewriting physician screeners from scratch because nobody can find the old ones, and two account teams serve competing device makers.

The COO starts with close-out tags on current projects, then back-tags studies from recent years. Screeners and discussion guides are masked and moved into a firm knowledge library. Decks and tables stay in client folders with access limited to account teams. Recordings past their retention date are deleted, and the transcription tool is set to purge on the same schedule.

Only then does the agency switch on AI search, scoped to the firm library plus each user's own client folders. A researcher who asks how the firm recruited a rare specialty now gets three masked screeners with incidence notes, each linked to its source, and nothing from the competing account.

How SourceX approaches research archives#

SourceX sees the same separation from the licensing side. When an agency asks whether any of its records could be licensed for AI development, the starting point is the firm knowledge tier, such as masked instruments, method notes and project workflow records, not client findings or respondent data.

Any review follows the SourceX five-step transaction of Supply, Rights, Preparation, Approval and Delivery, beginning with metadata only. The Rights step checks client contracts and respondent consent language study by study, and a SourceX Evidence Packet records provenance, permitted use and the privacy record for whatever the agency approves.