AI Can Fix Your Broken Research Repository

This week, Paul and Marcus dig into why traditional user research repositories fail almost everyone in an organization, and how AI is quietly changing the game. There's also an App of the Month pick that's a little too on-the-nose, some pointed Google bashing, and a sheep-based punchline.

AI-Powered User Research Repositories

The pattern in most organizations is depressingly familiar: user research gets done, a PowerPoint gets presented to stakeholders, everyone nods along or ignores it entirely, and then the research disappears. It might prompt some short-term action, but the knowledge evaporates. Nobody references it again six months later.

The traditional solution has been to build a research repository: a central place to store everything from interviews and surveys to usability tests and diary studies. The problem is that these repositories almost always become what Paul generously describes as "dumping grounds." Dense folder structures, difficult navigation, and search tools that require you to already know what you're looking for make them practically unusable for anyone outside the UX team. And who ends up using them? Other UX professionals, the people who already understand the research anyway. Everyone else ignores them.

AI changes this in three meaningful ways.

First, it makes the initial build far less painful. You can throw everything at it, PDFs, old PowerPoints, interview transcripts, survey exports, and AI will structure and organize that material into something coherent. What used to be a daunting, months-long project becomes manageable.

Second, it makes the repository accessible to people who aren't UX specialists. Instead of requiring a precise search query, a conversational interface lets anyone ask vague, natural questions. A product manager can ask "what do our users think about the checkout process?" and get a synthesized answer drawn from five different studies they never knew existed. That's a genuinely different kind of value.

Third, and this is the part Paul finds most compelling, it can identify gaps in your research. When someone asks the repository a question and there's no relevant research to draw on, a well-configured AI won't fabricate an answer. It flags the gap and notifies the UX team that this is an area worth investigating. Over time, the questions people ask become a demand-driven research roadmap, shaped by what people in the organization actually need to know rather than what the UX team assumes they need.

Marcus pushed back on the reliability question, which is fair given AI's well-documented habit of confidently inventing things. Paul's response: proper setup matters enormously. You instruct the AI explicitly not to fabricate, you add a quality gate that checks answers before they're returned, and you can even have it verify claims against source material. Even with pessimistic assumptions, say one in ten answers being wrong, that's still more useful than having nothing at all. And the failure mode is reassuring: if the AI can't find relevant research, it defaults to generic best practice rather than making something specific up about your users.

Paul then connected this to something he's discussed before: AI-powered virtual personas. The repository feeds the persona generation. AI analyzes the accumulated research and builds queryable personas from it. Unlike static persona documents that go stale almost immediately, these update as new research is added. And here's the detail Paul is clearly delighted by: put a QR code on your printed persona posters. Scan it, and you're now having a conversation with a virtual version of that persona. Marcus had recently written about the value of physical personas on walls as simple reminders of who you're designing for, and this neatly bridges the physical and digital.

The upshot: organizations that invest in an AI-powered research repository end up with something that prevents duplicate research, makes user insights accessible to everyone, identifies gaps in what's known, and gives the whole organization a quick way to gut-check decisions against actual user data. The reason more organizations aren't doing this, Paul notes with characteristic subtlety, is that UX teams are too small and too busy. "Hire me to do it" being the conclusion he arrived at, live on air.

App of the Month Notion

Paul's pick this month is Notion, which he acknowledges he's almost certainly recommended before, given that he runs his entire business on it and describes its potential failure as roughly equivalent to his own. The recommendation here is specific though: Notion as the platform for building AI-powered user research repositories.

Two things make it well-suited for this. First, structural flexibility: you can organize a repository however your organization needs, and bring in almost any format of research artifact. Second, Notion has a powerful built-in AI agent that can reference, search, and synthesize across everything stored in it.

That said, Paul mentioned conversations with the RNLI, who use SharePoint and Copilot to achieve essentially the same thing. The principle works across platforms. Notion is Paul's preference, but he'd be the first to acknowledge the bias.

Interesting Reads "Google is quietly rewriting headlines with AI in search results"

Dan at Headscape surfaced this one. Google has been quietly rewriting the titles of content in its search results, not a new practice, but one that has apparently accelerated significantly with the arrival of Gemini. The example from the article: a piece originally titled "I used the cheat on everything AI tool, and it didn't help me cheat on anything" was shortened to "cheat on everything AI tool." The meaning flips completely. Paul's view: this isn't really an AI problem so much as a "no human in the loop" problem. Remove human judgment from the process and you get outcomes like this.

"Testing suggests Google's AI overviews tell millions of lies per hour"

This one prompted a longer and more genuinely interesting conversation. The article references New York Times analysis suggesting Google's AI overviews are incorrect around 10% of the time. The illustrative example: AI Overview cited three sources to answer a question about when Bob Marley's home became a museum. Two of the sources didn't address the date at all. The third, Wikipedia, listed two contradictory years, and AI confidently picked the wrong one.

Paul and Marcus ended up in partial agreement. Paul's argument: we don't hold websites to a higher standard of accuracy than we hold AI, and the expectation of AI infallibility is inconsistent. The real issue is the word "confidently." AI states things with a certainty it hasn't earned, and the interface doesn't adequately signal uncertainty. Marcus's counter: AI summaries have effectively removed the click-through step, so an error now goes unchecked in a way a traditional search result didn't. They concluded it's largely a user interface problem, acknowledged that Google isn't going to remove the feature, and briefly proposed a BBC-funded public search engine before moving on.

Marcus' Joke

I'm entering the annual Give Helium to a Sheep contest again, and I'm a bit nervous. Last year the bar was very high.

Find The Latest Show Notes