What Happens After the AI Works?

For the past few years, the conversation around AI has focused on the technology. Which model is best. Which tools to use. How fast everything is changing. But once you start building with it, a different challenge emerges. The technology is often the easy part. The hard part is everything else. The definitions that don't match. The documentation nobody trusts. The tribal knowledge living in someone's head. The processes that work only because a few key people know how to navigate around the mess. Business intelligence exposed some of these problems years ago. AI is exposing even more of them.

For years, the people who cared about semantic models were mostly talking to each other. Everyone else had a simpler view: the dashboards worked, the BI nerds were overcomplicating things, and if a slightly different version of yesterday's question showed up, someone could always write more SQL. That worked well enough until AI agents became the ones asking the questions. Agents don't wait two weeks for a developer. They improvise. And the improvisation is different every time. That's the moment the semantic model stopped being a nice-to-have and started looking a lot more like a requirement. Every data quality problem that used to come home to roost the first time you built a dashboard is back, only now the list is longer. AI cares about policies, institutional knowledge, organizational context, and all the things that used to live quietly in people's heads. The one-version-of-the-truth problem just got a much bigger job description.

Along the way, Rob and Justin compare notes from the front lines of building with AI, from multi-agent systems and knowledge management to the unexpected ways these tools behave once they leave the lab and meet real organizations. There's a book update in here too. Fair Game is officially available for pre-order, and Rob shares why the independent bookstore route matters more than most people realize. If you've been wondering what happens after the AI works, this episode is a pretty good place to start.

Also in this episode:

Pre-order Fair Game: Customizing AI to Your Business Is Easier Than You Think

Fortune: Big Tech is laying off developers. My company just hired its first. We're both right about AI (By Rob Collie)