JP Beeghly & Josh Maynard on AI quality: Train agents like junior employees, not magic tools

When JP Beeghly, Senior Manager of Martech at Sonos, asks Chord's Commerce Copilot more questions than anyone else in their customer base, it's not because he's looking for basic answers. He's stress-testing whether AI can handle the institutional knowledge that separates a generic query from a Sonos-specific insight. Joined by Josh Maynard, who recently shifted from CTO at Ruggable to GM, Global eCommerce, MrBeast, this conversation cuts through the AI hype to reveal what's actually working at scale.

Both openly admit their organizations aren't doing enough to train AI properly. And that admission leads to the real conversation: how commerce operators are navigating the gap between AI's promise and the messy reality of implementation.

Topics discussed:

The context problem in enterprise AI. JP reveals how Chord's Copilot must reach across multiple data tables to answer seemingly simple campaign questions, highlighting why institutional knowledge and business-specific context matter more than raw data access. Josh emphasizes that without centralized, structured data, teams uploading different Excel files to ChatGPT will generate contradictory answers.

Why dimensional modeling isn't dead. Despite initial hopes that LLMs could handle unstructured data, the conversation confirms that well-structured data architecture is now more critical than ever. LLMs need to write reliable SQL, which requires data models built specifically to support AI query patterns, not just traditional BI dashboards.

Onboarding AI like you'd onboard junior employees. Rather than expecting immediate production-ready output, both leaders discuss treating AI tools as new hires who need access to systems, training on company-specific terminology, and gradual expansion of responsibilities. The parallel: you wouldn't give a junior employee your most complex task on day one.

The quality control gap. Josh compares AI-generated content to junior engineers copying from Stack Overflow without understanding context. The solution isn't banning these tools but implementing review processes and teaching teams to edit for accuracy, brand voice, and business fit rather than accepting first drafts.

AI as augmentation, not automation. Josh can now draft go-to-market strategies in minutes instead of days, but that speed creates new expectations. The conversation explores how AI compresses timelines for iteration while raising the bar for output quality, forcing leaders to rethink what "good enough" means.

The shift from memorization to ideation. Where previous generations valued information retrieval speed (think: pre-Google library research), and then finding answers quickly (Google era), the new competitive advantage is rapid ideation and iteration. AI tools enable this, but only for people who maintain curiosity and critical thinking.

Brand as fundamentally human territory. JP explains why Sonos can't fully automate customer experience decisions: brands are human by definition, built on emotional connections and nuanced understanding that AI can assist with but never own. The technology helps test variations faster, but brand strategy remains firmly in human hands.

Social skills as the underrated career differentiator. JP's advice for his teenagers extends to junior operators: putting down phones and developing face-to-face communication skills matters more than ever. The ability to present work, challenge assumptions, and navigate tough conversations with managers determines growth more than technical prowess alone.

The non-deterministic problem in business intelligence. When OpenAI's Chief Scientist acknowledges that models won't give the same answer twice, it creates trust issues for business-critical questions. The solution emerging: training AI with approved SQL queries and business rules as context, not expecting it to derive correct answers independently.