He Built an AI Audience Simulator. It’s the Future of Customer Research. - Ep. 49 with Michael Taylor

Michael Taylor has perfected the art of getting AI to speak in tongues. He’s taught it to mimic the voices of your customers—so you can see how they would respond before you ship. Michael is the creator of Rally, a market research tool that lets you simulate an audience of AI personas. He built a simulator that lets us A/B test Every’s headlines on an audience that mimics the real Hacker News audience. It’s become a part of my writing workflow, and I love it because you test your assumptions quickly, cheaply, and without any of the risks of putting something out into the world. Besides Rally, Michael co-authored a book on prompt engineering for O’Reilly, and he writes a column for Every about managing AI tools like you would people. In a past life, he founded a growth marketing agency which he grew to 50 people and sold in 2020. One of the reasons I’m drawn to Michael’s work is because he has a tinkerer’s mindset. He’s always exploring the limits of what a new technology can do, and what he’s into today, everyone else will likely discover six months later. We spent an hour talking about using language models to judge your work, best practices for assessing an AI’s performance, and Michael’s flow inside Cursor. He also demos Rally live on the show, testing three different potential headlines for an Every article. If you found this episode interesting, please like, subscribe, comment, and share! Want even more? Sign up for Every to unlock our ultimate guide to prompting ChatGPT here: https://every.ck.page/ultimate-guide-to-prompting-chatgpt. It’s usually only for paying subscribers, but you can get it here for free. To hear more from Dan Shipper: Subscribe to Every: https://every.to/subscribe Follow him on X: https://twitter.com/danshipper Timestamps: Introduction: 00:01:32 AI can simulate human personalities with remarkable precision: 00:04:30 How Michael simulated a Hacker News audience: 00:08:15 Push AI to be a good judge of your work: 00:15:04 Best practices to run evals: 00:19:00 How AI compresses years of learning into shorter feedback loops: 00:23:01 Why prompt engineering is becoming increasingly important: 00:27:01 Adopting a new technology is about risk appetite: 00:44:49 Michael demos Rally, his market research tool: 00:47:10 The AI tools Michael uses to ship new features: 00:54:53 Links to resources mentioned in the episode: Michael Taylor: @hammer_mt Join the waitlist for Rally, Michael’s synthetic market research tool: https://askrally.com/ The book Michael co-authored on prompt engineering: Prompt Engineering for Generative AI The column Michael writes for Every: Also True for Humans Michael’s article on personas of thought: "I Asked 100 AI Agents to Judge an Advertisement” Michael’s article on building a Hacker News simulator: "I Created a Hacker News Simulator to Reverse-engineer Virality”