From Specs to Monitoring: A CTO's Guide to QA Best Practices

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub tackle the evolving landscape of quality assurance in web apps.

They challenge the notion of QA as just "manual testers at the end," instead framing quality as a chain of practices across the entire SDLC: requirements, technical design, code reviews, unit tests, automation, monitoring, and stakeholder demos.

Topics include:

• Why QA begins upstream with clear requirements and design, not just testing at the end

• The role of test automation and real user monitoring (RUM) in catching defects before users do

• Code review culture as a cornerstone of quality (and why reviewers should run the code)

• Balancing QA investment for startups vs. enterprise-grade systems

• A proposed ROI heuristic for test automation (risk, cost of failure, complexity, churn, compliance)

• The role of AI in QA: pair code reviews, vibe coding risks, and the future of end-to-end automation

• How context-rich prompts (and even teaching AI like you'd teach a junior dev) improve code quality

• Why quality assurance is ultimately a cultural mindset, not just a checklist

The episode closes with an anecdote on peanut butter & jelly sandwiches, illustrating how precise instructions (and context) are at the heart of both QA and AI development.

• 00:00 Introduction to User Interaction Testing in AI
• 00:29 Welcome to the Podcast
• 00:42 Discussing QA Standards in Web Apps
• 01:22 Philosophical Approach to Quality Assurance
• 03:19 Importance of Test Automation and Monitoring
• 06:15 Real User Monitoring and Production Safety Nets
• 08:32 Challenges with Requirements and Product Debt
• 10:10 Sprint Demos and Stakeholder Reviews
• 15:33 Code Reviews and Implementation Plans
• 16:59 AI in Code Reviews and Pair Programming
• 18:39 Balancing Investment in QA for Startups
• 20:56 Evaluating Risk and Investment in Test Automation
• 22:47 The Importance of Code Reviews and Unit Tests
• 26:23 Challenges and Strategies in AI-Driven Development
• 32:48 The Role of Context in AI and QA
• 36:20 Concluding Thoughts and Anecdotes

Resources:

• How Many CTOs Pod: https://howmanyctospod.com
• Scott Porad: https://www.linkedin.com/in/scottporad/
• Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/

#TechPodcast #EngineeringPodcast #DevTalks #PodcastForDevs #HowManyCtos #Podcast #Ctos #CtoPodcast #Technology #Engineering #SoftwareDevelopment #TechLeadership #EngineeringLeadership #SoftwareQuality #QA #QualityAssurance #TestAutomation #CodeReviews #EngineeringCulture #AIinTesting