Token Maxxing and Local Inference: Navigating the Shift in AI Usage for Developers

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss token spending and the "token maxing" backlash, sparked by a GitHub Copilot budget issue and broader industry moves to optimize tokens. The hosts argues companies should focus on value and learning rather than premature token optimization, while still adding circuit breakers to prevent runaway costs. They explore shifting to local inference and open-weight/open-source models for cost, privacy, and control, citing concerns about vendor policy changes like Fable's retention and classifier behavior. The conversation expands into how AI shifts bottlenecks from coding to product management, experiments pairing product and engineers using SpecKit, and how AI may realize Agile's iterative promise. They close on distribution as a key frontier and the career opportunity of being early in AI, likening it to the early internet era. 00:00 Cold Open 00:27 Podcast Kickoff 00:46 Anniversary Trip in Maui 02:54 Copilot Budget Surprise 05:15 Token Panic Backlash 07:54 Value Over Tokens 13:33 Local Models Rising 15:46 SaaS Apocalypse Talk 17:29 Cycle Time vs Value 20:19 SpecKit Pairing Experiment 22:43 AI Meets Agile Promise 24:50 Spec First Agile Debate 29:40 Builders VS Managers 31:40 Scaffolding Vibe Coders 32:29 Distribution Is Hard 35:51 Snoring App Niche 38:43 Targeted Ads Reality 40:07 Open Source Pushback 42:11 Three Pressures On CTOs 44:56 Privacy And Data Control 53:38 Frontier Models Trust Issues 56:19 Three Truths For Teams 58:09 Early Internet Lessons 59:44 Wrap Up And Credits

Resources:

How Many CTOs Pod: https://howmanyctospod.com
Scott Porad: https://www.linkedin.com/in/scottporad/
Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/

#TechPodcast #EngineeringPodcast #DevTalks #PodcastForDevs #HowManyCTOs #Podcast #CTOs #CTOPodcast #ChiefTechnologyOfficer #Technology #Engineering #SoftwareDevelopment #SoftwareEngineering #TechLeadership #EngineeringLeadership #EngineeringCulture #TechDebates #Entrepreneurship #TokenMaxxing #LocalInference #AI #AIProgramming #AIAssistedProgramming Agile #Innovation #OpenSource #Privacy #OpenWeight #AIRevolution #Leadership #Innovation #AgileDevelopment

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(94)

What Broke in Tech: Code Quality, AI Guardrails, and the Evolving Software Engineer

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub continue to recap Gergely Orosz's "State of the Tech Industry" talk and focus on what "broke" and "what'...

3 Nov 36min

The State of Tech in 2026: What Changed And What Didn't

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss a Pragmatic Engineer talk on the state of the tech industry in 2026, focusing on what changed an...

27 Okt 32min

The Human Needs in Augmented Coding: A Conversation with Kent Beck

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub interview software pioneer Kent Beck about his background (JUnit, test-driven development, extreme progr...

20 Okt 1h 9min

Preparing for the Worst: Tabletop Exercises, Unit Tests, and Everything In Between

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss TaskRabbit's overnight AWS region migration from us-west-1 to us-west-2, describing the months o...

13 Okt 44min

Understanding The Fear: Adaptive Leadership in the AI Age

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss growing fear about AI among engineers and managers, framing it as an "adaptive moment" requiring...

6 Okt 34min

Simplifying Tech Through Empathy and Understanding with Frank Bravo

In this episode of "How Many CTOs Does It Take?" podcast, host Brad Hefta-Gaub is joined by guest Frank Bravo, a Bay Area technologist who spent 30 years as CTO in local media, helping put a newspaper...

29 Sep 50min

Managing Skepticism: Balancing Tradition and Technology with AI

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss how AI is changing software engineering referencing Kent Beck's idea that AI makes much existing...

22 Sep 49min

From Academia to Industry: Scott Hanson on the Future of AI and Edge Computing

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub interview Scott Hanson, founder and CTO of Ambiq, a company focused on ultra-low-power chips for "intell...

15 Sep 45min