Token Maxxing and Local Inference: Navigating the Shift in AI Usage for Developers

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss token spending and the "token maxing" backlash, sparked by a GitHub Copilot budget issue and broader industry moves to optimize tokens. The hosts argues companies should focus on value and learning rather than premature token optimization, while still adding circuit breakers to prevent runaway costs. They explore shifting to local inference and open-weight/open-source models for cost, privacy, and control, citing concerns about vendor policy changes like Fable's retention and classifier behavior. The conversation expands into how AI shifts bottlenecks from coding to product management, experiments pairing product and engineers using SpecKit, and how AI may realize Agile's iterative promise. They close on distribution as a key frontier and the career opportunity of being early in AI, likening it to the early internet era. 00:00 Cold Open 00:27 Podcast Kickoff 00:46 Anniversary Trip in Maui 02:54 Copilot Budget Surprise 05:15 Token Panic Backlash 07:54 Value Over Tokens 13:33 Local Models Rising 15:46 SaaS Apocalypse Talk 17:29 Cycle Time vs Value 20:19 SpecKit Pairing Experiment 22:43 AI Meets Agile Promise 24:50 Spec First Agile Debate 29:40 Builders VS Managers 31:40 Scaffolding Vibe Coders 32:29 Distribution Is Hard 35:51 Snoring App Niche 38:43 Targeted Ads Reality 40:07 Open Source Pushback 42:11 Three Pressures On CTOs 44:56 Privacy And Data Control 53:38 Frontier Models Trust Issues 56:19 Three Truths For Teams 58:09 Early Internet Lessons 59:44 Wrap Up And Credits

Resources:

How Many CTOs Pod: https://howmanyctospod.com
Scott Porad: https://www.linkedin.com/in/scottporad/
Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/

#TechPodcast #EngineeringPodcast #DevTalks #PodcastForDevs #HowManyCTOs #Podcast #CTOs #CTOPodcast #ChiefTechnologyOfficer #Technology #Engineering #SoftwareDevelopment #SoftwareEngineering #TechLeadership #EngineeringLeadership #EngineeringCulture #TechDebates #Entrepreneurship #TokenMaxxing #LocalInference #AI #AIProgramming #AIAssistedProgramming Agile #Innovation #OpenSource #Privacy #OpenWeight #AIRevolution #Leadership #Innovation #AgileDevelopment

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(85)

The Hardware/Software Blend: Visionary Innovations in Health Tech with Eyebot's Jack Moldave

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub interview Jack Moldave of Eyebot, a company he co-founded nearly six years ago to make getting glasses a...

1 Syys 52min

Fast, On Time, or Fully Utilized? Choosing What to Optimize in Engineering Delivery

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss a leadership conversation where three senior stakeholders wanted different outcomes from enginee...

25 Elo 16min

The Pace of Innovation: Systems Thinking for Faster Engineering

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub open with travel and sports talk, including World Cup rules, then pivot to Scott's change-management tra...

18 Elo 56min

Method to the Madness: A CTO Framework for Systemic AI Adoption

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss how CTOs must frame narratives that help teams organize around ideas, inspired by how Scott expl...

11 Elo 45min

There's A Pattern To Follow: An Interview with Robert "Uncle Bob" Martin

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub are honored to interview Robert "Uncle Bob" Martin, who recounts his 60+ year programming journey from a...

4 Elo 1h 1min

Is Uncle Bob Right? Reading Code vs. Architecting the Future

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss how Robert "Uncle Bob" Martin—author of Clean Code and Agile Manifesto signer—sparked online con...

28 Heinä 43min

This Has Always Been A Problem: Slot Machine Devs and Growing Junior Engineers in the Age of AI

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss how managing armies of coding agents can be physically draining, likening it to context switchin...

14 Heinä 56min