Tech Lead Journal10 Nov 2025

#238 - AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Why does an AI that brilliantly generates code suddenly fail at basic math? The answer explains why your LLM will fail when you least expect it.

In this episode, Emmanuel Maggiori, author of “Smart Until It’s Dumb” and “The AI Pocket Book,” cuts through the AI hype to reveal what LLMs actually do and, more importantly, what they can’t. Drawing from his experience building AI systems and witnessing multiple AI booms and busts, Emmanuel explains why machine learning works brilliantly until it makes mistakes no human would ever make.

He shares why businesses repeatedly fail at AI adoption, how hallucinations are baked into the technology, and what developers need to know about building reliable AI products.

Whether you’re implementing AI at work or concerned about your career, this conversation offers a grounded perspective on navigating the current AI wave without getting swept away by unrealistic promises.

Key topics discussed:

Why AI projects fail the same way repeatedly
How LLMs work and why they brilliantly fail
Why hallucinations can’t be fixed with better prompts
Why self-driving cars still need human operators
Adopting AI without falling into hype traps
How engineers stay relevant in the AI era
Why AGI predictions are mostly marketing
Building valuable products in boring industries

Timestamps:

(00:00:00) Trailer & Intro
(00:02:32) Career Turning Points
(00:06:41) Writing “Smart Until It’s Dumb” and “The AI Pocket Book”
(00:08:14) The History of AI Booms & Winters
(00:11:34) Why Generative AI Hype is Different Than the Past AI Waves
(00:13:26) AI is Smart Until It’s Dumb
(00:16:45) How LLM and Generative AI Actually Work
(00:22:53) What Makes LLMs Smart
(00:27:25) Foundational Model
(00:30:01) RAG and Agentic AI
(00:34:09) Tips on How to Adopt AI Within Companies
(00:37:56) How to Reduce & Avoid AI Hallucination Problem
(00:45:49) The Important Role of Benchmarks When Building AI Products
(00:50:57) Advice for Software Engineers to Deal With AI Concerns
(00:56:49) Advice for Junior Developers
(00:59:34) Vibe Coders and Prompt Engineers: New Jobs or Just Hype?
(01:01:55) The AGI Possibility
(01:07:23) Three Tech Lead Wisdom

_____

Emmanuel Maggiori’s Bio
Emmanuel Maggiori, PhD, is a software engineer and 10-year AI industry insider. He has developed AI for a variety of applications, from processing satellite images to packaging deals for holiday travelers. He is the author of the books Smart Until It’s Dumb, Siliconned, and The AI Pocket Book.

Follow Emmanuel:

LinkedIn – linkedin.com/in/emaggiori
Website – emaggiori.com

Like this episode?
Show notes & transcript: techleadjournal.dev/episodes/238.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Buy me a coffee or become a patron.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(282)

The IT/OT Handbook: From Silos to Sustainable Digitalization - David Ariens & Willem van Lammeren

Why can’t you “move fast and break things” in a factory? Because a small software bug doesn’t just crash an app, it can shut down a plant the size of a small city. David and Willem explain the hidden ...

28 Sep 1h 7min

Don't Outsource Your Thinking: How to Lead in the AI-Native Era - Emilie Schario

What’s the one rule that matters most when AI writes most of your code? Emilie says it comes down to a single line: don’t outsource your thinking.In this episode, Emilie Schario, co-founder of Kilo Co...

7 Sep 59min

The Father of AI Assistant on Why Joy Comes Before Success - Kevin Surace

If you’re not using AI five times an hour, you’re already behind, says the man who built the first AI assistant back in 1998. Kevin explains why that pace is now the baseline, not the exception, for d...

24 Aug 55min

Software Engineering Laws Every Developer Must Know in the AI Era - Milan Milanovic

Does moving faster with AI mean you get to skip the laws that governed every software project before it? Milan Milanovic argues the opposite, the old laws of software engineering now apply twice as ha...

10 Aug 1h 1min

Signals & Levers: Systems Thinking to Navigate Software Delivery Illusions - Elisabeth Hendrickson & Joel Tosi

(04:48) Brought to you by SpeechifyAITired of text-to-speech that sounds robotic or costs too much at scale? SpeechifyAI's new Simba 3.2 model ranks #1 on Artificial Analysis for realness, priced unde...

3 Aug 1h 17min

From Technical Debt to Triple Debt: The Hidden Cost of AI-Generated Code

Is vibe coding quietly draining the thing that makes teams effective: their shared understanding? Margaret breaks down her new “triple debt” model, and why cognitive debt might be the one nobody’s tra...

27 Juli 1h 10min

Ex-Gojek CTO: Why the Age of AI Rewards Builders as Judgment Becomes Expensive

If expertise is becoming almost free, why is judgment becoming the most expensive skill in tech? Ajey Gore, former Gojek CTO, explains why the age of AI agents rewards builders over typists.In this ep...

13 Juli 1h 22min

The AI Productivity Paradox: Why 10X Output Doesn't Mean 10X Business Outcome

What if optimizing for AI output is actually slowing your company down? When code becomes nearly free to produce, the organizations still measuring productivity by output are solving the wrong problem...

29 Juni 1h 5min