Tech Lead Journal10 Nov 2025

#238 - AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Why does an AI that brilliantly generates code suddenly fail at basic math? The answer explains why your LLM will fail when you least expect it.

In this episode, Emmanuel Maggiori, author of “Smart Until It’s Dumb” and “The AI Pocket Book,” cuts through the AI hype to reveal what LLMs actually do and, more importantly, what they can’t. Drawing from his experience building AI systems and witnessing multiple AI booms and busts, Emmanuel explains why machine learning works brilliantly until it makes mistakes no human would ever make.

He shares why businesses repeatedly fail at AI adoption, how hallucinations are baked into the technology, and what developers need to know about building reliable AI products.

Whether you’re implementing AI at work or concerned about your career, this conversation offers a grounded perspective on navigating the current AI wave without getting swept away by unrealistic promises.

Key topics discussed:

Why AI projects fail the same way repeatedly
How LLMs work and why they brilliantly fail
Why hallucinations can’t be fixed with better prompts
Why self-driving cars still need human operators
Adopting AI without falling into hype traps
How engineers stay relevant in the AI era
Why AGI predictions are mostly marketing
Building valuable products in boring industries

Timestamps:

(00:00:00) Trailer & Intro
(00:02:32) Career Turning Points
(00:06:41) Writing “Smart Until It’s Dumb” and “The AI Pocket Book”
(00:08:14) The History of AI Booms & Winters
(00:11:34) Why Generative AI Hype is Different Than the Past AI Waves
(00:13:26) AI is Smart Until It’s Dumb
(00:16:45) How LLM and Generative AI Actually Work
(00:22:53) What Makes LLMs Smart
(00:27:25) Foundational Model
(00:30:01) RAG and Agentic AI
(00:34:09) Tips on How to Adopt AI Within Companies
(00:37:56) How to Reduce & Avoid AI Hallucination Problem
(00:45:49) The Important Role of Benchmarks When Building AI Products
(00:50:57) Advice for Software Engineers to Deal With AI Concerns
(00:56:49) Advice for Junior Developers
(00:59:34) Vibe Coders and Prompt Engineers: New Jobs or Just Hype?
(01:01:55) The AGI Possibility
(01:07:23) Three Tech Lead Wisdom

_____

Emmanuel Maggiori’s Bio
Emmanuel Maggiori, PhD, is a software engineer and 10-year AI industry insider. He has developed AI for a variety of applications, from processing satellite images to packaging deals for holiday travelers. He is the author of the books Smart Until It’s Dumb, Siliconned, and The AI Pocket Book.

Follow Emmanuel:

LinkedIn – linkedin.com/in/emaggiori
Website – emaggiori.com

Like this episode?
Show notes & transcript: techleadjournal.dev/episodes/238.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Buy me a coffee or become a patron.

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(279)

Software Engineering Laws Every Developer Must Know in the AI Era - Milan Milanovic

Does moving faster with AI mean you get to skip the laws that governed every software project before it? Milan Milanovic argues the opposite, the old laws of software engineering now apply twice as ha...

10 Aug 1h 1min

Signals & Levers: Systems Thinking to Navigate Software Delivery Illusions - Elisabeth Hendrickson & Joel Tosi

(04:48) Brought to you by SpeechifyAITired of text-to-speech that sounds robotic or costs too much at scale? SpeechifyAI's new Simba 3.2 model ranks #1 on Artificial Analysis for realness, priced unde...

3 Aug 1h 17min

From Technical Debt to Triple Debt: The Hidden Cost of AI-Generated Code

Is vibe coding quietly draining the thing that makes teams effective: their shared understanding? Margaret breaks down her new “triple debt” model, and why cognitive debt might be the one nobody’s tra...

27 Jul 1h 10min

Ex-Gojek CTO: Why the Age of AI Rewards Builders as Judgment Becomes Expensive

If expertise is becoming almost free, why is judgment becoming the most expensive skill in tech? Ajey Gore, former Gojek CTO, explains why the age of AI agents rewards builders over typists.In this ep...

13 Jul 1h 22min

The AI Productivity Paradox: Why 10X Output Doesn't Mean 10X Business Outcome

What if optimizing for AI output is actually slowing your company down? When code becomes nearly free to produce, the organizations still measuring productivity by output are solving the wrong problem...

29 Jun 1h 5min

The Hidden Stories Sabotaging Your Culture Change

Why do 70% of change efforts fail — even when leadership is fully committed? The answer isn’t strategy or resources; it’s the hidden stories people unknowingly carry that silently block every initiati...

15 Jun 1h 8min

Creator of Meta's Hack: Your AI Will Always Cheat — Here's How to Stop It

What if your AI coding agent is quietly cheating on your tests — and how do you stop it? Julien Verlaguet, who built the type system Meta used to migrate tens of millions of PHP lines, is now building...

8 Jun 1h 18min

Eric Ries: Why Good Tech Companies Go Bad, and How to Stop It

Why do companies with the best intentions end up betraying their customers, employees, and mission? Eric Ries calls it “financial gravity” — an invisible force that pulls even the most principled comp...

1 Jun 1h