Unfaithful Chain of Thought

Unfaithful Chain of Thought

What's actually happening when an LLM "thinks out loud"? Research on human decision-making suggests that much of the reasoning we believe drives our choices is actually post hoc rationalization — we decide first, explain later. Katie and Ben get curious about whether the same might be true for large language models: when you watch a model reason through a problem in real time, is that chain of thought the genuine process, or just a plausible-sounding story told after the fact? It's a deceptively deep question with real stakes for how much we should trust model explanations. Miles Turpin et al., "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting" (NeurIPS 2023, NYU and Anthropic): https://arxiv.org/abs/2305.04388 Anthropic, "Reasoning Models Don't Always Say What They Think" (Alignment Faking research, 2025): https://www.anthropic.com/research/reasoning-models-dont-say-think

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(328)

Back in Two Weeks

Back in Two Weeks

Back in Two Weeks by Katie Malone

12 Loka 21s

Mechanistic Interpretability: How Researchers Try To See Inside Models

Mechanistic Interpretability: How Researchers Try To See Inside Models

As AI systems grow more powerful, they grow harder to understand — and that tradeoff has never mattered more. Most interpretability tricks give us approximations from the outside, but mechanistic inte...

5 Loka 29min

LLM As A Judge

LLM As A Judge

Labeled data is expensive, slow, and painfully hard to come by — and when you're trying to evaluate whether an AI workflow is actually doing what you want, the problem gets even messier. How do you ju...

28 Syys 30min

The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)

The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)

Originally aired on the Harvard Data Science Review podcast. What can podcasting teach us about AI — and what can AI teach us about the future of podcasting? Katie Malone (that's our host) joins Jon ...

21 Syys 29min

Better Know A Benchmark: ExploitGym

Better Know A Benchmark: ExploitGym

When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling. Katie and Phoebe unpack...

14 Syys 32min

Constitutional AI

Constitutional AI

How do you teach a model the difference between helpful and harmful when it has no inherent sense of either? This episode dives into Constitutional AI, Anthropic's framework for training AI systems to...

7 Syys 31min

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a reality check on AI. As one of the most seasoned observers of how businesses actually adopt trans...

31 Elo 40min

Understanding AI Text Watermarking

Understanding AI Text Watermarking

Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and while everyone else was busy having opinions about it, we were busy asking the more interesting...

24 Elo 29min