Better Know A Benchmark: ExploitGym
When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling. Katie and Phoebe unpack...
14 Sep 32min
Constitutional AI
How do you teach a model the difference between helpful and harmful when it has no inherent sense of either? This episode dives into Constitutional AI, Anthropic's framework for training AI systems to...
7 Sep 31min
A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)
Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a reality check on AI. As one of the most seasoned observers of how businesses actually adopt trans...
31 Aug 40min
Understanding AI Text Watermarking
Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and while everyone else was busy having opinions about it, we were busy asking the more interesting...
24 Aug 29min
Better Know a Benchmark: Humanity's Last Exam
Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI models can't. Originally dubbed "Humanity's Last Stand," this benchmark is a massive academic col...
17 Aug 23min
A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell)
When a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn Zhou's research says: not necessarily — sometimes confident phrasing correlates with *worse* ac...
10 Aug 33min
Reasoning Models: When LLMs Went Beyond Fancy Autocomplete
Reasoning models don't just answer your question — they *think out loud* first. In this episode we dig into the class of AI models that generate intermediate chains of thought before arriving at a fin...
3 Aug 25min
Distillation, or, How to Steal a Model
This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to train a smaller, cheaper "student" model that mimics it. They cover the two big reasons labs do...
27 Jul 23min
