KV Cache Fundamentals
The unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal — every token attending to every other token means costs ...
2 Nov 24min
AI Inference 101
Everyone talks about training the model — the data, the compute, the alignment magic. But what happens after the weights are frozen and millions of users start hammering it with prompts? That's infere...
26 Okt 27min
Back Next Week
Back Next Week by Katie Malone
19 Okt 18s
Back in Two Weeks
Back in Two Weeks by Katie Malone
12 Okt 21s
LLM As A Judge
Labeled data is expensive, slow, and painfully hard to come by — and when you're trying to evaluate whether an AI workflow is actually doing what you want, the problem gets even messier. How do you ju...
28 Sep 30min
The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post)
Originally aired on the Harvard Data Science Review podcast. What can podcasting teach us about AI — and what can AI teach us about the future of podcasting? Katie Malone (that's our host) joins Jon ...
21 Sep 29min
Better Know A Benchmark: ExploitGym
When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling. Katie and Phoebe unpack...
14 Sep 32min
