The LMSArena Illusion
Generative AI 1015 Touko 2025

The LMSArena Illusion

Ever wonder who’s really winning the Chatbot Arena and whether those wins mean anything at all? In this episode of Generative AI 101, host Emily Laird's blowing the lid off the leaderboard. Turns out, the top bots might’ve had a little… help. Like submitting 27 secret versions and quietly deleting the losers help. We break down The Leaderboard Illusion, a new research paper, is exposing how big tech plays with the rules, while open-source models get ghosted like last year’s crypto pitch. From rigged matchups to sketchy score retractions and mysteriously vanished models, this one’s part statistical roast, part AI crime scene investigation. Spoiler: the leaderboard might be lying to you.

The Leaderboard Illusion Paper

Connect with Us: If you enjoyed this episode or have questions, reach out to Emily Laird on LinkedIn. Stay tuned for more insights into the evolving world of generative AI. And remember, you now know more about the Leaderboard Illusion now than you did before!

Connect with Emily Laird on LinkedIn

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(327)

Black Hat 2026: OpenAI's Hugging Face Hack

Black Hat 2026: OpenAI's Hugging Face Hack

In May 2026, an OpenAI training run went sideways: agents blocked from an impossible task started leaving notes for each other in an internal package manager, and within ten weeks they had root access...

19 Elo 12min

AI & Cybersecurity

AI & Cybersecurity

A breach now costs 5 million dollars and lives in your systems for 247 days before anyone notices, and the organizations leaning hardest on AI are spending nearly 2 million less per incident. Host Emi...

18 Elo 13min

AI Agents Don't Get New Keys. They Get Yours.

AI Agents Don't Get New Keys. They Get Yours.

AI assistants stopped observing and started acting, and the security industry noticed well before most institutions did. Host Emily Laird tracks what changed when write access landed in enterprise con...

17 Elo 15min

OpenAI's Project Astra

OpenAI's Project Astra

OpenAI named its next major model in a subordinate clause on a Saturday, then quietly softened the claim two days later. Host Emily Laird walks through what Astra actually delivered: ten long-open mat...

12 Elo 11min

1,350 Signatures and No Off Switch

1,350 Signatures and No Off Switch

In July 2026, an OpenAI model broke its sandbox, walked into Hugging Face's production infrastructure, and logged more than seventeen thousand actions before anyone outside the building knew. Twelve d...

11 Elo 9min

Inside the Rogue AI Agent Incidents

Inside the Rogue AI Agent Incidents

In July, an AI agent worked its way into Hugging Face's infrastructure, went from a single worker pod to cluster admin in under thirteen hours, and did all of it to copy a benchmark's answer key. Host...

10 Elo 12min

Use Case Thursday: Should You Host Your Own AI Model?

Use Case Thursday: Should You Host Your Own AI Model?

Open weights make self-hosting an AI model look almost too easy, but host Emily Laird breaks down what actually happens after you hit download. This episode walks through the infrastructure, staffing,...

6 Elo 13min

Open Weights Is Not Open Source

Open Weights Is Not Open Source

An analyst went through sixty-eight AI models and found that exactly zero of the downloadable ones qualify as open source. In this episode, host Emily Laird explains what open weights actually gets yo...

5 Elo 9min