The Pre-Training Wall and the Treadmill After It

The Pre-Training Wall and the Treadmill After It

I've been confusing Don with frontier-lab links late at night for a bit.

Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall.

A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion.

Don was confused. So he came over and we made an episode about it.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(115)

Quick Update

Quick Update

A quick update from Adam about the podcast's current state, consistency challenges, and what's coming next.  Episode Page Support The Show Subscribe To The Podcast Join The Newsletter

2 Sep 20258min

Coding in the Red-Queen Era

Coding in the Red-Queen Era

What do we risk when we let AI do the heavy lifting in our coding? Are we giving up the thinking that makes us good at what we do? And as expectations keep rising to match productivy gains, is all thi...

6 Aug 202542min

When AI Codes, What's Left for me?

When AI Codes, What's Left for me?

I've always found meaning—and a lot of strength—in building things. Now, with AI coding agents changing the way we work, it's easy to feel threatened, like something essential might get taken away. Bu...

2 Juli 202539min

Story: Coding Through Chaos : Addiction, Recovery and Acceptance

Story: Coding Through Chaos : Addiction, Recovery and Acceptance

What if your search for connection took you somewhere you never meant to go—almost costing you everything? John Walker grew up building computers and exploring early internet forums, always looking fo...

3 Juni 202548min

Story: The Power of Context: Reimagining Learning

Story: The Power of Context: Reimagining Learning

Have you ever found yourself in a situation where learning felt like an uphill battle? Like no matter how hard you tried, the pieces just wouldn't fall into place?   Steve Krouse's story shows the pow...

2 Maj 202545min

Story: briffa_sep98_e.pro - The File That Sparked a Storm

Story: briffa_sep98_e.pro - The File That Sparked a Storm

Can a single line of code change the way we see science, policy, and trust?   In this episode we explore the "Climategate" scandal that erupted from leaked emails and code snippets, fueling doubts abo...

2 Apr 202557min

Story: Hatetris - Obsession, Friendship, and World Records

Story: Hatetris - Obsession, Friendship, and World Records

What if a simple game became a gateway to computational breakthroughs? David Freiberg and Felipe set out on a journey to conquer Hatetris, a notoriously difficult JavaScript game. Their interest ignit...

3 Mars 202548min

Story: One Million Checkboxes - The Viral Game That Defied Convention

Story: One Million Checkboxes - The Viral Game That Defied Convention

What if internet trolls could become your greatest collaborators? Nolen Royalty discovered this unexpected truth when his simple checkbox game went viral. It began with a school email prank that crash...

3 Feb 202552min

Populärt inom Politik & nyheter

aftonbladet-krim
motiv
p3-krim
aftonbladet-daily
svenska-fall
politiken
flashback-forever
rss-krimstad
spar
rss-sanning-konsekvens
rss-krimreportrarna
rss-expressen-dok
rss-vad-fan-hande
rss-frandfors-horna
rss-flodet
olyckan-inifran
kungligt
grans
blenda-2
dagens-eko