Luke Muehlhauser on the most important AI safety work nobody is doing

The US government spends 25 times more on military marching bands than on its AI safety institute. And that’s just one of the many institutions we’re missing as AI becomes more powerful and dangerous.

Aviation has the FAA to regulate safety and the NTSB to investigate crashes. Nuclear power has international inspectors and industry watchdogs. But for advanced AI, as Luke Muehlhauser puts it, “most of the institutions that need to exist don’t exist.”

Luke has worked on AI risk since 2011, and now leads AI governance and international policy grantmaking at Coefficient Giving. He thinks we urgently need organisations that can independently evaluate frontier models, investigate AI incidents, verify AI companies’ safety claims, and hold them accountable.

He also wants people to build more specialised organisations: one that tests whether AI systems are prone to lying or pursuing hidden goals; another that tracks whether we can still understand their reasoning; and others that probe how easily AI could enable biological attacks.

The encouraging news is that many of these projects could attract substantial philanthropic backing. What’s missing are people willing to build them.

Luke and host Luisa Rodriguez discuss which opportunities are most neglected, how Coefficient’s new Project Tailwind is helping new organisations get started, how recent AI warning shots are transforming politics, and why Luke thinks we may have much less time than we thought.

Learn more, video, and full transcript: https://80k.info/lm

This episode was recorded August 16–17, 2026.

Chapters:

• Cold open + a note from Luisa (00:00:00)
• Who’s Luke Muehlhauser? (00:02:27)
• The state of AI safety right now (00:03:07)
• The nonprofits Luke most wants to see founded (00:09:58)
• Mid-career pivots that paid off (00:18:58)
• How much did Mythos shift the Overton window? (00:30:38)
• Preparing for the next warning shot (00:37:33)
• How nonprofits shape policy (00:43:52)
• Are chip export controls good for safety? (00:50:17)
• The three worst-case AI scenarios (00:54:29)
• How Luke’s AI timelines have changed in the last year (01:00:13)
• Why alignment is not on track (01:14:31)
• How realistic is a deal with China? (01:20:58)
• Triaging AI risk (01:23:56)
• When AI safety was 10 people (01:30:36)
• AI safety’s mistakes, then and now (01:39:36)
• What is it like to do grantmaking at Coefficient Giving? (02:02:42)
• More projects Luke would like to see founded (02:20:45)
• Staying sane in a scary field (02:26:07)
• “The glorious, cyberpunk future” (02:34:07)

Our production team includes:

• Video editors: Josh Alward, Dominic Armstrong, Andrés Escobar, Milo McGuire, Luke Monsour, and Simon Monsour
• Producers: Elizabeth Cox and Nick Stockton
• Coordination and support: Katy Moore and Lou Moran
• Camera operator: Milo McGuire

Music: CORBIT