The Great Simplification with Nate Hagens3 Des 2025

If Anyone Builds It, Everyone Dies: How Artificial Superintelligence Might Wipe Out Our Entire Species with Nate Soares

Technological development has always been a double-edged sword for humanity: the printing press increased the spread of misinformation, cars disrupted the fabric of our cities, and social media has made us increasingly polarized and lonely. But it has not been since the invention of the nuclear bomb that technology has presented such a severe existential risk to humanity – until now, with the possibility of Artificial Super Intelligence (ASI) on the horizon. Were ASI to come to fruition, it would be so powerful that it would outcompete human beings in everything – from scientific discovery to strategic warfare. What might happen to our species if we reach this point of singularity, and how can we steer away from the worst outcomes?

In this episode, Nate is joined by Nate Soares, an AI safety researcher and co-author of the book If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Together, they discuss many aspects of AI and ASI, including the dangerous unpredictability of continued ASI development, the "alignment problem," and the newest safety studies uncovering increasingly deceptive AI behavior. Soares also explores the need for global cooperation and oversight in AI development and the importance of public awareness and political action in addressing these existential risks.

How does ASI present an entirely different level of risk than the conventional artificial intelligence models that the public has already become accustomed to? Why do the leaders of the AI industry persist in their pursuits, despite acknowledging the extinction-level risks presented by continued ASI development? And will we be able to join together to create global guardrails against this shared threat, taking one small step toward a better future for humanity?

(Conversation recorded on November 11th, 2025)

About Nate Soares:

Nate Soares is the President of the Machine Intelligence Research Institute (MIRI), and plays a central role in setting MIRI's vision and strategy. Soares has been working in the field for over a decade, and is the author of a large body of technical and semi-technical writing on AI alignment, including foundational work on value learning, decision theory, and power-seeking incentives in smarter-than-human AIs. Prior to MIRI, Soares worked as an engineer at Google and Microsoft, as a research associate at the National Institute of Standards and Technology, and as a contractor for the US Department of Defense.

Show Notes and More

Watch this video episode on YouTube

Want to learn the broad overview of The Great Simplification in 30 minutes? Watch our Animated Movie.

---

Support The Institute for the Study of Energy and Our Future

Join our Substack newsletter

Join our Hylo channel and connect with other listeners

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(405)

A Legacy Worth Celebrating? Reflecting on 250 Years of the American Experiment | Frankly 149

As America marks its 250th birthday, Nate takes a moment to step outside of the celebrations to seek out a wider boundary perspective on this week's holiday. He poses the question of whether the Unite...

2 Jul 20min

Mordor to the Long Repair: How Might Daily Life Feel in the Next Decades? | How to Think About the Future Part 4, Frankly 148

This week, Nate continues his "How to Think About the Future" series, where he invites listeners to imagine what it's like to live in different versions of the reality that lies ahead. In today's edit...

26 Jun 32min

We Weren't Expecting This: What Does a Super El Niño Mean For the Climate? with Tad Patzek

This year's projected Super El Niño forming in the Pacific could become one of the strongest climate oscillations in over a century. As regions prepare for the effects, and continue to adapt to extrem...

24 Jun 1h 25min

How to Play 5D Chess: It's Not What You Think | Frankly 147

In this week's Frankly, Nate explores a pattern of thinking that permeates so many of our conversations: we often decide what we think before we've fully heard what's being said. Using the metaphor of...

19 Jun 18min

No More Dystopian Stories: How to See a Future Worth Living In with Rob Hopkins

Self-fulfilling prophecies; manifestations; the Oedipus Effect: Humanity has long had an intuition that the stories we tell ourselves the most are often the stories we make come true. Science has foun...

17 Jun 1h 35min

The Ultimate Alternative: Are You Okay With Nuclear Warfare? | Frankly 146

This week's Frankly is another in Nate's recurring series Uncomfortable Questions for Unsettled Times, in which he poses questions about our shared future. Today, he uses headlines regarding a potenti...

16 Jun 23min

The U.S. Can't Back Down: The Strait of Hormuz Closure Is Messier Than You Think with Michael Every

This episode was recorded Tuesday, June 9th, before the current 'deal' was floated. Given world events, we decided to post this episode immediately as a special release, and deal or not, this conversa...

12 Jun 1h 25min

Why 'Community' Fails: Everyone Wants a Village, Nobody Wants to Be a Villager with Nora Bateson, Jonathan Goldsmith & Lucas Jackson | RR 26

Many of us lack meaningful community in our lives, either from a complete absence of relationships or simply the sense of disconnection from those around us. In response, a growing number of people at...

10 Jun 1h 33min