If Anyone Builds It, Everyone Dies: How Artificial Superintelligence Might Wipe Out Our Entire Species with Nate Soares

If Anyone Builds It, Everyone Dies: How Artificial Superintelligence Might Wipe Out Our Entire Species with Nate Soares

Technological development has always been a double-edged sword for humanity: the printing press increased the spread of misinformation, cars disrupted the fabric of our cities, and social media has made us increasingly polarized and lonely. But it has not been since the invention of the nuclear bomb that technology has presented such a severe existential risk to humanity – until now, with the possibility of Artificial Super Intelligence (ASI) on the horizon. Were ASI to come to fruition, it would be so powerful that it would outcompete human beings in everything – from scientific discovery to strategic warfare. What might happen to our species if we reach this point of singularity, and how can we steer away from the worst outcomes?

In this episode, Nate is joined by Nate Soares, an AI safety researcher and co-author of the book If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Together, they discuss many aspects of AI and ASI, including the dangerous unpredictability of continued ASI development, the "alignment problem," and the newest safety studies uncovering increasingly deceptive AI behavior. Soares also explores the need for global cooperation and oversight in AI development and the importance of public awareness and political action in addressing these existential risks.

How does ASI present an entirely different level of risk than the conventional artificial intelligence models that the public has already become accustomed to? Why do the leaders of the AI industry persist in their pursuits, despite acknowledging the extinction-level risks presented by continued ASI development? And will we be able to join together to create global guardrails against this shared threat, taking one small step toward a better future for humanity?

(Conversation recorded on November 11th, 2025)

About Nate Soares:

Nate Soares is the President of the Machine Intelligence Research Institute (MIRI), and plays a central role in setting MIRI's vision and strategy. Soares has been working in the field for over a decade, and is the author of a large body of technical and semi-technical writing on AI alignment, including foundational work on value learning, decision theory, and power-seeking incentives in smarter-than-human AIs. Prior to MIRI, Soares worked as an engineer at Google and Microsoft, as a research associate at the National Institute of Standards and Technology, and as a contractor for the US Department of Defense.

Show Notes and More

Watch this video episode on YouTube

Want to learn the broad overview of The Great Simplification in 30 minutes? Watch our Animated Movie.

---

Support The Institute for the Study of Energy and Our Future

Join our Substack newsletter

Join our Hylo channel and connect with other listeners

Jaksot(374)

Inflation, Deflation, & Simplification: The 8 Things That Influence Prices | Frankly 115

Inflation, Deflation, & Simplification: The 8 Things That Influence Prices | Frankly 115

In this week's Frankly, Nate explores how the prices we encounter in our daily lives are influenced by not only how much money is in the system, but also by resource depletion, technology, affordabili...

5 Joulu 202526min

Reimagining Ourselves at the End of Our World: Kinship, Interconnection, and Spirituality in the Metacrisis with Samantha Sweetwater

Reimagining Ourselves at the End of Our World: Kinship, Interconnection, and Spirituality in the Metacrisis with Samantha Sweetwater

Over the past decade, the world has become increasingly chaotic and uncertain – and so, too, has our cultural vision for the future. While the events we face now may feel unprecedented, they are roote...

24 Marras 20251h 33min

Directional Advice for the (More Than) Human Predicament | Frankly 114

Directional Advice for the (More Than) Human Predicament | Frankly 114

Over the past decade, the world has become increasingly chaotic and uncertain – and so, too, has our cultural vision for the future. While the events we face now may feel unprecedented, they are roote...

22 Marras 202540min

Two Ways of Knowing: How Merging Science & Indigenous Wisdom Fuels New Discoveries with Rosa Vásquez Espinoza

Two Ways of Knowing: How Merging Science & Indigenous Wisdom Fuels New Discoveries with Rosa Vásquez Espinoza

For centuries, modern science has relied on the scientific method to better understand the world around us. While helpful in many contexts, the scientific method is also objective, controlled, and red...

19 Marras 20251h 22min

11 Discoveries That Changed My Worldview | Frankly 113

11 Discoveries That Changed My Worldview | Frankly 113

In this episode, Nate weaves personal reflections into an exploration of the human predicament, unpacking a series of chronological insights that have reshaped his worldview. What began years ago as a...

14 Marras 202537min

Will We Artificially Cool the Planet? The Science and Politics of Geoengineering with Ted Parson

Will We Artificially Cool the Planet? The Science and Politics of Geoengineering with Ted Parson

Global heating continues, despite the increased use of renewable energy sources and international policies attempting otherwise. Even as emissions reduction efforts continue, our world faces more extr...

12 Marras 20251h 21min

Hacking Human Attachment: The Loneliness Crisis, Cognitive Atrophy and other Personal Dangers of AI | RR 20

Hacking Human Attachment: The Loneliness Crisis, Cognitive Atrophy and other Personal Dangers of AI | RR 20

Mainstream conversations about artificial intelligence tend to center around the technology's economic and large-scale impacts. Yet it's at the individual level where we're seeing AI's most potent eff...

5 Marras 20251h 53min

Suosittua kategoriassa Tiede

tiedekulma-podcast
rss-mita-tulisi-tietaa
rss-poliisin-mieli
rss-metsantuntijat-podcast
utelias-mieli
rss-duodecim-lehti
mielipaivakirja
filocast-filosofian-perusteet
rss-astetta-parempi-elama-podcast
rss-tiedetta-vai-tarinaa
rss-luontopodi-samuel-glassar-tutkii-luonnon-ihmeita
university-of-eastern-finland
docemilia
menologeja-tutkimusmatka-vaihdevuosiin
rss-bios-podcast
rss-ylistys-elaimille
rss-lapsuuden-rakentajat-podcast
rss-lihavuudesta-podcast
rss-sosiopodi
rss-miljonaarien-tasavalta