Win-Win with Liv Boeree21 Mai 2025

#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built

Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?

In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.

Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built?

We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.

Chapters:

0:00 - Intro

4:08 - Different Categories of Risks

10:51 - Why Can't the Market Manage AI Risks Sufficiently?

15:33 - The Dangers of Power Concentration

22:52 - What Solution Space Exists?

32:55 - Should Compute be Regulated?

38:38 - Taylor Capito - Are we at an Evolutionary Nexus

42:07 - JD Ross - Decentralization vs Centralization

46:46 - Isabelle Boemeke - Nuclear Power & AI

51:06 - Peter Wang - Wrong Stories Attract the Wrong People

57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition

1:05:59 - Can Win-Win be embedded into AI?

Links:

♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai

♾️ Future of Life Institute: https://futureoflife.org

♾️ MIRI: https://intelligence.org

♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834

Credits:

♾️ Hosted by Liv Boeree

♾️ Produced by Luca de Vico

The Win-Win Podcast:

Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.

Podcast links:

♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S

♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f

♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350

♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2

#AGI #podcast #aialignment

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(57)

#55 - Dean Ball x Daniel Kokotajlo Anti-Debate - Should GOVERNMENTS or MARKETS control AI?

Advanced AI is coming. Is it time for the government to step in and regulate? Or is it better in the hands of the market and tech CEOs? Or is there a more nuanced way to think about this topic entirel...

1 Apr 1h 25min

#54 - Greg Lukianoff - Free Speech in the Age of AI

Free speech is often treated as a political issue. But what if it’s the foundational technology that makes peaceful civilization possible in the first place?I sat down with Greg Lukianoff — free speec...

11 Mar 1h 3min

#53 - John Schultz - Why Google Made ChatGPT, Gemini & Claude Play 900,000 hands of Poker...

Which LLM is best at POKER? What about the social deception game Werewolf? This week I'm collaborating with ⁠ @googledeepmind ⁠ and ⁠ @kaggle ⁠ to explain their new “AI Game Arena” project, designed t...

6 Feb 35min

#52 - Will MacAskill - What Comes AFTER Superintelligence?

Superintelligence is predicted to arrive within a decade. If that's true, then what comes AFTER? And how can we prepare for it?Which is why I sat down with one of the few people actively thinking abou...

27 Jan 1h 29min

#51 - Tyler Cowen - Markets, Institutions and Why AI Hype is (Probably) Overblown

What if the biggest risk from AI isn’t superintelligence gone rogue, but the inflexibility of our human institutions?My guest today, Tyler Cowen, thinks so. Tyler is a professor at George Mason univer...

12 Jan 1h 52min

#50 - Samantha Sweetwater - Why Indigenous Wisdom Is Important In The AI Age

What if the purpose of life is to find ways of harmonizing the forces of nature, including the nature of ourselves?My guest this week thinks so. I sat down with systems thinker and "True Human" author...

17 Des 20252h 11min

#49 - British Politician Katie Lam - Defeating Grooming Gangs, Stagnation & Safetyism

Britain was once a beacon of liberty, and the birthplace of the industrial revolution. So why is its economy floundering, while the USA is surging? Has a culture of extreme bureaucracy turned the form...

25 Nov 20252h 20min

#48 - Jim Rutt - Why Capitalism Is Now A Game of Mind Control

Capitalism has historically been our biggest blessing. But what if it's now turning into our biggest curse? Is it time to change the game?My guest today believes so. Jim Rutt is an early internet pion...

8 Okt 20252h 30min