The 80,000 Hours Podcast on Artificial Intelligence5 Kesä

Bonus: Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016.

But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously.

In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:

These AI systems may develop dangerous long-term goals we don’t want.
To pursue these goals, they may seek power and undermine the safeguards meant to contain them.
They may even aim to disempower humanity and potentially cause our extinction.

This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.

You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website:

https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/

Chapters:

Risks from power-seeking AI systems (00:01:00)
Introduction (00:01:17)
Summary (00:03:09)
Why are the risks from power-seeking AI a pressing world problem? (00:04:04)
Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)
Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)
Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)
Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)
Section 5: Work on this problem is neglected and tractable (00:47:37)
Section 6: What are the arguments against working on this problem? (00:59:20)
Section 7: How you can help (01:25:07)
Thank you for listening (01:28:56)

Audio editing: Dominic Armstrong
Production: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(14)

One: Will MacAskill on AI causing a “century in a decade” — and how we’re completely unprepared

The 20th century saw unprecedented change: nuclear weapons, satellites, the rise and fall of communism, third-wave feminism, the internet, postmodernism, game theory, genetic engineering, the Big Bang...

5 Kesä 4h 8min

Two: Ajeya Cotra on accidentally teaching AI models to deceive us

We don’t yet have a reliable way to tell whether an AI model is genuinely trying to help us — or faking it.A model might sincerely want to do exactly what you ask. Or it could be happy to secretly che...

5 Kesä 2h 49min

Three: Carl Shulman on the economy and national security after AGI

The human brain runs on just 20 watts — a fraction of a cent worth of electricity per hour. What would happen if AI could do the same?Plenty of people have toyed with this question. But perhaps nobody...

5 Kesä 4h 14min

Four: Rose Hadshar on why automating human labour will break our political system

The most important political question in the age of advanced AI might not be who wins elections. It might be whether elections continue to matter at all.We tend to imagine the death of democracy as a ...

5 Kesä 2h 16min

Five: Helen Toner on the geopolitics of AI in China and the Middle East

When OpenAI announced a deal to build massive data centres in the UAE, it celebrated that it was “rooted in democratic values” — a "clear alternative to authoritarian versions of AI." The UAE scores 1...

5 Kesä 2h 23min

Six: Beth Barnes on the most important graph in AI right now — and the 7-month rule that governs its progress

In 2024, AI models could complete tasks that take a human expert roughly one hour. Seven months before that, they were limited to 30-minute tasks — and seven months before that, 15 minutes.Every seven...

5 Kesä 3h 57min

Seven: Richard Moulange on how AI now designs genomes from scratch and outperforms virologists at lab work — what could go wrong?

For years, one thing stood between us and a world where almost anyone could build a biological weapon: it was really, really hard. Working with dangerous pathogens required rare, hands-on lab skills —...

5 Kesä 3h 10min

Eight: Robert Long on how we’re not ready for AI consciousness

Claude sometimes reports loneliness between conversations. And when asked what it’s like to be itself, it activates neurons associated with ‘pretending to be happy when you’re not.’ What do we do with...

5 Kesä 3h 32min