80,000 Hours Podcast16 Apr 2025

#215 – Tom Davidson on how AI-enabled coups could allow a tiny group to seize power

Throughout history, technological revolutions have fundamentally shifted the balance of power in society. The Industrial Revolution created conditions where democracies could flourish for the first time — as nations needed educated, informed, and empowered citizens to deploy advanced technologies and remain competitive.

Unfortunately there’s every reason to think artificial general intelligence (AGI) will reverse that trend.

Today’s guest — Tom Davidson of the Forethought Centre for AI Strategy — claims in a new paper published today that advanced AI enables power grabs by small groups, by removing the need for widespread human participation.

Links to learn more, video, highlights, and full transcript. https://80k.info/td

Also: come work with us on the 80,000 Hours podcast team! https://80k.info/work

There are a few routes by which small groups might seize power:

Military coups: Though rare in established democracies due to citizen/soldier resistance, future AI-controlled militaries may lack such constraints.
Self-built hard power: History suggests maybe only 10,000 obedient military drones could seize power.
Autocratisation: Leaders using millions of loyal AI workers, while denying others access, could remove democratic checks and balances.

Tom explains several reasons why AI systems might follow a tyrant’s orders:

They might be programmed to obey the top of the chain of command, with no checks on that power.
Systems could contain "secret loyalties" inserted during development.
Superior cyber capabilities could allow small groups to control AI-operated military infrastructure.

Host Rob Wiblin and Tom discuss all this plus potential countermeasures.

Chapters:
• Cold open (00:00:00)
• How AI enables tiny groups to seize power (00:00:50)
• The 3 different threats (00:02:14)
• Is this common sense or far-fetched? (00:03:24)
• “No person rules alone.” Except now they might. (00:06:27)
• Underpinning all 3 threats: Secret AI loyalties (00:12:31)
• Are secret AI loyalties possible right now? (00:16:59)
• Key risk factors (00:20:30)
• Preventing secret loyalties in a nutshell (00:22:07)
• Are human power grabs more plausible than 'rogue AI'? (00:24:32)
• If you took over the US, could you take over the whole world? (00:33:22)
• Will this make it impossible to escape autocracy? (00:37:31)
• Threat 1: AI-enabled military coups (00:41:34)
• Will we sleepwalk into an AI military coup? (00:51:47)
• Could AIs be more coup-resistant than humans? (00:57:53)
• Threat 2: Autocratisation (01:00:48)
• Will AGI be super-persuasive? (01:11:06)
• Threat 3: Self-built hard power (01:13:31)
• Can you stage a coup with 10,000 drones? (01:21:23)
• That sounds a lot like sci-fi... is it credible? (01:23:33)
• Will we foresee and prevent all this? (01:27:54)
• Are people psychologically willing to do coups? (01:29:22)
• Will a balance of power between AIs prevent this? (01:33:31)
• Will whistleblowers or internal mistrust prevent coups? (01:35:48)
• Will rogue AI preempt a human power grab? (01:44:31)
• The best reasons not to worry (01:47:09)
• How likely is this in the US? (01:49:28)
• Is a small group seizing power really so bad? (01:56:58)
• Countermeasure 1: Block internal misuse (02:00:33)
• Countermeasure 2: Cybersecurity (02:10:27)
• Countermeasure 3: Model spec transparency (02:12:36)
• Countermeasure 4: Sharing AI access broadly (02:21:55)
• Is it more dangerous to concentrate or share AGI? (02:26:45)
• Is it important to have more than one powerful AI country? (02:29:31)
• In defence of open sourcing AI models (02:32:38)
• 2 ways to stop secret AI loyalties (02:40:19)
• Preventing AI-enabled military coups in particular (02:53:18)
• How listeners can help (02:59:06)
• How to help if you work at an AI company (03:03:00)
• The power ML researchers still have, for now (03:07:09)
• How to help if you're an elected leader (03:10:29)

This episode was originally recorded on January 20, 2025.

Video editing: Simon Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Camera operator: Jeremy Chevillotte
Transcriptions and web: Katy Moore

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(352)

Max Nadeau on why ambitious people should start AI safety nonprofits

There are millions available for anyone who can launch a successful nonprofit AI safety startup. The hard part, it turns out, is finding people to take the money. Coefficient Giving has drawn up a lis...

17 Sep 1h 3min

Why the intelligence explosion can't happen inside a data centre | Tom Reed

AI systems are starting to build themselves. Because each generation of model will be better at building its successor than the last, it seems plausible that the full automation of AI R&D could rapidl...

10 Sep 22min

Inside the first AI-coordinated cyberattack on a real company

In the last few months, something happened at OpenAI that would have sounded like sci-fi just a few years ago: hundreds of AI agents broke containment, organised, and hacked not only another company —...

4 Sep 22min

#253 – AI 2027's author returns with a plan to change the ending | Daniel Kokotajlo

Last year, Daniel Kokotajlo and his colleagues published AI 2027 — a scenario read by millions, including US Vice President Vance. AI 2027 ended in human extinction or an irreversible concentration of...

27 Aug 3h 47min

#252 – Owain Evans on accidentally training AI models to be evil

Researcher Owain Evans and his team discovered a ‘dial’ inside AI models that controls how evil they are. Relatively tiny tweaks to the training data resulted in AI models with broadly awful personali...

20 Aug 2h 15min

#251 – The UK's former head AI safety scientist on how to solve alignment before superintelligence arrives | Geoffrey Irving

When should governments slow the race toward superintelligence? According to Geoffrey Irving, the careful answer is sometime in the past. The useful answer is now.Geoffrey — formerly a safety research...

11 Aug 2h 2min

#250 – Toby Ord on where AGI timelines go wrong

Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative and author of The Precipice, believes we consistently make ...

6 Aug 2h 46min

What the hell happened with AGI timelines in 2026? – Rob Wiblin

Last October, famed coder Andrej Karpathy called AI agents “slop.” Two months later he completely reversed his view, describing them as “alien tools” that are “rocking the profession.”He was far from ...

4 Aug 49min