The UK's former head AI safety scientist on how to solve alignment before superintelligence arrives | Geoffrey Irving

When should governments slow the race toward superintelligence? According to Geoffrey Irving, the careful answer is sometime in the past. The useful answer is now.

Geoffrey — formerly a safety researcher at OpenAI and Google DeepMind and chief scientist at the UK AI Security Institute — expects full-blown superintelligence in roughly two to three years.

***
Want to work with Geoffrey to help align superintelligence? Resolution is hiring! https://80k.info/work-at-resolution
***

The leading AI companies all have broadly similar plans for keeping superintelligence under control:

Train models to have good character
Use increasingly capable AIs to supervise other AIs
Monitor them closely for signs of deception or scheming

Geoffrey thinks that combination could work. The alarming part is that nobody has a strong argument that it will. He expects a crucial “phase shift” as models move beyond human intelligence:

Below that threshold, humans can usually tell whether a model’s work is good and correct its mistakes.
Above it, the models themselves will increasingly determine the feedback used to train their successors.

In this episode, Geoffrey and new host Tom Reed explore what might go wrong with the companies’ plans; why Geoffrey’s new nonprofit, Resolution, is pursuing a portfolio of neglected research bets; and whether governments should slow AI development while we work out which methods can actually be trusted.

This episode was recorded on June 29, 2026.

Full transcript, video, and links to learn more: https://80k.info/gi

Chapters:

Cold open (00:00:00)
Meet Tom Reed — our newest host! (00:00:32)
Who’s Geoffrey Irving? (00:00:59)
What misaligned superintelligence will look like (00:01:38)
Why are AI companies more optimistic about alignment than Geoffrey? (00:12:30)
Why Geoffrey expects superintelligence in 2–3 years (00:28:05)
When and how to slow down frontier AI development (00:31:30)
Safety researchers can have more impact in governments than companies (00:39:22)
How Geoffrey’s new organisation plans to tackle alignment (00:46:55)
Post-ASI science: nanotech, solving ageing, and uploaded minds (00:50:29)
Why we should expect superintelligence to accelerate scientific progress (01:03:30)
Can good character training carry over to superintelligence? (01:11:03)
What the field of AI alignment still doesn’t know (01:16:44)
Lessons from politics on how to combat power seeking (01:24:36)
Solving Pentago and working at Pixar (01:29:22)
Geoffrey’s best prediction (01:32:40)
Geoffrey’s best bets on which alignment techniques will work (01:37:38)
Work with Geoffrey at Resolution (01:43:34)
The dangerous asymmetry between capabilities and alignment (01:54:17)

Our team is hiring! The 80,000 Hours Podcast aims to help the world safely navigate the transition to transformative AI. Help us make more great episodes as a producer, production coordinator, or special projects associate. https://80k.info/work

Our production team includes:

Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour
Producers: Elizabeth Cox and Nick Stockton
Coordination and support: Katy Moore and Lou Moran
Camera operator: Jeremy Chevillotte

Music: CORBIT

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(347)

Where AGI timelines go wrong | Toby Ord, Oxford University

Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative and author of The Precipice, believes we consistently make ...

6 Aug 2h 46min

What the hell happened with AGI timelines in 2026? – Rob Wiblin

Last October, famed coder Andrej Karpathy called AI agents “slop.” Two months later he completely reversed his view, describing them as “alien tools” that are “rocking the profession.”He was far from ...

4 Aug 49min

#249 – Spencer Greenberg on staying sane while trying to save the world

If you genuinely believe that humanity could be wiped out by AI or a pandemic, what is the appropriate amount of fear to feel?“As much as possible” can seem like the only reasonable answer. If the wor...

28 Jul 2h 9min

#248 – Jasmine Sun on what the people building AI really believe

Many AI researchers believe mass job displacement is coming — and some even think there’s a chance their technology will kill everyone. But they’re building it anyway. Writer and journalist Jasmine Su...

21 Jul 1h 6min

#247 – Anton Leicht on how middle powers avoid losing everything in a post-AI world

In a post-AGI world, can a country without access to frontier AI even be considered sovereign anymore?Anton Leicht says once frontier AI becomes a core economic input, the countries that own it will p...

14 Jul 1h 33min

#246 – Sneha Revanur on how a small team of activists helped pass America's landmark AI safety laws

Six years ago, aged just 15, Sneha Revanur founded the AI advocacy nonprofit Encode AI — back when AI felt like a niche issue. Now the world’s caught up with her, and she’s ready to share everything s...

8 Jul 52min

We can guess what intergalactic war would look like. And strangely, it matters.

Intergalactic war is probably billions of years away — yet physics can already tell us how it ends. And strangely that conclusion is relevant to decisions people have to make today.In this video, Rob ...

18 Jun 15min