#184 – Zvi Mowshowitz on sleeping on sleeper agents, and the biggest AI updates since ChatGPT

#184 – Zvi Mowshowitz on sleeping on sleeper agents, and the biggest AI updates since ChatGPT

Many of you will have heard of Zvi Mowshowitz as a superhuman information-absorbing-and-processing machine — which he definitely is. As the author of the Substack Don’t Worry About the Vase, Zvi has spent as much time as literally anyone in the world over the last two years tracking in detail how the explosion of AI has been playing out — and he has strong opinions about almost every aspect of it.

Links to learn more, summary, and full transcript.

In today’s episode, host Rob Wiblin asks Zvi for his takes on:

  • US-China negotiations
  • Whether AI progress has stalled
  • The biggest wins and losses for alignment in 2023
  • EU and White House AI regulations
  • Which major AI lab has the best safety strategy
  • The pros and cons of the Pause AI movement
  • Recent breakthroughs in capabilities
  • In what situations it’s morally acceptable to work at AI labs

Whether you agree or disagree with his views, Zvi is super informed and brimming with concrete details.


Zvi and Rob also talk about:

  • The risk of AI labs fooling themselves into believing their alignment plans are working when they may not be.
  • The “sleeper agent” issue uncovered in a recent Anthropic paper, and how it shows us how hard alignment actually is.
  • Why Zvi disagrees with 80,000 Hours’ advice about gaining career capital to have a positive impact.
  • Zvi’s project to identify the most strikingly horrible and neglected policy failures in the US, and how Zvi founded a new think tank (Balsa Research) to identify innovative solutions to overthrow the horrible status quo in areas like domestic shipping, environmental reviews, and housing supply.
  • Why Zvi thinks that improving people’s prosperity and housing can make them care more about existential risks like AI.
  • An idea from the online rationality community that Zvi thinks is really underrated and more people should have heard of: simulacra levels.
  • And plenty more.

Chapters:

  • Zvi’s AI-related worldview (00:03:41)
  • Sleeper agents (00:05:55)
  • Safety plans of the three major labs (00:21:47)
  • Misalignment vs misuse vs structural issues (00:50:00)
  • Should concerned people work at AI labs? (00:55:45)
  • Pause AI campaign (01:30:16)
  • Has progress on useful AI products stalled? (01:38:03)
  • White House executive order and US politics (01:42:09)
  • Reasons for AI policy optimism (01:56:38)
  • Zvi’s day-to-day (02:09:47)
  • Big wins and losses on safety and alignment in 2023 (02:12:29)
  • Other unappreciated technical breakthroughs (02:17:54)
  • Concrete things we can do to mitigate risks (02:31:19)
  • Balsa Research and the Jones Act (02:34:40)
  • The National Environmental Policy Act (02:50:36)
  • Housing policy (02:59:59)
  • Underrated rationalist worldviews (03:16:22)

Producer and editor: Keiran Harris
Audio Engineering Lead: Ben Cordell
Technical editing: Simon Monsour, Milo McGuire, and Dominic Armstrong
Transcriptions and additional content editing: Katy Moore

Avsnitt(333)

Rob Wiblin on plastic straws, nicotine, doping, & whether changing the long-term is really possible

Rob Wiblin on plastic straws, nicotine, doping, & whether changing the long-term is really possible

Today's episode is a compilation of interviews I recently recorded for two other shows, Love Your Work and The Neoliberal Podcast.  If you've listened to absolutely everything on this podcast feed, y...

25 Sep 20193h 14min

Have we helped you have a bigger social impact? Our annual survey, plus other ways we can help you.

Have we helped you have a bigger social impact? Our annual survey, plus other ways we can help you.

1. Fill out our annual impact survey here. 2. Find a great vacancy on our job board. 3. Learn about our key ideas, and get links to our top articles. 4. Join our newsletter for an email about what's n...

16 Sep 20193min

#63 – Vitalik Buterin on better ways to fund public goods, blockchain's failures, & effective giving

#63 – Vitalik Buterin on better ways to fund public goods, blockchain's failures, & effective giving

Historically, progress in the field of cryptography has had major consequences. It has changed the course of major wars, made it possible to do business on the internet, and enabled private communicat...

3 Sep 20193h 18min

#62 – Paul Christiano on messaging the future, increasing compute, & how CO2 impacts your brain

#62 – Paul Christiano on messaging the future, increasing compute, & how CO2 impacts your brain

Imagine that – one day – humanity dies out. At some point, many millions of years later, intelligent life might well evolve again. Is there any message we could leave that would reliably help them out...

5 Aug 20192h 11min

#61 - Helen Toner on emerging technology, national security, and China

#61 - Helen Toner on emerging technology, national security, and China

From 1870 to 1950, the introduction of electricity transformed life in the US and UK, as people gained access to lighting, radio and a wide range of household appliances for the first time. Electricit...

17 Juli 20191h 54min

#60 - Phil Tetlock on why accurate forecasting matters for everything, and how you can do it better

#60 - Phil Tetlock on why accurate forecasting matters for everything, and how you can do it better

Have you ever been infuriated by a doctor's unwillingness to give you an honest, probabilistic estimate about what to expect? Or a lawyer who won't tell you the chances you'll win your case? Their beh...

28 Juni 20192h 11min

#59 – Cass Sunstein on how change happens, and why it's so often abrupt & unpredictable

#59 – Cass Sunstein on how change happens, and why it's so often abrupt & unpredictable

It can often feel hopeless to be an activist seeking social change on an obscure issue where most people seem opposed or at best indifferent to you. But according to a new book by Professor Cass Sunst...

17 Juni 20191h 43min

#58 – Pushmeet Kohli of DeepMind on designing robust & reliable AI systems and how to succeed in AI

#58 – Pushmeet Kohli of DeepMind on designing robust & reliable AI systems and how to succeed in AI

When you're building a bridge, responsibility for making sure it won't fall over isn't handed over to a few 'bridge not falling down engineers'. Making sure a bridge is safe to use and remains standin...

3 Juni 20191h 30min

Populärt inom Utbildning

historiepodden-se
rss-bara-en-till-om-missbruk-medberoende-2
det-skaver
harrisons-dramatiska-historia
nu-blir-det-historia
rss-viktmedicinpodden
johannes-hansen-podcast
not-fanny-anymore
roda-vita-rosen
rss-foraldramotet-bring-lagercrantz
sektledare
allt-du-velat-veta
i-vantan-pa-katastrofen
sa-in-i-sjalen
rss-sjalsligt-avkladd
sex-pa-riktigt-med-marika-smith
rss-basta-livet
rikatillsammans-om-privatekonomi-rikedom-i-livet
rss-pa-insidan-med-bjorn-rudman
rss-traningsklubben