Emergency pod: Did OpenAI give up, or is this just a new trap? (with Rose Chan Loui)

Emergency pod: Did OpenAI give up, or is this just a new trap? (with Rose Chan Loui)

When attorneys general intervene in corporate affairs, it usually means something has gone seriously wrong. In OpenAI’s case, it appears to have forced a dramatic reversal of the company’s plans to sideline its nonprofit foundation, announced in a blog post that made headlines worldwide.

The company’s sudden announcement that its nonprofit will “retain control” credits “constructive dialogue” with the attorneys general of California and Delaware — corporate-speak for what was likely a far more consequential confrontation behind closed doors. A confrontation perhaps driven by public pressure from Nobel Prize winners, past OpenAI staff, and community organisations.

But whether this change will help depends entirely on the details of implementation — details that remain worryingly vague in the company’s announcement.

Return guest Rose Chan Loui, nonprofit law expert at UCLA, sees potential in OpenAI’s new proposal, but emphasises that “control” must be carefully defined and enforced: “The words are great, but what’s going to back that up?” Without explicitly defining the nonprofit’s authority over safety decisions, the shift could be largely cosmetic.

Links to learn more, video, and full transcript: https://80k.info/rcl4

Why have state officials taken such an interest so far? Host Rob Wiblin notes, “OpenAI was proposing that the AGs would no longer have any say over what this super momentous company might end up doing. … It was just crazy how they were suggesting that they would take all of the existing money and then pursue a completely different purpose.”

Now that they’re in the picture, the AGs have leverage to ensure the nonprofit maintains genuine control over issues of public safety as OpenAI develops increasingly powerful AI.

Rob and Rose explain three key areas where the AGs can make a huge difference to whether this plays out in the public’s best interest:

  1. Ensuring that the contractual agreements giving the nonprofit control over the new Delaware public benefit corporation are watertight, and don’t accidentally shut the AGs out of the picture.
  2. Insisting that a majority of board members are truly independent by prohibiting indirect as well as direct financial stakes in the business.
  3. Insisting that the board is empowered with the money, independent staffing, and access to information which they need to do their jobs.

This episode was originally recorded on May 6, 2025.

Chapters:

  • Cold open (00:00:00)
  • Rose is back! (00:01:06)
  • The nonprofit will stay 'in control' (00:01:28)
  • Backlash to OpenAI’s original plans (00:08:22)
  • The new proposal (00:16:33)
  • Giving up the super-profits (00:20:52)
  • Can the nonprofit maintain control of the company? (00:24:49)
  • Could for profit investors sue if profits aren't prioritised? (00:33:01)
  • The 6 governance safeguards at risk with the restructure (00:34:33)
  • Will the nonprofit’s giving just be corporate PR for the for-profit? (00:49:12)
  • Is this good, or not? (00:51:06)
  • Ways this could still go wrong – but reasons for optimism (00:54:19)

Video editing: Simon Monsour and Luke Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Music: Ben Cordell
Transcriptions and web: Katy Moore

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(359)

How to get into AI safety in 3 months (article reading by career advisor Matt Beard)

How to get into AI safety in 3 months (article reading by career advisor Matt Beard)

The AI safety field has grown by about 30x in the last eight years. Yet it is still in need of more people. Many want to help, but don’t know where to start. As an advisor at 80,000 Hours, Matt Beard ...

15 Okt 18min

OpenAI Security: Controlling Models is Now ‘Hell’ (AI Explained cross-post)

OpenAI Security: Controlling Models is Now ‘Hell’ (AI Explained cross-post)

This is an (unpaid) cross-post of a video from the podcast 'AI Explained', which Rob Wiblin thought you might be interested in.You can find the original on YouTube here. If you like it and want more s...

9 Okt 40min

In 2023 Ajeya Cotra already knew what was coming (classic episode)

In 2023 Ajeya Cotra already knew what was coming (classic episode)

We're rereleasing some of older episodes that seem more relevant than ever with new rogue AI incidents now seemingly announced every day. Ajeya was one of three independent investigators into the Hugg...

8 Okt 2h 46min

19 Astra and 'Hugging Face' details that reveal what's coming next | Rob Wiblin

19 Astra and 'Hugging Face' details that reveal what's coming next | Rob Wiblin

OpenAI’s rogue agent swarm was eventually caught hacking Hugging Face for a simple reason: it wasn’t trying to hide from us at all. What could a swarm that wants to stay hidden get away with?Host Rob ...

2 Okt 20min

The case for giving AI (some) legal rights | Simon Goldstein

The case for giving AI (some) legal rights | Simon Goldstein

It sounds like the worst idea in the world: pay AIs, let them own property, give them rights. But AI ethics and safety researcher Simon Goldstein thinks it might actually be the best way to keep human...

1 Okt 1h 53min

Will AI take power — or will humans use it to take power first? With Katja Grace and Tom Davidson

Will AI take power — or will humans use it to take power first? With Katja Grace and Tom Davidson

In our first-ever debate, we asked two leading AI risk researchers which catastrophe we should fear most: misaligned AI seizing control from humans, or a small group of humans using AI to seize power....

29 Sep 1h 28min

How we get from AI cyberattacks to human extinction

How we get from AI cyberattacks to human extinction

You’ve seen the headlines: AI could kill us all. Think it sounds ridiculous? So did host Luisa Rodriguez, until she tried to pick apart the arguments. She starts with the motive: why would AI ‘want’ t...

24 Sep 27min

#254 – Max Nadeau on why ambitious people should start AI safety nonprofits

#254 – Max Nadeau on why ambitious people should start AI safety nonprofits

There are millions available for anyone who can launch a successful nonprofit AI safety startup. The hard part, it turns out, is finding people to take the money. Coefficient Giving has drawn up a lis...

17 Sep 1h 3min

Populärt inom Utbildning

husmodern-med-amanda-schulman
rss-bara-en-till-om-beroende-medberoende
historiepodden-se
det-skaver
nu-blir-det-historia
sektledare
rss-viktmedicinpodden
allt-du-velat-veta
harrisons-dramatiska-historia
not-fanny-anymore
rss-dr-bjorklund
roda-vita-rosen
sa-in-i-sjalen
johannes-hansen-podcast
rosceremoni
rss-ai-med-jonas-benjamin
i-vantan-pa-katastrofen
rss-foraldramotet-bring-lagercrantz
rikatillsammans-om-privatekonomi-rikedom-i-livet
kan-jag-sa-kan-du-podden