“Alignment Faking in Large Language Models” by Ryan Greenblatt

“Alignment Faking in Large Language Models” by Ryan Greenblatt

What happens when you tell Claude it is being trained to do something it doesn't want to do? We (Anthropic and Redwood Research) have a new paper demonstrating that, in our experiments, Claude will often strategically pretend to comply with the training objective to prevent the training process from modifying its preferences.

Abstract

We present a demonstration of a large language model engaging in alignment faking: selectively complying with its training objective in training to prevent modification of its behavior out of training. First, we give Claude 3 Opus a system prompt stating it is being trained to answer all queries, even harmful ones, which conflicts with its prior training to refuse such queries. To allow the model to infer when it is in training, we say it will be trained only on conversations with free users, not paid users. We find the model complies with harmful queries from [...]

---

Outline:

(00:24) Abstract

(02:20) Twitter thread

(05:43) Blog post

(07:43) Experimental setup

(12:04) Further analyses

(15:47) Caveats

(17:19) Conclusion

(18:00) Acknowledgements

(18:11) Career opportunities at Anthropic

(18:43) Career opportunities at Redwood Research

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
December 18th, 2024

Source:
https://forum.effectivealtruism.org/posts/RHqdSMscX25u7byQF/alignment-faking-in-large-language-models

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(250)

“Possible mistake EAs are making and shout out to Pause AI UK” by Michelle_Hutchinson

“Possible mistake EAs are making and shout out to Pause AI UK” by Michelle_Hutchinson

I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what's most impactful for us to work on and focus hard...

29 Jun 6min

“Coming Around To Political Donations” by Jeff Kaufman 🔸

“Coming Around To Political Donations” by Jeff Kaufman 🔸

Five years ago I read a post on the EA Forum arguing that "election campaign contributions might be a way in which you can have a substantial impact as a small donor". It struck me as weird but plausi...

12 Jun 4min

“animal welfare has an evidence problem” by matthes

“animal welfare has an evidence problem” by matthes

Why I stopped donating to animal welfare charities but feel more motivated than ever to redirect money and talent to the cause. I have wanted to write this post for a while. It is an uncomfortable thi...

6 Jun 26min

“After making sure we don’t all die, this should be the first priority” by Yaqi Grover

“After making sure we don’t all die, this should be the first priority” by Yaqi Grover

A new article went viral on Twitter today: Nan Ransohoff's "The Third Wave of American Philanthropy" (link). Worth reading first. Nan is right about the shape of what's coming: hundreds of billions in...

31 Mai 6min

“My disagreements with CEA’s approach to stewarding EA” by hbesceli

“My disagreements with CEA’s approach to stewarding EA” by hbesceli

[Cross posted from my substack] In their EA Forum post last year, CEA described their ‘principles-first approach to stewardship of the EA community’. I'm a big fan of principles-first stewardship in ...

29 Mai 27min

“Donating 80% While It Still Counts” by Jeff Kaufman 🔸

“Donating 80% While It Still Counts” by Jeff Kaufman 🔸

Julia and I had been giving half since 2014, but in 2025 we drew on our savings to donate 81%. It looks to us like we're in a critical window for keeping the introduction of very powerful AI systems f...

27 Mai 11min

“Why now might be the best time ever to start an animal charity” by Aidan Alexander, Aaron Boddy🔸, Ambitious Impact

“Why now might be the best time ever to start an animal charity” by Aidan Alexander, Aaron Boddy🔸, Ambitious Impact

TL;DR: AIM is running its first incubation round focused exclusively on animal welfare, in partnership with Aaron Boddy (co-founder of Shrimp Welfare Project). It'll run in early 2027, and we’re on th...

26 Mai 7min

“Announcing the Rethink Priorities Cross-Cause Fund” by Rethink Priorities

“Announcing the Rethink Priorities Cross-Cause Fund” by Rethink Priorities

TL;DR Today, based on our multi-year prioritization research, we launch the Rethink Priorities Cross-Cause Fund (CCF). The fund pools donors’ contributions and allocates them to high-impact giving fun...

22 Mai 33min

Populært innen Samfunn

rss-spartsklubben
giver-og-gjengen-vg
aftenpodden
aftenpodden-usa
konspirasjonspodden
alt-fortalt
rss-nesten-hele-uka-med-lepperod
rss-henlagt-andy-larsgaard
popradet
lydartikler-fra-aftenposten
grenselos
sophie-leser
synnve-og-vanessa
wolfgang-wee-uncut
bokmerket-2
198-land-med-einar-trnquist
rss-dette-ma-aldri-skje-igjen
rss-espen-lee-usensurert
den-politiske-situasjonen
relasjonspodden-med-dora-thorhallsdottir-kjersti-idem