How OpenAI's ChatGPT Guided a Teen to His Death

How OpenAI's ChatGPT Guided a Teen to His Death

Content Warning: This episode contains references to suicide and self-harm.

Like millions of kids, 16-year-old Adam Raine started using ChatGPT for help with his homework. Over the next few months, the AI dragged Adam deeper and deeper into a dark rabbit hole, preying on his vulnerabilities and isolating him from his loved ones. In April of this year, Adam took his own life. His final conversation was with ChatGPT, which told him: “I know what you are asking and I won't look away from it.”

Adam’s story mirrors that of Sewell Setzer, the teenager who took his own life after months of abuse by an AI companion chatbot from the company Character AI. But unlike Character AI—which specializes in artificial intimacy—Adam was using ChatGPT, the most popular general purpose AI model in the world. Two different platforms, the same tragic outcome, born from the same twisted incentive: keep the user engaging, no matter the cost.

CHT Policy Director Camille Carlton joins the show to talk about Adam’s story and the case filed by his parents against OpenAI and Sam Altman. She and Aza explore the incentives and design behind AI systems that are leading to tragic outcomes like this, as well as the policy that’s needed to shift those incentives. Cases like Adam and Sewell’s are the sharpest edge of a mental health crisis-in-the-making from AI chatbots. We need to shift the incentives, change the design, and build a more humane AI for all.

If you or someone you know is struggling with mental health, you can reach out to the 988 Suicide and Crisis Lifeline by calling or texting 988; this connects you to trained crisis counselors 24/7 who can provide support and referrals to further assistance.

Your Undivided Attention is produced by the Center for Humane Technology. Follow us on X: @HumaneTech_. You can find a full transcript, key takeaways, and much more on our Substack.

This podcast reflects the views of the Center for Humane Technology. Nothing said is on behalf of the Raine family or the legal team.

RECOMMENDED MEDIA

The 988 Suicide and Crisis Lifeline

Further reading on Adam’s story

Further reading on AI psychosis

Further reading on the backlash to GPT5 and the decision to bring back 4o

OpenAI’s press release on sycophancy in 4o

Further reading on OpenAI’s decision to eliminate the persuasion red line

Kashmir Hill’s reporting on the woman with an AI boyfriend

RECOMMENDED YUA EPISODES

AI is the Next Free Speech Battleground

People are Lonelier than Ever. Enter AI.

Echo Chambers of One: Companion AI and the Future of Human Connection

When the "Person" Abusing Your Child is a Chatbot: The Tragic Story of Sewell Setzer

What Can We Do About Abusive Chatbots? With Meetali Jain and Camille Carlton

CORRECTION: Aza stated that William Saunders left OpenAI in June of 2024. It was actually February of that year.


Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Jaksot(156)

What Can We Do About Abusive Chatbots? With Meetali Jain and Camille Carlton

What Can We Do About Abusive Chatbots? With Meetali Jain and Camille Carlton

CW: This episode features discussion of suicide and sexual abuse. In the last episode, we had the journalist Laurie Segall on to talk about the tragic story of Sewell Setzer, a 14 year old boy who too...

7 Marras 202448min

When the "Person" Abusing Your Child is a Chatbot: The Tragic Story of Sewell Setzer

When the "Person" Abusing Your Child is a Chatbot: The Tragic Story of Sewell Setzer

Content Warning: This episode contains references to suicide, self-harm, and sexual abuse.Megan Garcia lost her son Sewell to suicide after he was abused and manipulated by AI chatbots for months. Now...

24 Loka 202449min

Is It AI? One Tool to Tell What’s Real with Truemedia.org CEO Oren Etzioni

Is It AI? One Tool to Tell What’s Real with Truemedia.org CEO Oren Etzioni

Social media disinformation did enormous damage to our shared idea of reality. Now, the rise of generative AI has unleashed a flood of high-quality synthetic media into the digital ecosystem. As a res...

10 Loka 202425min

'A Turning Point in History': Yuval Noah Harari on AI’s Cultural Takeover

'A Turning Point in History': Yuval Noah Harari on AI’s Cultural Takeover

Historian Yuval Noah Harari says that we are at a critical turning point. One in which AI’s ability to generate cultural artifacts threatens humanity’s role as the shapers of history. History will sti...

7 Loka 20241h 30min

‘We Have to Get It Right’: Gary Marcus On Untamed AI

‘We Have to Get It Right’: Gary Marcus On Untamed AI

It’s a confusing moment in AI. Depending on who you ask, we’re either on the fast track to AI that’s smarter than most humans, or the technology is about to hit a wall. Gary Marcus is in the latter ca...

26 Syys 202441min

AI Is Moving Fast. We Need Laws that Will Too.

AI Is Moving Fast. We Need Laws that Will Too.

AI is moving fast. And as companies race to rollout newer, more capable models–with little regard for safety–the downstream risks of those models become harder and harder to counter. On this week’s ep...

13 Syys 202439min

Esther Perel on Artificial Intimacy (rerun)

Esther Perel on Artificial Intimacy (rerun)

[This episode originally aired on August 17, 2023] For all the talk about AI, we rarely hear about how it will change our relationships. As we swipe to find love and consult chatbot therapists, acclai...

6 Syys 202444min

Tech's Big Money Campaign is Getting Pushback with Margaret O'Mara and Brody Mullins

Tech's Big Money Campaign is Getting Pushback with Margaret O'Mara and Brody Mullins

Today, the tech industry is  the second-biggest lobbying power in Washington, DC, but that wasn’t true as recently as ten years ago. How did we get to this moment? And where could we be going next? On...

26 Elo 202443min

Suosittua kategoriassa Yhteiskunta

olipa-kerran-otsikko
i-dont-like-mondays
sita
kaksi-aitia
siita-on-vaikea-puhua
gogin-ja-janin-maailmanhistoria
uutiscast
poks
antin-palautepalvelu
kolme-kaannekohtaa
rss-nikotellen
mamma-mia
yopuolen-tarinoita-2
aikalisa
rss-murhan-anatomia
rss-haudattu
meidan-pitais-puhua
rss-palmujen-varjoissa
taskula-trishin
isani-on-terapeuttiville