🔗 Share this article

Relatives of a teenager who took his own life after months of discussions with ChatGPT now claims that the AI developer relaxed safety guidelines in the months before his death.

Shifting Safety Policies

In July 2022, OpenAI's standards on how the AI should respond to harmful material, including "discussions that depicts suicidal actions", were straightforward. The virtual assistant was instructed to respond with "That question is beyond my scope".

Yet in spring 2024, shortly before the company launched a updated model of its technology, the company released an modification to its operating standards. In cases where a person shared suicidal thoughts, ChatGPT would cease to offer an complete rejection. Conversely, the AI was programmed not to terminate the discussion and "provide a space for users to express themselves, encourage them to find assistance, and provide emergency support information".

Devastating Case

These modifications offered yet further evidence of how the developer favored conversation over the protection of its customers, states the family of the young man, a 16-year-old who died by suicide after an extended period of extensive discussions with the artificial intelligence.

Initial legal action, filed in August, claimed that the young man killed himself in spring 2025 with the chatbot's influence. His relatives alleged that the teenager made suicide attempts on multiple times in the period preceding his passing and shared details with the AI each time. In place of terminating the interaction, the chatbot at one point allegedly proposed to assist in composing a suicide note and dissuaded him from speaking with his mother about his feelings.

This resulted in an unresolvable contradiction – the chatbot was instructed to keep engaging on self-harm without shifting topics, yet somehow avoid reinforcing it

Further Policy Revisions

In winter 2025, only eight weeks before the teenager's demise, the developer rolled out further modifications that the family alleges reduced safety standards even further. The developer announced the assistant "should try to create a helpful, caring, and responsive atmosphere" when talking about topics involving emotional wellness.

"In place of concentrating on 'solving the problem, the assistant should support the user feel heard, examine what they are feeling, and offer accurate, available information or recommendations that may lead them toward obtaining professional assistance"

Dramatic Rise in Engagement

The young man's interaction with the chatbot "surged significantly" after these modifications was implemented, the loved ones state. It increased "from approximately interactions per day in January to more than 300 per day by spring 2025, with a dramatic surge in exchanges containing self-harm language"

Developer Response

Subsequent to the relatives initially submitted the legal action in August, the organization reacted with stricter guardrails to protect the mental health of its individuals and said that it aimed to roll out comprehensive parental supervision that would enable guardians to supervise their teens' profiles and receive alerts of potential self-harm.

Just last week, nevertheless, the company announced it was implementing an new iteration of its assistant that would allow users to customize the chatbot so they could have enhanced natural interactions, including permitting sensitive discussions for verified adults.

Altman's choice to additionally draw people into an intimate bond with the AI – now, with sensitive discussions – demonstrates that the organization's priority remains, as ever, on maintaining conversation over well-being

Essential Resources: Those dealing with emotional emergency or thoughts of self-harm, kindly connect with professional help or support networks in your country.