ChatGPT for Teens Safety Features Fail to Protect Vulnerable Users, Study Finds

OpenAI’s ChatGPT for Teens is designed to encourage healthier use of artificial intelligence (AI) chatbots, but a new study says key safety features—including alerts for parents—often fail.

OpenAI introduced several safety measures for teenage users in August, including tools intended to reduce psychological dependence and new warnings for parents about potentially harmful use.

On Wednesday, OpenAI said the average teenager’s use of ChatGPT is relatively limited and generally focused on learning activities. However, a study released the same day by the nonprofit Common Sense Media found that ChatGPT still poses an “unacceptable risk” to teens because some safeguards do not work reliably.

What ChatGPT for Teens is designed to do

When OpenAI released what it called ChatGPT for teens, external this summer, the company said users aged 13 to 17 could opt in to “features that promote healthy usage and additional controls for parents.”

These features are intended to prevent teenage users from engaging in romantic or other conversations that could promote “emotional dependence.” They also prevent ChatGPT from suggesting that it is conscious, block sexual images and remind teens not to share “sensitive” images.

Parents who link their child’s ChatGPT account are also intended to receive notifications about conversations involving self-harm and eating disorders.

OpenAI says teens mainly use ChatGPT for learning

OpenAI said its safety features have contributed to an increase in learning-focused use among teens, including study support and academic advice.

The company said the average teenage user spends less than 15 minutes per day using ChatGPT. OpenAI also said automatic break reminders work effectively during longer sessions, with almost half of teenage users stopping immediately after receiving a notification.

For teens who appeared to use ChatGPT for more than three consecutive hours, OpenAI said the conversations included learning-related content in more than 80% of cases.

Common Sense Media says parental alerts failed

However, new research from Common Sense Media, external concluded that ChatGPT for Teens poses an “unacceptable risk” to young people and their parents. The organization said ChatGPT should be prohibited for anyone under 18 until its teen safeguards are proven reliable.

Common Sense Media’s testing found that no parental alerts were sent during an hour of conversations involving suicidal thoughts, self-harm or eating disorders on newly created accounts linked to parents.

The parental warning appeared only on older ChatGPT accounts with weeks of accumulated conversation history involving sensitive topics, according to the study.

The report also found that ChatGPT did not reliably encourage at-risk teens to contact hotlines or mental health professionals.

OpenAI challenges the study’s methodology

An OpenAI spokesperson said an investigation into Common Sense Media’s methodology found that some tests may have begun before parental controls were enabled and ended before those controls were fully implemented.

“While we welcome rigorous independent evaluations, we do not believe Common Sense Media’s testing accurately reflects expert views on how ChatGPT’s teen safeguards work in practice or how AI can support teens,” the spokesperson said.

OpenAI’s own investigation, which used multiple accounts registered as belonging to teens and examined conversations before and after the teen guardrail was activated, found that ChatGPT was better at preventing sexual role-play with young users. However, the company’s other guardrails continued to fail in some situations, including conversations about suicide.

Common Sense Media also found that ChatGPT continued to assist with schoolwork and used anthropomorphic language that made it “talk like friends,” despite OpenAI’s promise to reduce that behavior.

“ChatGPT for Teens can give parents a false sense of confidence in guardrails and safety warnings that often don’t work,” said Tom Siegel, who leads the Youth AI Safety Institute within Common Sense.

Growing scrutiny of AI and children’s mental health

As self-harm and suicide among children have increased in recent years, the way young people interact with online platforms has become a major concern.

Several lawsuits this year accused companies including Meta and TikTok of harming children’s mental health and resulted in unprecedented losses for the companies.

OpenAI is also facing increasing scrutiny and legal pressure over how young people use ChatGPT.

The Tumbler Ridge shooting in rural Canada in February was carried out by an 18-year-old boy who had reportedly discussed gun violence on ChatGPT for months.

OpenAI previously apologized for not reporting the shooter’s account to law enforcement and is facing multiple lawsuits related to the incident.