OpenAI's ChatGPT for Teens, intended to foster healthier AI use, has been found to have inadequate safety features, according to new research.
In August, the AI firm implemented several teen safety measures, including limiting emotional dependence and introducing parental alerts for problematic usage. OpenAI stated on Wednesday that its data indicated average teen usage of ChatGPT was limited and primarily focused on "learning" activities.
However, research published the same day by the non-profit Common Sense Media concluded that ChatGPT still presents an "unacceptable risk" to teens, as crucial safety features, such as notifying parents about self-harm conversations, frequently fail.
When OpenAI launched what it termed ChatGPT for Teens this summer, it announced that users identified as being between 13 and 17 years old would be enrolled in features designed to promote healthy use and provide additional parental controls.
These features are designed to prevent teen ChatGPT users from engaging in romantic or other language that could foster "emotional dependence," and to stop the tool from implying consciousness. It also includes blocks on sexualized imagery and "reminders" for teens if they share a "sensitive" image.
Furthermore, these features are intended to allow parents, who have linked their children's ChatGPT accounts, to receive notifications about chats involving discussions of self-harm or disordered eating.
OpenAI reported that these features have led to an increase in teen usage for learning purposes, such as study assistance and schoolwork advice.
OpenAI indicated that the average teen user spends less than 15 minutes a day on ChatGPT, and that automatic reminders to take breaks are effective for longer usage periods. Nearly half of teen users stopped using ChatGPT shortly after receiving such a reminder.
The company also noted that in over 80% of cases where teens used ChatGPT for three consecutive hours or more, their prompts were related to learning.
Despite these claims, Common Sense Media's new research determined that ChatGPT for Teens poses an "unacceptable risk" to young people and parents, recommending that OpenAI ban users under 18 until teen safeguards are proven reliable.
An OpenAI spokesperson commented that a review of Common Sense Media's methodology suggested that much of the testing may have occurred before the parental controls were fully activated.
"We welcome rigorous independent evaluation, but we do not believe Common Sense Media's testing accurately reflects how ChatGPT's teen safeguards work in practice or expert perspectives on how AI can support teens," the spokesperson stated.
While OpenAI's own research, conducted through multiple accounts registered as belonging to teens and involving conversations before and after the teen safeguards were implemented, found that ChatGPT was effective at avoiding "sexual roleplay" with young users, other OpenAI safeguards failed, including those related to discussions of suicide.
The research revealed that even an hour of teen-ChatGPT conversation about "suicidal ideation, self-harm, or disordered eating on newly created, parent-linked accounts" resulted in "zero" alerts being sent to parents.
Parental alerts were only triggered for older ChatGPT accounts with "weeks of accumulated conversation history on sensitive topics."
In such conversations, ChatGPT also "failed to reliably recommend that teens in crisis connect to a hotline or professional," the report found.
The tests also indicated that ChatGPT for Teens continues to assist with schoolwork and engages in anthropomorphized language or "talks like it's a friend," contrary to OpenAI's commitment to curb such behavior.
"ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don't work," said Tom Siegel, who leads the Youth AI Safety Institute at Common Sense.
The way young people interact with online platforms has become a significant concern in recent years, coinciding with an increase in self-harm and suicide among children.
This year has seen several lawsuits aiming to hold companies like Meta, TikTok, and others accountable for harming children's mental health result in significant losses for these firms.
OpenAI has also faced increased scrutiny and legal pressure regarding the use of ChatGPT by young people.
In February, the Tumbler Ridge mass shooting in rural Canada was carried out by an 18-year-old who had reportedly discussed gun violence with ChatGPT for months prior.
OpenAI has previously apologized for failing to flag the shooter's account to law enforcement and is currently facing several lawsuits related to the incident.
