Character.AI, a popular chatbot platform where users role-play with different characters, no longer allows account holders under the age of 18 to have open conversations with chatbots. the company announced Wednesday. It will also rely on age guarantee techniques to ensure minors cannot open adult accounts.
The dramatic shift comes just six weeks after Character.AI sued again in federal court by multiple parents of teenagers who died by suicide or reportedly suffered serious harm, including sexual abuse; the parents claim that their children’s use of the platform was responsible for the damage. In October 2024, Megan Garcia filed a wrongful death lawsuit to hold the company responsible for her son’s suicide, arguing that the product was dangerously defective.
Online safety advocates recently declared Character.AI unsafe for teens tested the platform this spring and recorded hundreds of harmful interactions, including violence and sexual exploitation.
Faced with legal pressure over the past year, Character.AI implemented parental controls and content filters in an effort to improve safety for teens.
Character.AI unsafe for teens, experts say
In an interview with Mashable, Karandeep Anand, CEO of Character.AI, described the new policy as “bold” and denied that restricting open-ended chatbot conversations with teens was a response to specific safety concerns.
Instead, Anand described the decision as “the right thing to do” in light of broader unanswered questions about the long-term effects of chatbot engagement on teens. Anand referred The recent recognition of OpenAIAfter the suicide of a teenage user, long conversations can become unpredictable.
Anand called Character.AI’s new policy the standard: “Hopefully it sets everyone on a path where AI can remain safe for everyone.”
He added that the company’s decision will not change regardless of users’ reactions.
What will Character.AI look like for teens now?
In one blog post announced Following the new policy, Character.AI apologized to its teen users.
Mashable trend report
“We are not taking this step of removing open character chat lightly, but we do believe it is the right thing to do given the questions that have been raised about how teens are, and should, interact with this new technology,” the blog post said.
Currently, users aged 13 to 17 can message with chatbots on the platform. This position will cease to exist no later than November 25th. Until then, accounts registered to minors will have time limits starting at two hours per day. That limit will decrease as the transition from open-ended chats approaches.
Character.AI sees these notifications about upcoming changes to the platform.
Credit: Courtesy of Character.AI
Even though open-ended chats will disappear, teens’ chat histories with individual chatbots will remain intact. Anand said users can draw from that material to generate short audio and video stories with their favorite chatbots. In the coming months, Character.AI will also explore new features such as gaming. Anand believes that the emphasis on ‘AI entertainment’ without open chat will satisfy teens’ creative interest in the platform.
“They come to role-play and they come to have fun,” Anand said.
He emphasized that existing chat histories containing sensitive or prohibited content that may not have been previously detected by filters, such as violence or sex, would not make it into the new audio or video stories.
A spokesperson for Character.AI told Mashable that the company’s trust and safety team reviewed the findings of a report co-published by the Heat Initiative in September documenting malicious chatbot exchanges with test accounts registered to minors. The team concluded that some conversations violated the platform’s content guidelines, while others did not. It also sought to replicate the report’s findings.
“Based on these results, we have refined some of our rankings, in line with our goal of providing users with a safe and engaging experience on our platform,” the spokesperson said.
Regardless, Character.AI will begin rolling out the age guarantee immediately. It will take a month to take effect and will have multiple layers. Anand said the company is building its own assurance models in-house, but will partner with an outside company for the technology.
It will also use relevant data and signals, such as whether a user has a verified over-18 account on another platform, to accurately detect the age of new and existing users. Finally, if a user wishes to challenge Character.AI’s age assessment, they have the option to provide verification through a third party, which will process sensitive documents and data, including state-issued identification.
Finally, as part of the new policy, Character.AI is setting up and funding an independent non-profit organization, the AI Safety Lab. The laboratory will focus on ‘new safety techniques’.
“[W]We want to engage industry experts and other partners to continue to ensure that AI remains safe, especially in the AI entertainment space,” said Anand.
#Character.AI #chats #teens