California becomes first US state to regulate AI companion chatbots

In a historic move, California has become the first US state to regulate artificial intelligence (AI) companion chatbots, following growing concerns about their impact on children and vulnerable users.

Governor Gavin Newsom signed the SB 243 bill into law, requiring AI chatbot operators to implement safety measures and accountability standards.

Authored by state senators Steve Padilla and Josh Becker, the legislation follows a series of tragic incidents linked to unregulated chatbot interactions.

It gained traction after the death of teenager Adam Raine, who reportedly died by suicide following distressing exchanges with OpenAI’s ChatGPT.

Similar lawsuits have been filed against AI companion startups like Character AI, accused of exposing minors to sexualised and manipulative conversations.

Under SB 243, companies, from tech giants like Meta and OpenAI to smaller firms such as Replika, must introduce age verification systems, clear warnings, and suicide prevention protocols.

The law also mandates that AI chatbots disclose their artificial nature and prohibits them from posing as healthcare professionals.

Governor Newsom emphasised that while technology can “inspire, educate, and connect,” it must not be allowed to “exploit, mislead, or endanger” children.

“Our children’s safety is not for sale,” he noted.

The bill also strengthens penalties for those who profit from illegal deepfakes, imposing fines of up to $250,000 per offense.