EU Proposes New Rules for AI Chatbots and Children

EU plans In an effort to prevent children from becoming emotionally dependent on technology, chatbots would be prohibited from simulating emotions or interpersonal relationships when interacting with people under 18.

The rules would become part of the EU Kids Act proposed on Thursday, which includes similar restrictions on minors’ access to social media and video-sharing platforms. Children under 13 would be prohibited from accessing chatbots without parental supervision. Digital platforms would also be required to include child-safety features in their products or face fines of up to 6 percent of their global sales, according to a draft bill obtained by WIRED.

Chatbots operating within the EU would also not be allowed to retain memories of individual conversations under the proposed rules. The European Commission is concerned that retaining memories could allow systems to accumulate sensitive data about minors over time and reinforce harmful patterns of interaction.

Platforms Could Face Restrictions on AI Companions

The proposed law would also stipulate that online social networking sites, video-sharing platforms, and gaming providers cannot automatically enable or encourage the use of AI chatbots on their services. This includes not pinning AI companions or general conversational chatbots to the top of minors’ contact lists, as Snapchat has done.

The proposal is limited to AI companions and general conversational chatbots that the commission deems to pose a significant risk to the health, safety, or well-being of minors. Simeon de Brouwer, a policy advisor at the Brussels-based advocacy group Europe Digital Rights, told WIRED that while he welcomed the anti-chatbot measures, he questioned why they would be limited to young people.

“Shouldn’t we all be protected from AI that is designed to be addictive, that creates emotional dependence, that is enabled by default and forced on users?”

The EU says its Digital Fairness Act, expected to be published later this year, will propose a broader crackdown on addictive designs.

EU Raises Concerns About AI Advice and Children’s Mental Health

In a document obtained by WIRED, the European Commission said it was concerned that reliance on chatbots could negatively affect children’s development and relationships with other humans. The commission also raised concerns that chatbots are giving minors dangerous advice about topics such as mental health and self-improvement, potentially shaping and encouraging dangerous behavior.

“Platforms have been playing fast and loose with children’s mental health,” President Ursula von der Leyen said at a press conference on Thursday.

Almost half of Europeans aged 11 to 25 have discussed intimate or personal matters with an AI chatbot, according to an Ipsos BVA study conducted across France, Germany, Sweden, and Ireland. Australia and the UK have both sought to limit minors’ access to chatbots through social media bans.

EU Proposal Also Targets AI-Generated Deepfakes

The document also highlighted the harm caused by deepfakes generated by chatbots. Von der Leyen addressed the issue in Wednesday’s State of the Union address, saying, “Girls shouldn’t have to accept having their photos used in AI-generated sexual images…Europe has the power to act.”

EU AI legislation has required synthetic content to be labeled as AI-generated since August, but the rule currently applies only to deepfakes generated for commercial purposes.

Study Links AI Chatbot Use With Lower Test Scores

The European Commission also blamed digital distraction for the decline in young people’s basic skills. A study spanning 91 countries by the Organization for Economic Co-operation and Development found that children who reported using AI chatbots for schoolwork had lower test scores than those who did not.

The study found that concerns about cognitive impacts and de-skilling “make it clear that children’s rights are not present in the design of GenAI’s tools,” researchers from the independent network EU Kids Online said.

I wrote in the report.