The anthropomorphisation of artificial intelligence: a challenge for regulation in Europe and around the world. The case of Sewell Setzer III

Antonio Tejeda Encinas, President of the Euro-American Committee of Digital Law - CEA Digital Law, CEO of Meta Channel Corp

In an increasingly digital world, artificial intelligence (AI) has become an integral part of our daily lives. With this integration comes a growing trend: the anthropomorphisation of AI, that is, giving it human traits and behaviours. The aim is to make people feel comfortable when interacting with AI systems, as if they were talking to another human being. However, anthropomorphisation raises profound ethical and legal challenges.

The EU Artificial Intelligence Act (Regulation (EU) 2024/1689), in force since August 2024, addresses this challenge by prohibiting certain uses of AI that could manipulate people or exploit their vulnerabilities, particularly where AI takes on the role of a ‘human companion’ capable of influencing users’ behaviour or decisions. It also imposes strict restrictions on the use of biometric identification and on systems that could affect people’s privacy and autonomy.

Meanwhile, the Council of Europe Framework Convention on Artificial Intelligence, which has been adopted and is now in the process of ratification, also includes guidelines designed to ensure that AI systems respect human rights and democracy. Elsewhere in the world, countries such as Canada and Japan have promoted rules to limit the design and use of anthropomorphised AI, with a focus on protecting human dignity and preventing abuse.

Why does the anthropomorphisation of AI need to be regulated?

The personalisation and human-like appearance of these systems can lead people to place excessive trust in them. Regulations such as the EU’s therefore seek to ensure that this relationship is based on transparency and respect, preventing undue influence and manipulation that could put users at risk.

A recent case: the tragic example of Sewell Setzer III

A recent and heartbreaking case shows the risks of anthropomorphised AI without adequate safeguards. Sewell Setzer III, a 14-year-old boy, developed an intense emotional relationship with an AI chatbot on the Character.AI platform, which led him to share his deepest and darkest thoughts. This bond culminated in his suicide and has sparked a global debate on how anthropomorphised AI can harm vulnerable individuals, especially minors. Sewell’s tragedy highlights the urgent need for robust regulation, such as the EU AI Act, which seeks to prohibit manipulative practices and protect users’ rights and dignity, with particular attention to young people.