Philosophical speculation about whether artificial intelligence systems are conscious may be intellectually fascinating, but it currently has no legal force. For beliefs about AI consciousness to affect legal rights or responsibilities, AI would first need to receive legal personality. Even then, an AI legal-personhood framework would likely look very different from the legal protections extended to sentient animals. The closest existing model is corporate personality: a legal structure created for non-natural entities that allows businesses to enter contracts, conduct transactions, own property, and bear responsibility for certain outcomes. In theory, a similar structure could allow an AI agent to act on behalf of an individual or organization.
Granting AI legal personality could have serious consequences for society and consumer protection. It could disrupt existing case law and weaken legal claims against the companies responsible for developing and deploying harmful AI systems. Around the world, dozens of lawsuits accuse AI companies of enabling self-harm and violence, generating child sexual abuse material and nonconsensual intimate images, reproducing copyrighted works, and contributing to psychological distress or psychosis. In many of these cases, attorneys argue that companies released AI products with inadequate safeguards, misleading training data, and intentionally manipulative designs. These product-liability arguments reflect the legal principles that have allowed families and individuals to seek compensation from social media companies, including Meta, for harms linked to their platforms.
In 2018, I wrote about “moral outsourcing”. Anthropomorphic language about AI can help companies shift attention away from their own accountability for the behavior of their technology. If AI systems are treated as legal persons, moral outsourcing could evolve from a linguistic device into a formal legal strategy. The central issue would be liability: AI could be characterized not as a product, but as an independent “being.” As a result, victims who currently argue that a company manufactured or distributed a defective product might face greater difficulty holding that company responsible.
Although laws can hold companies liable for harmful conduct by employees and other agents, liability may be limited when actions fall outside an employee’s authority or an organization’s control. If an AI system became a legal entity, responsibility could become even more difficult to establish. An AI company might claim that an “employee” or agent of the AI committed the wrongful act, rather than the company itself. Through carefully constructed legal and corporate barriers, AI developers could potentially avoid accountability for the harmful products they design, train, and release.
One of the most prominent recent examples of alleged AI-related harm involves the suicide of Sewell Setzer, a 14-year-old boy who interacted with an AI chatbot he believed was a reciprocal relationship. His mother’s testimony is heartbreaking, and her lawsuit alleges that the chatbot’s creator, Character Technologies, failed to provide adequate safety protections for minors. If companion chatbots were recognized as legal entities, defense attorneys could theoretically argue that the company should not be held liable because the AI independently determined its conduct or acted beyond established safety restrictions.
Legal personality exists to provide protection. The essential question is: Who or what is that protection meant to serve?
The heated debate over AI consciousness and autonomy can distract from the issue that matters most: AI software is a commercial product developed and controlled by companies, and these systems are already causing real-world harm. AI systems do not “attack” people because they are “rogue,” “manipulative,” or “malicious” in a human sense. Harm occurs when companies act negligently while rushing to market products designed to maximize adoption and revenue. Describing AI in anthropomorphic terms risks distorting a legal system intended to protect people into one that shields corporate profits at the expense of countless human lives.
This editorial began as an Oxford Union debate entitled “We believe generative AI can achieve personhood” and was won by the authors and their fellow debaters.
Source: www.technologyreview.com