Anthropic Updates AI Policy to Ban Sustained and Unnecessary Abuse

Anthropic has raised eyebrows after announcing that it will ban “sustained and unnecessary” abuse of its AI tools.

The maker of Claude announced Thursday that it would update its usage policy to address “cruel” interactions with its tools, while acknowledging that it had permitted such interactions in “rare” cases before.

What Anthropic’s Updated AI Policy Prohibits

The list of prohibited behaviors will now include bullying others, promoting self-harm and creating intimate images without consent.

Anthropic said the policy would apply only in “extreme cases” involving repeated abuse. However, the update has reignited debate about how people should communicate with AI tools.

The company said the provision does not apply to “generic versions of user complaints, backlash, dark creative themes, or model testing and research.” This suggests the policy would apply only when abusive behavior is clearly intentional.

However, it remains unclear what specific forms of “abusive or cruel conduct” Anthropic will consider unacceptable when directed at its AI models.

Online Debate Over AI Abuse Rules

Screenshots of the new policy have been widely shared and discussed on social media. Some people praised the update as a way to promote good manners and “exemplary welfare.”

Others criticized the plan. One commenter called it “terribly wrong”, external, while some suggested that the policy could undermine efforts to address abuse against humans and animals.

“Don’t be cruel to numbers and math,” wrote Dr. Barry Scannell, technology partner at Irish law firm William Fry, on LinkedIn, external.

“This level of anthropomorphism of AI is harmful. It makes people believe that the AI is something that is not real.”

Criticism of Anthropic’s AI Anthropomorphism

Microsoft’s AI director Mustafa Suleiman, who criticized rival company Anthropic in September for treating AI as if it were human, responded to the policy update with a melting face emoji.