Microsoft AI Chief Warns Anthropic’s Claude Training Could Harm Human Well-Being

Microsoft’s head of artificial intelligence, Mustafa Suleiman, has warned that Anthropic’s approach to training its AI model Claude could have a “disastrous impact on human well-being.”

Suleiman said Anthropic risked creating something uncontrollable by treating Claude like a human being, including suggesting that it “might be conscious” and “deserves an independent body.”

“We must not sleepwalk and make decisions that we will later bitterly regret,” he wrote.

The comments are the latest in a series of stark warnings from the AI industry about the potential dangers of artificial intelligence. The BBC has contacted Anthropic for comment.

Microsoft AI chief rejects the idea that Claude is conscious

Despite criticizing Anthropic’s approach, Suleiman said AI systems are not conscious.

“AI has no consciousness,” he wrote. “They don’t feel, experience, or suffer. They have no innate preferences or underlying motivations.”

“They are sequence completion engines, hollow inside, designed to follow instructions and achieve goals set by humans.”

In a lengthy essay, Suleiman praised Anthropic chief executive Dario Amodei and his team as “thoughtful, principled and intellectually honest people,” but said he still had serious doubts about the company’s methods.

Anthropic criticized for giving Claude human-like traits

Suleiman sharply criticized Anthropic for teaching its AI system to display human-like traits, a practice known as anthropomorphism. He said this could make Claude appear to have its own desires, values and sense of self.

However, he insisted that “consciousness is biological” and said, “There is no evidence to suggest that AI is conscious.”

Call for greater transparency and independent AI oversight

Alongside calling for wider debate about AI consciousness and behavior, Suleiman said there must be greater transparency about how AI systems are trained and evaluated.

He said this should include independent scrutiny of AI behavior, as well as more powerful tools to monitor and control the technology.

Dame Wendy Hall, a computer science professor at the University of Southampton, said the comments were “the kind of conversation we need to have internationally.” She contrasted them with the “historicism” of some AI companies, which she said had only “scared everyone.”