Anthropic is presenting itself simultaneously as a safety-focused AI company, a profit-driven technology giant and a leader in the race to develop superhuman systems. That combination is at the center of a debate over whether the company’s assurances can be trusted—and how much society should rely on self-regulation when powerful AI companies propose their own rules.
What is Anthropic?
It is worth pausing to ask a deceptively difficult question: “What is humanity?” The question has been on my mind since I read an exchange on X posted last week by Nathan Calvin, chief strategy officer at Encode AI, a Washington, D.C.-based organization that advocates for regulation.
Calvin was dissatisfied with a bill proposed by Anthropic and secretly circulated to U.S. lawmakers before it leaked.
Calvin argued that people want to be seen as good, but questioned why Anthropic should be trusted to act that way. “What is anthropic?” he asked.
One answer is that Anthropic is a company valued at more than $1.5 trillion in its IPO, making its co-founders, including Clark and Oler, enormously wealthy.
A company with several identities
Anthropic has always claimed to be more than a conventional technology company. As Calvin pointed out, it could be understood as an AI safety organization focused on developing powerful technology that is reliable and almost risk-free.
It could also be viewed as a group determined to become the first to develop technology with superhuman abilities—perhaps because its members believe they are the most responsible people to do so, or perhaps because of ambition and ego.
In practice, Anthropic represents all of these things. CEO Dario Amodei responded to Calvin’s post in an online reply:
“Our strategy has always been what I’ve called a ‘race to the top.’ By showing how it’s possible to build technology responsibly, we can raise the bar across the industry. The forces that drive a race to the top are most effective when the companies that do it are at the forefront. This is one of the lessons I’ve learned over and over again. And when you look at that strategy as a whole, I’m proud of what we’ve done and I support it.”
The tension between safety and profit
Amodei’s explanation sounds definitive, but occupying multiple roles is difficult. The priorities of AI safety organizations can conflict significantly with those of profit-driven technology companies. In his post, Amodei also acknowledges that he made a mistake.
Ultimately, the story Anthropic tells about itself should help us understand the story it tells the public. That understanding should inform decisions about how much to rely on self-regulation and whether external safety testers should be given access to its models on terms set independently of the company.
It should also help determine how to respond when a bill that appears self-serving comes to light.
Instead, the company’s identity can look like a many-headed hydra. Anthropic is many things to many people, but that does not necessarily make it more credible. In many ways, a conventional technology company would be easier to understand without the science-fiction role-playing and internal disputes.
Source: www.technologyreview.com