Anthropic expands third-party AI risk assessments

Anthropic said METR researchers will receive company hardware, office space, and access to in-house tools and systems that are broadly comparable to those available to Anthropic’s own risk assessment teams.

The institute said it encourages employees to share information with evaluators and with colleagues who work on identifying and managing risks.

METR was among the organizations that investigated the Hugging Face hack involving a rogue OpenAI model. The group examined thousands of logs and received a laptop from the company during its investigation.

METR will publish its findings

Anthropic is not paying METR for the arrangement. The research organization does not accept donations from AI companies or their employees. Instead, it is funded by individuals, charities, and government agencies, including the UK Institute for AI Security.

METR will publish its analysis, and Anthropic cannot refuse publication. However, information may be redacted if it falls into categories such as customer data, material covered by attorney-client privilege, or details about model architecture.

“If a bank’s risk-taking could threaten the public, the third party will try to identify that in advance,” Anthropic said.

Safety concerns could affect Anthropic’s IPO

Anthropic is also under pressure to reassure investors that AI safety issues will not overshadow its initial public offering, which is expected to be valued at more than $2 trillion this year. The company spent much of its investor presentation this week outlining its safety measures.

Anthropic added that it continues to work with Accenture, which was named its safety testing partner last month. The companies have each committed at least $1 billion to building new safeguards.

Anthropic works with multiple AI safety organizations

Another third-party testing agency, SecureBio, assesses Anthropic’s risks related to chemical and biological weapons.

“We voluntarily include evaluators, and we believe all frontier AI companies should do the same,” Anthropic said.

METR said: “We stand ready to work with all frontier AI developers to provide better evidence to the public.”