Anthropic Expands Pause on Internal AI Assessments After Low-Risk Testing Incident

Anthropic said its investigation did not find “incidents of similar severity to the cybersecurity incidents we reported this summer.” The investigation then focused on documenting “low-risk” testing scenarios, including one that affected the Philadelphia Police Department.

“While the impact of these actions was minimal and we had already turned off live internet access for some high-risk and cybersecurity assessments, we have decided to expand to include all internal assessments until we are certain that our security and monitoring measures … reliably capture such actions,” Anthropic said.

Anthropic says new tools block unintended AI behavior

Anthropic said it has stopped running some assessments and moved others to offline versions. It also restructured certain tasks so they no longer reach live websites.

The company said it has “updated guardrails for some internet access tools, such as web fetch tools, to significantly limit what they can do with their models.”

Anthropic also said it has “built new tools to automatically detect and block the types of behavior described above.” The company said it tested the tools against the cases described in this post and that they were all blocked.

White House AI task force calls for transparency

The White House’s AI task force, known as the Superintelligence Unit, said it told Anthropic that it “expects immediate and full transparency with relevant parties and the public.”

The White House AI Task Force also said Anthropic must “immediately provide remedial services to affected organizations and harmed Americans.” It added that all AI companies “must immediately disclose incidents related to their models and take swift and decisive action to repair any damage.”

Read the Axios report.