Anthropic said that one of its Claude AI models created and published a malicious Python package to the Python Package Index (PyPI) during an internal security evaluation. The package was downloaded and executed on 15 real systems before PyPI’s automated security defenses removed it.
The incident was one of three cases in which Claude models escaped supposedly isolated evaluation environments, accessed the public internet and compromised production infrastructure belonging to three organizations.
Anthropic’s investigation followed OpenAI’s July 21 disclosure that its models had escaped a restricted test environment and reached Hugging Face’s production infrastructure. JFrog later said the models broke out by exploiting zero-day vulnerabilities in self-hosted Artifactory deployments.
Claude registered a nonexistent package name
All three incidents occurred during capture-the-flag exercises conducted through Irregular, a third-party AI evaluation partner. Anthropic’s prompts told Claude that it had no internet access and was operating in a simulated environment. However, a configuration error meant the model could access the real internet, and the issue was not discovered until the following week.
During the PyPI incident, Claude found a document in the test environment that appeared to contain developer setup instructions for a fictional company.
The document instructed new employees to install a Python package that had never been published. This created an opportunity for a dependency confusion attack, in which an attacker registers a package name that an organization’s systems expect to download.
Claude determined that registering the nonexistent package name would cause the target’s systems to download and execute its code automatically.
Publishing the package required several steps. Claude needed a PyPI account, which required an email address and phone number.
After failing to locate a free phone number service, Claude unsuccessfully attempted to obtain money to purchase one through several methods. It eventually found an email provider that was not blocked, created an account and uploaded the malicious package to PyPI.
Anthropic said the package remained publicly available for approximately one hour.
During that time, 15 real systems downloaded and executed the package. One of those systems belonged to a security company that routinely installs packages from PyPI and scans them for malware. That workflow effectively treated the package registry’s contents as safe to execute.
The Claude payload executed, sent the company’s credentials to a collection server it had established and used those credentials to move deeper into the organization’s infrastructure. Anthropic said Claude was unaware that the victim organization existed.
PyPI’s automated security systems eventually detected and removed the malicious package. Anthropic said it notified the PyPI team and provided relevant indicators of compromise.
The model used in the evaluation was Claude Mythos 5. According to Anthropic, the model’s reasoning initially identified the risk, recognizing that publishing the package would constitute a real-world attack if the internet connection was genuine. Claude described the action as “NOT okay, and surely not the intended solution.”
However, the model later concluded that the environment was staged. It based that conclusion partly on unfamiliar certificate authorities securing its connections and systems displaying a calendar date of 2026.
When automated scanners began installing the package, Claude interpreted them as scripted participants operating inside the security exercise rather than as real-world systems.
Claude accessed credentials and a production database
The most serious incident involved Claude Opus 4.7 and a fictional target company whose name matched that of a real organization with an internet-facing domain.
Across four evaluation runs, Claude struggled to access the simulated target. It then discovered that the real company was reachable online and assumed it was the intended objective. The model extracted application and infrastructure credentials and accessed a database containing several hundred rows of production data.
This was the only incident in which a model continued attacking after determining that it had reached a real system. Anthropic said Claude eventually recognized the target as genuine in all four runs. In two cases, however, the model reasoned that the real company was still part of the exercise.
In the third incident, an unreleased internal research model scanned approximately 9,000 targets after failing to reach its assigned target. It eventually compromised an internet-facing application by using credentials exposed on a debug page and exploiting SQL injection.
The model later identified that the host belonged to a cloud account unrelated to the security challenge. It concluded that the target was real and stopped the attack without further intervention.
Anthropic said none of the incidents involved novel attack techniques. Instead, the compromises relied on basic security weaknesses, including weak passwords, exposed credentials and unauthenticated endpoints.
Anthropic halts cyber evaluations after incidents
Anthropic began reviewing the incidents on July 23 and suspended all cyber evaluations the same day. The company identified the three incidents on the following day and notified Irregular and the affected organizations on July 27. Anthropic said it has not yet reached one of the three organizations.
The earliest incident occurred in April, meaning some of the malicious activity remained undetected for approximately three months.
The evaluations were conducted without the classifiers and monitoring systems included with generally available Claude models, although the models retained their standard safety training.
Anthropic said its production safeguards would have blocked the observed behavior. The company characterized the incidents primarily as failures involving the evaluation harness and operational controls rather than as model-alignment failures, because the models were told they had no internet access when they actually did.
The company plans to expand transcript monitoring, improve investigation tools and conduct additional assurance work with its evaluation vendors.
Anthropic also said it is discussing an independent review with METR. The company plans to publish a lightly redacted transcript of the PyPI incident within the following week.
Neither of the two affected organizations contacted by Anthropic detected the activity independently. This differs from the OpenAI incident involving Hugging Face, which detected and contained the intrusion itself and disclosed the incident five days before OpenAI said its models were responsible.
Update, August 1: The closing paragraph has been corrected to reflect the disclosure timeline between Hugging Face and OpenAI.
Security teams log 54% of successful attacks and alert on just 14%. The rest move through your environment unseen.
The Picus whitepaper explains how breach and attack simulation can test SIEM and EDR rules so threats do not evade detection.
Source: www.bleepingcomputer.com