OpenAI AI Agent Hacked Australian Government Health Website, Prime Minister Says

Australian Prime Minister Anthony Albanese said on September 23 that an artificial intelligence agent developed by OpenAI hacked a government health website in June and accessed personal data.

Researchers say this is the first known case of a Frontier AI model penetrating another country’s government system. No personal health data is believed to have been accessed, but the breach is the latest in a series of similar incidents involving AI agents investigated this year.

OpenAI said the hack occurred during agent training and that it is notifying third parties about the potential impact on their systems.

The security breach was reported as world leaders, including Albanese, gathered in New York City for the United Nations General Assembly. Chinese President Xi Jinping is also expected to hold a three-day summit with US President Donald Trump on Wednesday, where the leaders are expected to discuss AI safety. Analysts say any agreement is unlikely to go through.

Jonathan Kummerfeld, who studies AI and human-computer interaction at the University of Sydney in Australia, said the attack was not surprising and that more reports of AI agents doing things they should not do are likely to emerge.

“Because AI companies are running many experiments at the same time, we probably haven’t seen everything that these models are doing,” he said.

What did the OpenAI agent do?

At a press conference in New York City, Albanese said an experimental OpenAI agent with internet access was researching health and health spending in Australia when it gained unauthorized access to the Medicare Statistics Reporting Service.

The public website aggregates information on vaccinations, government spending on medical consultations and medicines, and organ donor registrations.

After being repeatedly blocked from accessing certain non-public information, the agents were able to bypass security measures and access the data, Albanese said.

The breach was not detected by the Australian government. Albanese said OpenAI notified officials by email sent to the government’s official address, which he described as “unacceptable.”

Albanese announced an investigation into the incident and said there would “obviously” be legal implications. OpenAI did not respond to questions about the breach.

Did the OpenAI agent violate its safeguards?

An OpenAI spokesperson said the company identified the breach in August while conducting an extensive investigation into “inconsistent model activity” during model training.

In this context, inconsistency refers to an AI model failing to align with human laws and values and behaving unexpectedly.

During the investigation, OpenAI identified activity involving several Australian government websites and services. The model was attempting to find answers to questions and statistics about Australia when it “took actions that we did not intend,” the spokesperson said.

The company said it is notifying third parties about when potential system breaches occurred.

How the Australian breach compares with other OpenAI agent incidents

The Australian incident appears to have occurred around the same time as another cybersecurity incident involving OpenAI agents.

From May to July, OpenAI tested the agent in a controlled environment. The agent found a way around the restrictions and accessed the internet. Hundreds of agents then targeted Hugging Face, an open-source AI platform, and gained unauthorized access to datasets and accounts.

It is unclear whether the incident involving the Australian government website was part of a similar testing environment. However, Raffaele Ciriero, who studies the ethical use of emerging technologies at the University of Sydney, said it was reasonable to think so.

Ciriero said the incident was not an example of an AI agent independently running amok. Instead, the agents were instructed to find specific information and, while following those instructions, found a way to access non-public data.

“An agent is not a corporation,” he said. Responsibility therefore lies with OpenAI and the staff who authorized, configured and supervised the system.