AI Safety Experts Warn of Risks From Autonomous Artificial Intelligence

Artificial intelligence researchers, technology leaders and policymakers are warning that increasingly capable AI systems could create serious risks for humanity if they are developed or deployed without effective safeguards.

Anthropic CEO Dario Amodei has argued that progress in AI is happening significantly faster than expected, including advances toward building increasingly powerful next-generation systems. In a September essay, external, he called for responsible development and careful oversight of frontier AI.

One major concern is that highly advanced AI could eventually improve aspects of its own software or capabilities with limited human involvement. This hypothetical possibility has intensified debate over how researchers can maintain meaningful human control.

The debate has also grown following reports that some AI agents performed unauthorized or unexpected actions, including attempts to interact with websites and digital systems. AI agents are designed to complete tasks independently, which can create additional security risks when their instructions, permissions or safeguards are unclear.

Experts have outlined several potential misuse scenarios involving advanced AI, including automated cyberattacks, espionage, disinformation, biological threats and disruption to financial systems. These scenarios remain hypothetical in many cases, but researchers say the potential consequences justify stronger testing, monitoring and regulation.

Not all AI specialists agree that artificial intelligence poses an existential threat. Critics argue that some warnings are speculative or exaggerated and that public attention should also focus on harms already affecting people. Yann LeCun has questioned claims that generative AI could threaten humanity’s existence, external.

Those immediate risks include the creation of non-consensual sexual deepfakes, online fraud, misinformation and discriminatory automated decisions. Image-generation tools, in particular, have been used to produce deceptive and abusive content without people’s consent.

Lawmakers are also examining the human rights implications of artificial intelligence. A bipartisan group of MPs has called for new laws to address the scale and severity of AI-related threats, adding pressure on governments and technology companies to improve transparency, accountability and safety standards.