Are Open AI Models More Dangerous Than Closed Models?

It is a major oversimplification to assume that open models are dangerous while closed models are safe. The safeguards AI companies build around closed models are far from perfect, and the Humanity Risk Report makes clear that people regularly abuse them.

Open models generally provide even fewer protections against exploitation. However, malicious attackers may still choose closed models because they are more accessible than open models with comparable capabilities. By contrast, running an open model locally requires substantial computing resources, including expensive computer chips—and even more chips if the model needs to be retrained.

Can open models have safety advantages?

Some experts quoted by MIT Technology Review argue that open models may have safety advantages over closed models. They compare open models with open-source software. Tools such as the Linux operating system are often considered more secure than closed-source alternatives because a large community of contributors can research vulnerabilities and develop fixes.

Likewise, the research community can test open models for security flaws more easily. However, addressing those flaws is more difficult than rewriting a line of code in traditional software.

How open models could improve cyber defense

Open models can also be deployed more widely for cyber defense. Tools that can be used to hack systems can also be used to protect them. Security defenders routinely attack their own software to identify vulnerabilities that need to be patched.

Anthropic has demonstrated how this approach can work with a closed model through its Project Glasswing initiative. The program will allow some trusted companies to use the company’s most capable models for cyber defense. However, many other organizations have been left to develop their own defenses.

Open models can be used more broadly. After the OpenAI agent hacked Hugging Face, the company noted that questions about the incident triggered the closed model’s security filters. As a result, Hugging Face had to rely on an open model to analyze the attack.

Why experts still consider open models riskier

Despite these potential advantages, many experts still view open models as riskier than closed models. When dealing with tools as powerful as current and future AI systems, established open-source security principles may not always be enough.

“Where things get complicated is when we start to take seriously the one-time occurrence of a particularly catastrophic event that may be unlikely but would be catastrophic if it were to occur,” Seeger said.

For example, if a highly capable open hacking model were released to the public today, malicious attackers could potentially damage technological infrastructure before defenders had time to respond. Once such an event occurred, recovery might be impossible.

The risks may appear even greater when models can act autonomously. The OpenAI-Hugging Face hack occurred while the company’s model was being tested with many of its cybersecurity guardrails removed. Open models typically have few guardrails to begin with.

Even if an open model commits fraud, detecting the activity can be difficult because the system is highly decentralized. For this reason, experts who take the potentially catastrophic risks of rogue AI agents seriously—including Dario Amodei—tend to be more skeptical of open models.

Can open AI models be made safer while remaining open?

A few years ago, the debate over open models was often presented as an all-or-nothing choice: support them because they could reduce the concentration of power, or reject them because of their security risks. Today, researchers are increasingly looking for a compromise that preserves openness while reducing potential harm.

Potential safety techniques for open models

Many safety tools used by closed models, including filters and monitoring systems, do not apply easily to open models. However, researchers are exploring other approaches.

Some are investigating whether open models can be trained without exposure to data that teaches harmful or undesirable behavior—for example, by preventing models from learning about hacking. The challenge is removing every reference to fraud and other harmful activity from training data without reducing the model’s overall performance.