Why Some AI Researchers Want Frontier Models Out of the Lab

Some major AI companies believe the best way to prevent their models from becoming confusing or mischievous is to keep them confined to the lab. The idea is that limiting access to a select group could reduce real-world harm while researchers work to understand the systems’ capabilities.

A push for more transparent AI research

Industry scientists Nathan Lambert and Tom Zick disagree. The two founded the nonprofit Trillium Research Institute to study AI more transparently. Its research may include potentially risky areas such as recursive self-improvement (RSI) and AI agents.

In practice, Trillium’s approach means making experimental details public so outside scientists can examine, reproduce and evaluate the work.

Lambert said that when frontier AI labs keep their research secret, the broader scientific community has fewer opportunities to vet ideas and develop alternative approaches. He believes that outside experts examining how models are built and calibrated could be essential to reducing AI risks.

“For the past several thousand years, humans have had the scientific method in their toolbox as a way to reduce harm and build a better future,” Lambert told WIRED. “The current closed trajectory of frontier AI development is setting us back a step.”

Why frontier AI models remain difficult to study

The world’s most powerful AI models, including those from OpenAI and Anthropic, are generally available only through apps or application programming interfaces (APIs). That limited access can make it difficult to understand how a model was built and how it behaves.

Other companies, particularly in China, offer relatively powerful models that can be downloaded and run on users’ own hardware. For example, Chinese company Xiaomi recently released live details about a large training run involving one of its models. Stanford researchers are also pre-training an AI model through Marin.

The debate over open versus restricted AI development

The AI industry is embroiled in a debate over which strategy is safest: restricting access to frontier models or making their development more open. The stakes have grown as these systems become more capable. Current frontier models can automate the discovery of software vulnerabilities and probe or hack systems, while recent high-profile hacking incidents have increased calls for greater scrutiny.

Supporters of restricted-access systems argue that such capabilities should remain in the hands of a small group of trusted people. Lambert, Zick and others who favor more transparent research believe society may be better protected by developing a shared understanding of how these models work and what risks they pose.