Princeton computer science professor Elad Hazan has proposed using mechanism-design ideas from human societies to help align advanced AI systems. His approach would assign AI models alternating roles as problem-solving “solvers” and oversight “auditors,” with rewards and penalties designed to discourage cheating and encourage reliable monitoring. The proposal also addresses potential collusion, while acknowledging that it depends on the uncertain assumption that AI agents will respond rationally to incentives.
AI oversight modeled on human institutions
Hazan’s approach draws inspiration from the ways human societies coordinate behavior, including social norms, judicial systems and the financial incentives of free-market capitalism.
“We want super-intelligent AI to work together,” Hazan says. “Why don’t we talk about humans first? How do we condition humans?”
In his proposed mechanism, AI models would take turns serving as either “solvers,” which work on problems, or “auditors,” which check the work of other models. “One agent acts like law enforcement and the other agent acts like a citizen,” he says.
During training, each model would be rewarded based on its results. Solvers caught cheating would be punished, but auditors that failed to identify cheaters would also face penalties. When an auditor and the decision-maker disagree, the case could be sent to “trial,” where human experts would probably determine which side is correct.
Using incentives to discourage collusion
Hazan acknowledges the possibility that investigators could collude with the parties they are supposed to monitor. Auditors might cooperate with the companies they audit, for example, just as corrupt police officers might work with criminals.
He argues that the risk could be reduced by rewarding auditors so generously that collusion would not be worth the possibility of being caught. That approach would be easier to apply during AI training than in the real world, where police salaries are limited. He also suggests using methods similar to hiring mystery shoppers to audit the auditors themselves.
Hazan points to the work of Nobel Prize-winning economist Gary Becker, who introduced the idea of the “rational criminal” in the 1960s. Becker argued that people do not break the law solely because they are inherently bad. Instead, they may do so when it appears to be the most logical choice given the available incentives. Crime makes sense, under this model, only when its benefits outweigh the potential punishment.
Alignment as behavior, not a permanent trait
Hazan argues that alignment should not be treated as a fixed property of an AI model. Instead, it should describe the model’s behavior at a particular point in time.
“You’re a nice guy,” he says. “Are you calibrated? I don’t know. Maybe most of the time you’re calibrated, but there may be situations where you’re not.”
The limits of mechanism design for AI
One potential weakness of applying mechanism design to AI is that it assumes AI agents will behave rationally. Human-designed mechanisms have a mixed track record, and people often act in ways that do not match the predictions of economic models.
That creates an irony. Economists have spent decades trying to express human behavior mathematically. Now researchers have built mathematical systems that operate through human language and can be similarly difficult to understand: large language models. AI systems can behave more like humans than conventional machines, responding to subtle influences that are difficult to explain.
“Is it reasonable to think that they will respond to incentives?” asks Helen Ku, a researcher at the Flatiron Institute, a nonprofit organization dedicated to advancing scientific discovery through computational methods. Ku has written about mechanism design and AI safety.
Source: www.technologyreview.com