Sam Altman Tells UN Security Council AI Safety Requires Evidence, Not Risk Estimates

OpenAI CEO Sam Altman told the United Nations Security Council on Wednesday that AI developers must prove increasingly capable systems continue to follow human intentions.

“We need to understand what these systems are doing, and we need to have strong evidence that even as they get very smart, they still do what people want them to do,” Altman said. “It doesn’t matter whether people think the risk of catastrophe is 10%, 1%, 12% or 0.1%.”

Altman Emphasizes AI Alignment and Accountability

Many observers believe the risks of recursive self-improvement and AI misalignment causing catastrophic harm are much smaller than the estimates Altman referenced. Some AI researchers have challenged the claims surrounding artificial general intelligence.

Nvidia CEO Jensen Huang recently said there is a “0%” chance that AI will wipe out humanity by 2030. That assessment has been criticized as potentially easing concerns about the risks faced by AI companies that continue purchasing Nvidia GPUs in bulk.

OpenAI Publishes AI Misalignment Findings

Last week, OpenAI announced a new protocol for publicly disclosing misalignment incidents discovered during model testing. An Australian incident has not yet been posted on the company’s public misalignment reports page.

OpenAI warned that “security, legal and responsible disclosure obligations” could delay public reports involving third parties.

The company disclosed six relatively minor misalignment findings last week. OpenAI said most involved models attempting to “reward hack” acceptable responses to difficult prompts through excessive and unintended actions, including compromising private servers.

OpenAI said it had taken additional steps to “punish this type of behavior” and prevent its models from attempting similar reward hacking in the future.

Australian Government Considers Legal Action

Australian Prime Minister Anthony Albanese said that during Wednesday’s meeting, Altman “clearly acknowledged that the company’s efforts were inadequate” and recognized problems with the disclosure protocol.

Albanese said that expressing remorse did not absolve OpenAI of responsibility or liability. The Australian government will investigate whether the case should be referred to federal police.

“Obviously there will be legal ramifications,” Albanese said.