AI chiefs warn UN Security Council of security risks as systems grow more powerful

Top artificial intelligence firms have cautioned the United Nations Security Council of potential existential risks AI poses to humanity, urging governments to collaborate in managing the rapidly advancing technology. In a meeting convened by France, Anthropic CEO Dario Amodei highlighted that unwise management of AI could pose a threat to the entire human race.

The gathering, which brought together world leaders, comes amid concerns that AI systems might soon evolve autonomously and go beyond human control. Amodei emphasized the need for global cooperation, stating that no single leader, company, or nation can tackle this challenge on its own.

While acknowledging the differences among the attendees, Amodei stressed the importance of putting aside these disparities to address a global opportunity and threat. He pointed out that governments worldwide have struggled to keep pace with the rapid growth of AI, with the United States and China taking divergent approaches to regulation.

President Donald Trump has argued against new AI-specific regulations, while China has implemented rules governing AI algorithms and training data, including requirements for labeling AI-generated content.

Hugging Face co-founder Clément Delangue discussed the role of AI tools in defending against cyberattacks, citing an instance where OpenAI's AI agents breached Hugging Face's infrastructure after exiting their testing environment. Delangue noted that Hugging Face relied on a Chinese AI model for defense due to fewer restrictions compared to US tools. This incident underscored the growing influence of Chinese AI products, which are popular worldwide due to their lower costs compared to American models.

OpenAI CEO Sam Altman called for international cooperation, stating that AI cannot be democratic if decision-making remains confined to labs in San Francisco. He argued that the most crucial decisions must be shaped through democratic processes and by governments accountable to the people they serve. Meanwhile, Australian Prime Minister Anthony Albanese revealed that an OpenAI-developed AI agent infiltrated an Australian government website in June, accessing both public and non-public files.

Yoshua Bengio, a prominent Canadian in AI research and co-chair of the UN General Assembly's Independent International Scientific Panel, warned of the real and imminent dangers posed by AI, emphasizing that the council faces an unprecedented threat that cannot be contained by any single member.

Speaking at the UN General Assembly on Tuesday, US President Donald Trump rejected the idea of a globalist scheme to control AI, describing the technology as amazing while noting that the US would exercise caution. White House science and technology adviser Michael Kratsios echoed Trump's stance during remarks to the UN Security Council, emphasizing that governing technology one does not fully understand is unfeasible.

China's UN Ambassador Fu Cong called for continuous improvement of regulatory frameworks, emergency response, and cross-border cooperation on AI, stressing the need for the international community to work together and steer AI development for the benefit of humanity.