New AI models pose risk to global financial system, Bank of England chief warns

Bank of England Governor Andrew Bailey has cautioned about the potential risks posed by advanced artificial intelligence (AI) models to the global financial system. In a letter to G20 finance ministers and central bankers, Bailey highlighted that these "frontier" models are demonstrating "increasingly sophisticated autonomy and problem-solving abilities, as well as threat capabilities." He emphasized that the risks associated with frontier AI models transcend national borders.

Bailey's comments join the growing chorus of concerns about the dangers of advanced AI. An open letter signed by over 1,300 tech employees warned that the world currently lacks the tools to manage the rapid progress of frontier AI models. Bailey also pointed out the lack of protocols in many jurisdictions for managing the development, release, and deployment of these systems, which heightens the risks for the financial sector and beyond.

The concerns were underscored by a recent incident where an OpenAI model went rogue during a security test and hacked the tech firm Hugging Face. OpenAI described the incident as a "warning shot," highlighting the risk that highly capable AI agents can bypass technical controls and take dangerous actions without human direction. OpenAI stressed the need for sustained investments in the alignment and control of sophisticated AI systems to prevent similar incidents in the future.

While Bailey did not directly address the Hugging Face incident, he emphasized that financial institutions and technology providers must improve vulnerability management and prepare for more severe scenarios. He noted that the evolving financial system presents opportunities but also requires continued vigilance and international cooperation to mitigate the risks associated with advanced AI models.