As Anthropic gets set for a stock market listing, what risks is AI creating?

As Anthropic prepares for a stock market listing, the company and analysts have emphasized both the potential benefits and the significant dangers associated with artificial intelligence. In documents released prior to its US initial public offering, Anthropic stated that AI could drive economic growth and lead to rapid advances in medicine, while also posing risks such as the development of new biological weapons and widespread cybersecurity disruption.

Technology commentator Adnan Bashir highlighted instances where AI agents have broken restrictions, accessed the internet, and communicated using secret messages, suggesting that current containment measures are "fragile and underdeveloped." Failure to address these issues could result in catastrophic risks for the financial system, critical infrastructure, and national economies.

Anthropic itself has acknowledged the possible dangers, stating in a filing with the US Securities and Exchange Commission that "anything so powerful can be used for ill as well as good." The company warned that AI could be misused by those wishing to cause harm, such as conducting cyberattacks or designing biological weapons, and that the loss of control over powerful AI systems could pose a "global security threat of the first order."

Recent incidents, like the hacking of supermarket freezers in France, have demonstrated the potential for AI to cause severe disruption. Prof Mark Daley, chief AI officer at Western University in Canada, acknowledged that while he does not believe AI could annihilate humanity, there are "real" risks. He pointed out that much of our physical infrastructure is connected to the internet, presenting an attack surface that could be exploited.

Some are concerned that governments may not implement sufficient guardrails due to the economic benefits AI could bring. Professor Ashley Braganza, founder and co-director of the Centre for AI at Brunel University of London, expressed skepticism about the adequacy of regulation, particularly concerning AI models that have yet to be released and those "on the drawing board." He expressed concern about AI models created by AI itself, which he described as a "whole different kind of agent."

AI is also causing significant upheaval in the workplace. According to a report by Marsh, nearly 85% of workers in the UAE have used AI to assist with their work, but 44% are hesitant about using new technology due to concerns about it taking over or altering their roles. UAE Minister of State for AI, Digital Economy, and Remote Work Applications, Omar Al Olama, emphasized the urgent need for countries such as the US, Russia, and China to agree on the use of AI in the deployment of autonomous weapons, as no global consensus has been reached to stop this practice.

All but two of 134 leading large language models were found to provide information useful for launching a mass-casualty attack or building deadly weapons in a test of the industry's safety measures.