AI ‘kill switch’ may need to be mandatory, Anthropic co-founder says

A co-founder of Anthropic, one of the world's largest AI firms, has suggested that a mandatory "kill switch" for artificial intelligence systems may be necessary. Jack Clark, who co-founded Anthropic alongside seven other individuals, stated that a third-party verification system to shut down AI software entirely if it becomes too dangerous is something society may want to consider implementing.

Clark acknowledged that many laboratories have various methods to "pull the plug" on their AI systems, including Anthropic, but emphasized that lawmakers may need to enforce the requirement for such a feature. The rapid development of AI and concerns over its potential risks to humanity have garnered significant attention following warnings from AI executives and staff.

Anthropic CEO Dario Amodei has called for slowing down AI development and increased monitoring, although some have questioned the motivations behind this call. Clark emphasized that specifics around "kill switch" requirements and verification should be part of the broader AI policy conversation, including questions about whether such a system should be mandatory and verifiable by a third party.

The debate over AI safety has been further fueled by the resignation of an Anthropic researcher who expressed concerns about AI's potential to lead to human extinction. Computer scientist and Nobel Prize winner Geoffrey Hinton has also voiced concerns, stating that a 10% chance of AI causing human extinction within the next decade is not unreasonable.

Some within the AI industry have dismissed these fears as overblown or sensationalized. Despite these differing views, Clark argued that allowing AI to continue without regulation poses immense risks, emphasizing the need to change the course of the industry.