NYC Lawmakers Grill AI Executives and Former Employees as City Weighs Guardrails

Former Anthropic researcher Jacob Coxon warned City Council members that there is a high likelihood of humanity losing control to AI, potentially leading to extinction, during a hearing on Monday. He criticized the companies for their recklessness in advancing the technology. Coxon joined a panel of former AI employees, OpenAI, Anthropic, Meta, and Google executives to discuss the risks posed by AI.

City Council Speaker Julie Menin asked the executives to quantify the risk in a worst-case scenario, but received varied responses. Morgan Dwyer, OpenAI's head of policy development and operations, expressed uncertainty and stated that she believed any risk level was unacceptable. New York City aims to establish guardrails and protections against AI's rapid development, with bills proposed to provide whistleblower protections and a city-run "kill switch."

The City Council's approach could serve as a national model, according to Menin. The hearing covered various AI-related threats, such as the potential for AI to aid in weapon development and rogue AI agents causing damage. Executives emphasized the importance of safety protocols and getting AI technology into the hands of as many people as possible for its benefits.

Former AI employees warned of the dangers of an AI arms race and the need for reining in the technology.