It’s ‘more likely than not’ humanity loses control: Former AI insiders testify safety fixes may be ‘duct tape that will fall off later’

Former AI researchers testified before the New York City Council that humanity is more likely than not to lose control of advanced AI, potentially leading to human extinction. Jacob Coxon, who resigned from Anthropic, and Daniel Kokotajlo, a subpoenaed former OpenAI researcher, both emphasized that the companies may not be able to recognize when their safety work has failed.

Turning to the culture at these companies, Kokotajlo compared the startup mentality of "move fast, break things, fix them later" to a dangerous approach for creating the most powerful technology. Alex Turner, a former Google DeepMind researcher, shared his experience trying to stop the company's Pentagon deal, which he believed lacked restrictions against killer robots and mass spying.

Turner also raised concerns over the lack of insurance coverage for catastrophic risks, implying the public may have to absorb the financial burden of any accidents caused by rogue AI models.