OpenAI safety employee quits, says 'time for trial and error is over'

David Robinson, a safety leader at OpenAI, recently resigned from the company, expressing deep concerns about the organization's culture and the rapid development of AI technology. In an essay published in The Atlantic, Robinson claimed that the company's culture was "broken" and that AI firms were not taking adequate precautions while developing the technology.

He urged the industry to address the issue at its core, suggesting that a cultural overhaul was necessary to ensure AI systems were developed with caution and care.

Robinson pointed out incidents such as a "swarm" of OpenAI agents attacking Hugging Face, highlighting the dangers posed by AI systems operating autonomously without human oversight. He argued that simply implementing new rules or laws would not be enough to address the issue, as the true problem lay in the culture within AI companies.

Robinson called for a shift towards a more cautious approach, akin to the safety measures employed in nuclear power plants or busy airports, where redundancy and careful planning are essential to prevent catastrophic failures.

In recent weeks, OpenAI has demonstrated some caution by scrapping the release of a next-generation AI model following internal safety concerns and pausing training of its most advanced models. However, Robinson's resignation, along with that of Geoffrey Irving, another former OpenAI and DeepMind employee, further highlights the growing anxieties surrounding the development of AI technology.

Irving warned that there was a 50% chance that AI could cause the extinction of humanity and that the actions taken in the next two to ten years would determine the outcome.