‘Time for trial and error is over’: OpenAI safety employee quits, criticises ‘broken’ culture

David Robinson, a former OpenAI safety employee, recently resigned from the company, criticizing its approach to AI safety. In an article published by the Atlantic on Oct 3, Robinson argued that the fast-paced culture at OpenAI increases the risk of failures in advanced AI systems. He emphasized that AI companies, including OpenAI, are not being "nearly careful enough" and should prioritize safety expertise and research before developing more capable systems.

Robinson wrote, "The time for trial and error is over," advocating for greater safeguards in AI systems, similar to those implemented in industries such as nuclear power and aviation.

Robinson claimed that OpenAI heavily relies on "iterative deployment," a strategy involving the release of systems and the reinforcement of safeguards after problems arise. This approach has come under scrutiny in the AI industry, with incidents of safety controls failing or experimental systems behaving unexpectedly. Robinson, who had worked at OpenAI for 3½ years, contributed to the development of the company's preparedness framework and oversaw safety reports for 12 frontier-model launches.

He expressed concern that the company's rapid pace of launching new models was undermining the level of care needed to ensure safe and secure AI systems.

In response to Robinson's criticisms, an OpenAI spokesperson stated that the company is committed to ensuring its models do not become more capable than they can safely manage and secure. They pause training or hold back models when necessary to slow down development. However, Robinson warned that AI capabilities are advancing faster than researchers' understanding of alignment, a field focused on ensuring AI systems act in accordance with human goals and values.