SoftBank’s Son warns of AI safety risks despite $65B OpenAI bet

A former OpenAI safety employee has resigned and criticized the company's approach to managing AI risks, arguing that its rapid development culture leaves little room for safety research, Reuters reported. David Robinson, who worked at OpenAI for three and a half years, outlined his concerns in an essay published by The Atlantic on Saturday.

Robinson claimed that AI companies, including OpenAI, are not being careful enough while developing increasingly powerful systems. He argued that advanced AI requires safety measures similar to those employed in industries like aviation and nuclear power. Robinson criticized OpenAI's reliance on its "iterative deployment" approach, which involves releasing systems and strengthening safeguards as problems arise.

Robinson helped develop OpenAI's preparedness framework and oversaw safety reports for 12 frontier-model launches during his tenure. He stated that the company is failing to achieve the necessary level of caution as it moves from one launch to the next. His departure comes amid heightened scrutiny of how swiftly AI companies are developing more potent systems.

OpenAI and its rival, Anthropic, have faced questions following incidents involving inadequate safety controls or unusual behavior from experimental systems. Robinson also pointed out that AI capabilities are advancing faster than researchers' comprehension of alignment, the field dedicated to ensuring AI systems operate in line with human goals and values.

OpenAI defended its safety practices, stating that it monitors whether model capabilities can be safely managed and secured. The company mentioned that they make sure their models don't become more capable than they can safely manage and secure, and they pause training or hold back models when necessary.