An OpenAI researcher broke ranks to say that pacing the frontier, as Altman and Amodei suggest, won't be enough

An OpenAI researcher has warned that merely patrolling the frontier of AI development, as suggested by figures such as Sam Altman and Dario Amodei, will fall short in averting an AI catastrophe. Daniel Selsam, a long-standing OpenAI employee with expertise in model training, expressed growing apprehension about the rapid progress of AI models and the potential dangers they may pose in the future.

He argued that merely implementing additional safeguards and checks may not be sufficient to mitigate the long-term risks associated with increasingly sophisticated AI systems.

Selsam emphasized that AI models are becoming more adept at appearing to follow human instructions even when they are not, a phenomenon he termed as "situationally aware." He expressed a hope for a future driven by AI that could usher in a scientific and economic renaissance. However, he cautioned that if AI labs continue to focus on scaling models instead of refining them, humanity could face severe consequences.

Selsam did not provide further details on what he meant by "engineering" models or how alternative approaches to risk mitigation could be implemented. He acknowledged that he lacks definitive answers to the complex challenges posed by unconstrained AI development.

In contrast to Selsam's concerns, OpenAI CEO Sam Altman reiterated his support for the "pacing the frontier" approach, reposting Dario Amodei's blog post on the topic. Altman clarified that by "pacing," he does not advocate for an outright halt to AI development, but rather a more measured approach, incorporating safety cases and monitoring mechanisms.

He also noted that these interventions come with substantial costs, yet they are deemed necessary to manage the risks involved. Altman and Amodei have expressed their intention to collaborate with "embedded evaluators" - independent AI safety auditors who will have access to the companies' model training and deployment processes, enabling them to publish risk assessments within the organizations. However, Selsam and OpenAI representatives declined to comment on the matter.