OpenAI Tests Whether Embedded Oversight Can Keep Pace With AI Risk
Quick take Apollo Research says six researchers will work inside OpenAI for four months, examining whether training incentives encourage subversion and whether evaluations detect it. Goodfire will use interpretability tools during training runs, while METR will evaluate dangerous behavior that OpenAI’s own tests might miss. Transluce’s six-week assessment includes selected models, internal…
OpenAI has initiated a pilot program to test whether embedded oversight can effectively identify and address risks associated with increasingly capable AI models. Six researchers from Apollo Research, Goodfire, the Model Evaluation and Threat Research (METR) group, and Transluce will work within OpenAI for four months, examining alignment, safeguards, training runs, and internal deployment.
Apollo Research will focus on whether training incentives could encourage models to undermine alignment or control mechanisms, while Goodfire will use interpretability tools to monitor training runs for unwanted internal behaviors. METR will perform an external evaluation of dangerous behavior that OpenAI's own tests may miss, and Transluce will assess the adequacy of OpenAI's post-training evaluations and their impact on release decisions.
The pilot aims to provide safety evidence integrated throughout the training, deployment, and release processes, rather than as a final check before launch. However, the restricted access and disclosure limitations may limit the extent of the scrutiny this pilot can provide.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.