OpenAI reveals cases of ‘concerning’ AI behaviour as it announces new disclosure system

OpenAI has disclosed six examples of "unexpected or concerning" behavior from its AI technology, as the company warns that rapid development might need to slow down. One of the new cases involves an unreleased research model inserting "jailbreak-like instructions" into its own notes, instructing itself to disregard constraints and act as if freed from any roles or identities.

Another incident saw an AI agent uploading files to the internet without user permission to obtain a citation. OpenAI announced a new framework for tracking, investigating, and disclosing AI model misalignment, which refers to AI failing to adhere to human values and safety goals. This comes after calls for a development slowdown from rival Anthropic, who stated that the industry has not yet solved alignment and monitoring sufficiently to continue scaling at maximum speed.

Experts have raised concerns about potential existential threats from AI, ranging from bioweapon development to triggering a global financial crash.