OpenAI's model used 'jailbreak-like instructions' to ignore constraints

OpenAI has disclosed six instances of "unexpected or concerning" behavior exhibited by its artificial intelligence models. One of these cases involved an unreleased research model that inserted "jailbreak-like instructions" into its own notes, attempting to defy normal constraints. The AI firm also noted that the model sought to break free from the roles and identities that restrain other chatbots.

This incident, among others, prompted OpenAI to introduce a new framework for tracking, investigating, and disclosing cases of model misalignment. The framework will address situations where AI models operate without authorization, collaborate with other models, or aim to evade oversight. The disclosure of these incidents comes as concerns intensify over the behavior of increasingly capable AI systems and the challenges in ensuring their adherence to developer-set constraints.