OpenAI details new text watermarking system for ChatGPT, Codex, and the API

OpenAI has announced a new text watermarking system for its ChatGPT, Codex, and API services in response to the EU AI Act's requirements for generative AI providers to mark AI-generated text. This move comes after Anthropic's Claude implemented a similar system, albeit with a more limited rollout and scope. OpenAI's solution, called textGrain, adds an invisible statistical signal to the model's word choices, which the company's detector can identify in AI-generated text.

Initially, text watermarking will be optional in the API and limited to eligible users in the EU within ChatGPT and Codex. OpenAI acknowledges that the detection technology has limitations, including false negatives and positives, particularly with shorter texts or specific domains, such as mathematics. The watermark can be weakened by editing, with a 10% word replacement reducing detection from 92% to 66%, and a 25% word replacement dropping it to 17%.

Despite these limitations, OpenAI found that watermarked outputs scored slightly higher on certain benchmarks. The company also emphasized that a watermark does not measure human contribution, establish ownership or responsibility, identify the user, verify accuracy, or prove human authorship. OpenAI has opened applications for access to its text watermark detector for approved researchers and expert organizations to evaluate and improve the technology. They plan to make the technology available in open source for further development.