OpenAI will start watermarking ChatGPT’s text in the EU

OpenAI announced on Monday that it will begin watermarking text generated by ChatGPT and Codex in the European Union in order to adhere to the EU AI Act's transparency rules, which took effect on August 2. The EU AI Act mandates that AI companies embed a recognizable mark in AI-generated content that other systems can identify. OpenAI explained that the watermark will gradually roll out to eligible ChatGPT and Codex users on all subscription plans, but will only be available to users in the EU.

Developers utilizing OpenAI's API worldwide can enable the watermark for specific models starting today, with it being disabled by default. The watermark is not a visible symbol but rather subtly alters the model's word choices, leaving a pattern that can be detected by a detector but remains invisible to readers. Importantly, the watermark does not reveal the user's identity.

OpenAI tested the watermark and found no significant impact on the models' performance when it is active. The company also released a technical report for its textGrain method, developed in collaboration with researchers from the University of Pennsylvania and Yale. The report provides an example of using a secret key to guide subsequent word predictions in finishing a sentence, with the detector able to identify AI-generated content using only the text and the key.

However, OpenAI's tests revealed that the watermark can be removed by editing. For instance, replacing 10% of the words with synonyms reduced detection accuracy from approximately 92% to 66%. OpenAI noted that short passages, math answers, and translated text are more challenging to detect. To address these limitations, OpenAI has decided to initially provide access to the detector only to approved researchers and expert organizations, who can help evaluate its reliability and responsible uses.

OpenAI cautioned that the absence of a watermark does not confirm human authorship; the text could be too brief, heavily edited, or created by another company's AI. The watermark can indicate that an OpenAI system contributed to a passage but does not quantify the extent of human input, editing, or creativity. OpenAI's announcement follows Anthropic's decision to watermark text generated by Claude, a move it is applying globally.

Anthropic's announcement in late July drew criticism from some Claude users who argued that they supplied the instructions, context, and decisions while Claude acted as a tool.