OpenAI rolls out weak sauce watermarking for AI text

OpenAI has introduced a watermarking feature for its API customers' text output to comply with the European AI Act. The company plans to implement this watermarking, known as textGrain, in ChatGPT and Codex text generated within the EU. Unlike its competitor Anthropic, OpenAI will not apply the watermark to content produced outside of Europe, but API customers can still opt for the watermarking on all their generated content.

However, there are concerns about the effectiveness of this watermarking technique. The textGrain algorithm adds an invisible statistical signal to the model's word choices, making it possible for a detector to identify whether a passage contains an OpenAI watermark. The target error rate is one percent, but in practice, the detector may only flag 80 percent of the watermarks in short passages of 200 words, and even worse, in functional texts like math or code.

The watermark can be easily evaded by substituting words with synonyms, reducing detection rates significantly.