OpenAI Announces Their Text Watermarking Plans
OpenAI, today: The EU AI Act requires generative AI providers to make generated text identifiable in a machine-readable way. Text watermarking and detection remain early technologies with significant limitations, and views about their benefits and responsible uses are still developing. Our phased approach reflects both the EU AI Act requirements as well as the technology’s limitations, with an…
OpenAI has announced plans to implement text watermarking for its generative AI models in response to the EU AI Act. This requirement mandates that generative AI providers make generated text identifiable in a machine-readable way. However, text watermarking and detection are still early technologies with known limitations, and their benefits and responsible uses are subjects of ongoing debate.
OpenAI's phased approach to text watermarking reflects both the EU AI Act's requirements and the current limitations of the technology. Starting today, API customers worldwide will have the option to enable text watermarking for select models, with text watermarking remaining off by default. Over the coming weeks, OpenAI will introduce an invisible watermark to eligible ChatGPT and Codex text output within the European Union.
Access to OpenAI's text watermark detector is currently limited to approved researchers and expert organizations tasked with evaluating and improving the technology. Unlike some competitors, OpenAI is opening applications for access to this detector, allowing developers to assess whether this watermarking may impact text output in real-world scenarios.
OpenAI's text watermarking technology, known as textGrain, incorporates an invisible statistical signal into the model's word choices. The accompanying detector analyzes this signal to determine if a passage contains an OpenAI watermark. More technical details about textGrain can be found in the company's technical report, which will be updated with additional information in the coming weeks. OpenAI also plans to make the technology available in open source, enabling others to build upon it.
During evaluations, textGrain has shown performance comparable to or better than other approaches, including Google's SynthID for text. However, the effectiveness of text watermarking in real-world applications remains uncertain. Strong performance under controlled conditions does not guarantee reliable detection in everyday use, especially when prompts contain instructions designed to circumvent the watermarking.
Written by urgent.news from Daring Fireball's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
