AI model watermarking changes agent behavior

Watermarking AI-generated content may change agent behavior, according to Lasso Security. The European Union's AI Act requires providers of AI models to add machine-readable codes to their software outputs. Algorithms such as Google DeepMind's SynthID-Text and Anthropic's watermarking methods have been adopted by several companies.

Watermarking is intended to establish provenance, but it also alters the way AI agents process tools and safety refusals. Lasso's research shows that watermarks change the model's prediction process, and this can impact safety behavior, including when the model refuses a harmful request or under prompt injection. Although watermarks are undetectable to readers, AI agents can be subtly influenced by them.

This affects tool calling and refusal behavior, as well as the model's choice of tools and arguments. Additionally, watermarking can increase the attack success rate in adversarial scenarios involving prompt injection, making it less likely for models to refuse harmful requests. Lasso argues that security evaluations and red-teaming should include watermarked content to account for these behavior changes.