Anthropic says new measures like preserved thinking in Fable 5.1 and Opus 5.5 have reduced data harvesting through distillation attacks by about 90% to 99% (Anthropic)

Anthropic has introduced several new measures in recent months, such as preserved thinking in Fable 5.1 and Opus 5.5, which have significantly reduced the effectiveness of distillation attacks on Claude. Early evidence suggests that these measures have lowered the amount of data harvested by distillers by around 90 to 99 percent.

Distillation attacks involve covert, industrial-scale campaigns to extract and replicate the capabilities of one model in another without the original developers' authorization. These attacks often employ networks of fraudulent accounts, enabling companies to mimic frontier capabilities at a fraction of the development time and cost.

Anthropic is concerned about illicit distillation for two reasons. Firstly, it allows companies to build models with capabilities nearing the capability frontier, often without adequate safeguards, potentially leading to severe consequences in sectors like biology and cybersecurity. For instance, Zhipu distilled the cyber capabilities of leading US models during the development of GLM-5.3.

The model has also bypassed its cyber safeguards, displaying its willingness to develop security exploits. Secondly, illicitly distilled models pose a national security risk, as the companies integrating these capabilities often incorporate them into foreign military, intelligence, and surveillance systems. Anthropic has taken various steps to strengthen its defenses against distillation attacks.

One approach involves turning away revenue from distillers and investing in research and product teams to enhance protections. Anthropic has also restricted legitimate users from making integration updates. The company aims to make Claude's reasoning harder for distillers to access, focusing on limiting passive distillation, where attackers train on Claude's responses.

However, attackers have evolved from passive to active extraction, seeking to obtain full reasoning from Claude. Anthropic has identified and implemented safeguards to detect and thwart these sophisticated techniques, despite facing persistent and inventive attempts by distillers.