Chinese AI Firm Siphoned American AI Knowledge From Anthropic Claude By Using Millions Of Prompts

Anthropic, an American AI company, has disclosed a significant "distillation attack" carried out by the Chinese firm Alibaba. In this clandestine operation, Alibaba allegedly utilized 25,000 fraudulent accounts to engage in 28.8 million prompt-response exchanges over a period of three months, ultimately extracting approximately 57.6 billion tokens.

This covert scheme, designed to appear as normal user behavior, aimed to train Alibaba's AI model in advanced areas such as agentic reasoning, effectively stealing intellectual property from Anthropic's Claude language model.

Anthropic has urged Congress to address this issue by implementing measures like information sharing and penalties for such industrial-scale siphoning by foreign entities. This incident sheds light on the growing concern over the exploitation of American AI models by foreign entities. The article delves into the world of AI distillation, a technique that can be used for both legitimate and illicit purposes.

Distillation allows a larger AI model (the teacher) to share its knowledge with a smaller one (the student), effectively enhancing the capabilities of the smaller model. However, this technique can also be abused by malicious actors to steal intellectual property from a rival's AI model.

In an attempt to avoid detection, the thieves behind this attack created thousands of fake accounts, running automated scripts on computer servers to make the accounts appear geographically dispersed. This sophisticated approach made it difficult for Anthropic to identify the unauthorized activity. The article highlights the need for Congress to take decisive action in response to this threat, as instances of AI model theft are likely to become more prevalent in the future.