Irony alert: OpenAI whines that Chinese model stole its special IP that it stole from everybody else
US model makers can train on web data - but distilling theirs is a 'national security risk'
OpenAI, a company known for its vast collection of internet content, has accused Chinese AI firm Moonshot AI of engaging in a "distillation attack" that began on July 1st. This attack aimed to extract reasoning from OpenAI's models at scale, potentially enabling rivals to train capable models without retaining the same safety measures. Distillation is a machine learning technique where one model's outputs are used to train another, often in an adversarial manner to reproduce the larger model's reasoning and capabilities.
In a blog post, OpenAI revealed that it had detected and stopped the adversarial distillation campaign that ran for most of July. The operators did not breach OpenAI's encryption, databases, or access user conversations directly. Instead, they manipulated model interactions to reproduce protected reasoning in a coordinated and scaled manner, violating OpenAI's terms of service.
The queries began slowly but saw high-volume spikes on July 24 and 25, with 16,000 requests using a specific extraction pattern from over 4,000 users. OpenAI traced the origin of the theft back to Moonshot AI, which developed Kimi. While it's unclear if all operators were affiliated with just one rival AI company, the "core cluster" of the theft came from Moonshot AI.
OpenAI emphasized the safety and national security risks posed by adversarial distillation, stating that extracted reasoning could be used to train another model without preserving the safeguards applied to the original model's user-facing outputs. The company also noted that at scale, distillation can accelerate the transfer of advanced capabilities without requiring the same investment in safety.
In response to the incident, OpenAI banned the model-copying accounts, tightened signup and infrastructure controls, expanded monitoring efforts, and closed a pathway that allowed users to replay and recover encrypted reasoning. The company also collaborated with service providers to prevent such distillation activity from moving to third-party services. Furthermore, OpenAI shared its investigation findings with other AI firms through the Frontier Model Forum and government information-sharing programs.
Written by urgent.news from The Register's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
