Garry Tan wants US open-weight AI labs to 'distill' frontier models, too

Garry Tan, CEO of Y Combinator, advocates for U.S. AI labs to engage in distillation techniques alongside frontier model makers, rather than restricting such activities. Distillation, a method where a model learns from another by extensive prompting, is commonly used in AI labs to train new models. Anthropic recently accused Chinese labs of engaging in "illicit distillation attacks" by hiding identities, relying on fraud, and using stolen credentials.

However, Tan disagrees with regulators cracking down on distillation, arguing that it's an overreach for AI labs to dictate their customers' usage of model information. Tan believes that American labs should be free to distill, as it promotes open-weight AI models and gives people access to intelligence trained on public data. He also feels it's important to strike a balance between open-weight and frontier AI labs, as the worst-case scenario would be a single, proprietary company holding immense power in AI.