Why I Was Wrong About AI Companies Pumping the Brakes: It Wasn’t About Data

A recent article I wrote claimed that the recent hesitation from leading AI companies was not due to a desire for safety, but rather a practical issue known as the "data wall." I explained that the public internet had been largely exhausted, and that human-generated text would soon be in short supply. The dominance of synthetic data was putting pressure on the current scaling methods used by these companies.

While my argument about the physical constraints was accurate, I made a critical mistake in my assessment of their strategic motives. I gave these executives too much credit for humility. The reality is that these companies are deliberately imposing restrictions on open-source AI to prevent a surge of competition. In early 2023, a leaked internal Google document, disclosed by SemiAnalysis, revealed that while major tech firms were investing heavily in proprietary models, open-source developers were creating comparable capabilities at a fraction of the cost.

Through techniques such as LoRA fine-tuning, quantization, and lightweight architectures like Meta’s Llama, open-source alternatives were achieving 90% of the performance for 1% of the infrastructure expenses, running locally and privately. This posed a significant threat to the economic model of closed-source AI labs, which relied on high pricing and premium API fees to sustain their operations.

These companies were facing a dilemma: either maintain their control over model access by charging exorbitant prices, or risk losing market dominance to the rapidly advancing open-source community. To prevent this, they resorted to a combination of regulatory measures designed to stifle open-source competition. First, they proposed arbitrary compute thresholds that would require significant investments in security clearances and audits, effectively creating a barrier to entry for smaller players.

Second, they advocated for downstream strict liability, which would hold open-source creators legally responsible for any downstream uses of their models, making it difficult for them to distribute their work. Lastly, they pushed for mandatory kill switches, which would force open-source developers to incorporate remote shutdown capabilities, effectively preventing decentralization.

These regulatory tactics create an environment where open-source AI becomes practically illegal to distribute or develop. The AI industry's current situation is reminiscent of the Linux battle in the late 1990s, where open-source software faced similar threats to its existence. The key difference is that in this case, the stakes are much higher, as the open-source movement could erode the dominant business models of the closed-source AI companies.