The AI Data Satiation Point: Why 'Wild' AI Text is Breaking Chinchilla Scaling Laws

As of August 2026, nearly one-third of all web tokens collected are AI-generated, marking a significant shift in the digital landscape. This influx of AI-generated content, or "wild" AI text, presents a major challenge for researchers working on the next generation of frontier models. A new study from Pangram Labs and the University of Massachusetts Amherst, titled "How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text," provides valuable insights into the impact of this data on model training.

Contrary to earlier fears of "model collapse," the study reveals that AI-generated tokens can initially help data-starved models by providing a scaffold for learning grammar, reasoning, and information retrieval. However, once a model has been trained on a substantial amount of high-quality human text, adding more AI tokens begins to harm the model's performance. The study emphasizes that the value of an AI token is not static but changes based on the model's existing data intake.

The researchers propose a new scaling law that distinguishes between the beneficial and harmful effects of AI tokens, allowing for a more accurate prediction of model performance as the model scales. This law, which accounts for the varying quality and origin of tokens, challenges the traditional assumption that all tokens are of equal value.

To address the issue of "wild" AI text, the study emphasizes the importance of filtering AI-generated content for human-centric tasks. While AI text may still offer some benefits for certain tasks, it should not replace the high-quality human data that is crucial for generalizing model performance. The researchers release the WildAI corpus, an 83-billion-token dataset labeled by topic, format, and source, to aid researchers in identifying and filtering AI-generated content.

In summary, the rise of AI-generated content poses a complex challenge for AI model training. While some AI tokens can initially help data-poor models, adding too many AI tokens can harm a model's performance. Researchers must carefully filter AI-generated content and use the new scaling law to optimize their training strategies.