OpenAI shares its models' perplexity, or how well they predict data, and proposes pacing limits on such metrics, citing a possible 2027 "avalanche" of advances (Jakub Pachocki/OpenAI)
Difficulty of comparison across labs. … - Detachment from actionable evaluations. … - Not accounting for advances outside of pretraining.
We haven't written up this one. Techmeme has the full story — the link below goes straight to it.
Source: openai.com