OpenAI’s chief scientist has thoughts on how to limit AI development
To establish a speed limit on AI development, you need to agree on how to measure speed.
In a recent blog post, OpenAI's chief scientist Jakub Pachocki has proposed potential limits on AI development, focusing on a measurement called perplexity. This notion gained renewed attention following former OpenAI and Anthropic researcher's concerns about the companies' risks to society, including Anthropic's IPO filing that highlighted the potential for catastrophic or existential risks.
In response, Anthropic CEO Dario Amodei suggested a plan to "pace the frontier" of AI development, an idea supported by OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Amodei advocated for companies to agree on "common safety standards as well as limits on the rate of unchecked AI progress."
However, to implement such limits, a universally accepted measurement for AI speed is required. Pachocki's post, considered by Altman as the most significant work OpenAI has published in a while, suggests that perplexity may serve as this measurement. Perplexity, distinct from the company with the same name, calculates how accurately a model predicts the next token, with lower perplexity indicating better performance.
Although benchmarks that evaluate AI's downstream capabilities have become outdated due to rapid advancements, perplexity has remained informative.
Pachocki includes a chart in his post demonstrating the decline in perplexity for OpenAI's models since GPT-3.5. He explains that GPT-4, with a perplexity of approximately 0.91, demonstrated proficient chain-of-thought reasoning, motivating further development of modern reasoning models. As AI models' perplexity has dropped below 0.8, transformative capabilities have emerged in areas like cybersecurity and mathematics.
By setting a shared benchmark for general intelligence based on perplexity, AI companies could then establish "common limits" on AI development. Pachocki proposes that once perplexity falls below a certain threshold, companies would commit to not surpassing this limit "until our ability to understand and monitor AI behavior achieves a concrete milestone."
Rewritten by urgent.news; may contain errors — check the original.
Source: techcrunch.com