Inside the Race to Make AI Build Itself

Jack Clark, co-founder of Anthropic, returned from paternity leave in February, only to discover that his colleagues were no longer writing code. Instead, they were managing multiple versions of Anthropic's AI, Claude, which was increasingly taking over the research process. This phenomenon, reminiscent of recursive self-improvement, has long been a feared possibility in the AI field.

As models become more proficient, they can expedite research, potentially leading to an "intelligence explosion" where AI surpasses human understanding and control. Clark's concerns prompted him to post blogs on the topic, deliver a presentation on recursive self-improvement in May, and lead an Anthropic report in June titled "When AI Builds Itself," arguing that the technology is already accelerating its development.

The report noted a significant increase in code production per person at Anthropic, with Claude writing 80% of the code. However, skeptics argue that technological progress has always compounded, questioning whether Claude's advancements signal a genuine paradigm shift. The core of the issue lies in the fact that Claude, and other large language models, are not written in code at all.

They are developed through a process of setting growth conditions, pouring an internet's worth of text through them, and allowing them to adjust themselves billions of times until they can perform various tasks. This trial-and-error method of progress raises questions about whether AI could take over this cycle, accelerating development more rapidly or suddenly.

While Anthropic's tests suggest that Claude is increasingly capable of running experiments and even attempting tasks independently, the true extent of this acceleration remains uncertain. The people building the technology may be as much in the dark as everyone else, as evidenced by a survey conducted by Anthropic, which found that five out of 16 researchers believed Claude could replace an entry-level colleague.

This revelation underscores the potential for AI to displace human workers, even as it continues to evolve.