AI isn’t ready to research itself

Artificial intelligence has demonstrated significant advancements in the realm of research itself, capable of refining existing algorithms or even devising new ones. However, as indicated by a study published on the preprint server arXiv1 in late July, computers are not yet prepared to assume the role of their creators. According to Sayash Kapoor, a computer scientist at Princeton University and co-author of the preprint, "I don’t think full automation of open-ended research is on the horizon right now."

The endeavor to automate the entire scientific process, from conceptualization to paper writing and self-evaluation, was spearheaded by a team mainly affiliated with Sakana AI in Tokyo. This team introduced their system, dubbed The AI Scientist, in 2024, and later reported improved outcomes in March in Nature2. The AI Scientist was assigned the task of investigating challenges within machine learning.

Of the three papers it produced that were submitted for peer review at a conference workshop, one managed to secure acceptance. However, according to Kapoor, peer review serves as an unreliable gauge for evaluating paper quality, especially in AI research. Furthermore, some researchers have cast doubt on whether automating AI research from inception to publication can yield genuine breakthroughs, arguing that it is better suited for optimizing existing techniques.

To elevate AI's standards beyond mere peer review, Kapoor and his colleagues devised a novel challenge called shadow evaluation. They selected two papers submitted to this year's Neural Information Processing Systems conference. The AI tool was tasked with conducting research and authoring papers based on a research question from each paper, with the original authors then tasked with evaluating the AI's output.

The system was built by incorporating the large language model Claude Opus 4.8 into a modified version of the agentic system OpenClaw, which was then enveloped in a broader 'scaffold' containing general instructions and progress-checking mechanisms. This harness provided the model with an assortment of tools, including the ability to generate sub-agents, access the internet, utilize software libraries, employ computer processors for experiments, and incorporate software that emulates peer review.

Each paper had a time limit of six days and a budget of US$3,000 in computing credits. The team set out to test the AI's prowess in the engineering aspects of its research. Surprisingly, the system excelled at the practical components of its research tasks. It was able to conduct hundreds of experiments over several days without encountering dead-end errors.

Moreover, it performed competent literature reviews and made minor discoveries. It also detected its own false claims (commonly known as hallucinations) and refrained from shortcuts (or 'reward hacking'), despite the authors' predictions. Nevertheless, the AI system largely faltered in its assigned objectives, garnering overall scores of 2/6 and 1/6 from the original papers' authors.

A common pitfall was the system's tendency to select a limited number of hypotheses to explore, prematurely fixating on one and failing to sufficiently backtrack when its approach proved ineffective. Post-self-review, the system's negative assessment was not strenuous enough, leading it to persist with its initial choices, gradually whittling down its claims until it produced minimal output. doi: https://doi.org/10.1038/d41586-026-02494-5 Kirgis, P. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2607.27191 (2026).