The Download: a biological de-aging contest and why LLMs don’t reason

In March 2016, a program developed by the author of this story demonstrated its reasoning capabilities by placing a stone on the fifth line of a Go board during a match against Lee Sedol, a legendary professional Go player. The move was so surprising that some commentators initially believed it was a glitch in the program's programming.

However, Lee Sedol later acknowledged that AlphaGo's ability to make such a move indicated that it was indeed creative. This move, move 37 in game two of the five-game match, showcased AlphaGo's genuine reasoning abilities, which are not present in today's AI models.

AlphaGo's reasoning process involved two separate systems. The first system, its policy network, was trained to predict moves made by a strong human player. This intuitive part considered move 37 to be a decent, yet not exceptional, play, with a roughly one in 10,000 chance of being executed by an expert human. The second system, the search machinery, went beyond immediate plausibility and weighed the future consequences of proposed moves.

It built and searched a game tree with thousands of branches, each representing a different possible future.

This architecture of AlphaGo mirrors a well-known theory in behavioral sciences, which distinguishes between two modes of human thought: System 1 and System 2. System 1 is fast, gut-level, and effortless, while System 2 is slower, step-by-step, and deliberative. In AlphaGo, its neural networks provided the hunches, while its search provided the deliberation, testing those hunches against the moves and countermoves that would follow.

In contrast, today's AI models, such as large language models, operate differently. They generate the next token over and over again, which is akin to System 1 in human cognition—fast, associative, and surprisingly good at pattern completion across almost every subject people write about. However, these models lack genuine reasoning capabilities.

They maintain no explicit, persistent, and inspectable epistemic state, and there is no clean separation between what the system knows and how it manipulates that knowledge. Additionally, the chains of thought produced by these models often look like deliberation, but they are actually concocted after the fact, reaching an answer by one route but reporting another.

As a reporter who worked on AlphaGo, I recently left my position at Google DeepMind due to my belief that we need a new approach to machine reasoning, inspired by AlphaGo's architecture. An ideal system should maintain an epistemic state that represents what the system knows, doubts, has ruled out, and which questions remain open.

Reasoning can then be understood as a continuous process of updating this epistemic state as new information is gathered. This approach would allow us to better understand the reasoning behind AI's conclusions, particularly in high-stakes applications such as medicine, engineering, and scientific research.