Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model
Looped models can scale their effective depth and enable multi-step reasoning through recurrent blocks. However, they lack an explicit measure of progress, making it challenging to determine when to stop looping and how to enhance the current state. We introduce Energy-Based Reasoners (EBRs) that learn an energy function over latent reasoning states and implement looped updates as gradient descent on this energy.
The energy scores latent states, with its gradient guiding refinement and signaling when to cease. To train EBRs effectively, we also present exploration-based training, which uncovers additional modes and refines the energy landscape across a wider range of states.
EBRs surpass previous recurrent reasoning models in four algorithmic reasoning benchmarks, achieving 99.9% accuracy on Sudoku-Extreme while utilizing 345× less inference compute than the leading model and preserving comparable training compute. On all multi-solution benchmarks, EBRs provide state-of-the-art coverage of unique valid solutions while preserving near-100% single-sample accuracy.
The energy function evaluates the compatibility between the reasoning state z and the initial board x; a lower score indicates better compatibility. Refinement occurs as the gradient transforms the score into an improvement direction. The state stops refining when the gradient is minimal, acting as a stop criterion. Among multiple candidate solutions, the one with the lowest energy is chosen.
Additional refinement steps improve the answer, and more trajectories provide the energy with more candidates to select the lowest-energy solution from. Each trajectory concludes once its energy gradient falls below a threshold, with easy puzzles finishing in a few steps and harder ones continuing further.
EBRs function similarly to stochastic gradient descent, enabling them to capture a multitude of solutions. Coverage measures the proportion of each input's distinct valid solutions that 20 samples uncover.
Written by urgent.news from Lobsters's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.