Researchers say LLMs can be good at natively reading and writing neural network weights; GPT-6 Astra edited Qwen3.5-0.8B's weights to make it better at math (Rikiya Takehi)

Researchers have discovered that large language models (LLMs) can effectively read and write neural network weights, surpassing what would typically be expected from humans or conventional algorithms. This capability could have significant implications for optimization and interpretability. GPT-6 Astra, for example, can analyze the raw weights of a network and determine what it was trained to do without any environment feedback.

Moreover, it can generate small game-playing networks that outperform networks trained for 100 million steps using PPO, without any gradient descent or training loops. By identifying an issue in its weights that causes reasoning loops in Qwen3.5-0.8B, Astra was able to correct this issue by editing just 0.016% of the weights. This correction resulted in a substantial improvement, raising MATH-500 from 7.9% to 48.1% and enhancing IFEval by three times.

This new approach is referred to as Weight-Space Programming, where LLMs reason about weights rather than relying on traditional optimization methods.