MIT Researchers Used GPT-6 Astra To Edit Qwen's Open Weights Directly, Boosting Math Scores

MIT researchers from the Computer Science & Artificial Intelligence Laboratory (CSAIL) have utilized OpenAI's GPT-6 Astra language model to significantly enhance the mathematical abilities of a smaller open-source model named Qwen. By directly editing the weights of Qwen, the researchers managed to boost its math score by sixfold.

This innovative technique, dubbed "weight-space programming," allowed the Astra model to interpret and modify Qwen's neural network without relying on traditional training methods. The most remarkable result showcased Astra's ability to increase the MATH-500 score of Qwen3.5-0.8B from a mere 7.9% to an impressive 48.1%, while altering only about 0.016% of its weights.

The improvement was primarily achieved by correcting repetitive reasoning patterns. Yuchen Jin, a technical staffer at Databricks, hailed the research as "cool as heck" and compared it to gene editing for neural networks, suggesting it could revolutionize the field. Chandan Singh, a staff researcher at Voyage AI and former Microsoft AI employee, further emphasized the potential of the approach, expressing that the researchers have likely "undersold" the impact of their findings.