Astra reads and writes weights to improve Qwen
The gist
GPT-6 Astra edited 0.016% of Qwen3.5-0.8B's weights, lifting MATH-500 from 7.9% to 48.1%.
The authors call it early evidence LLMs can reason directly about neural weights.
How it unfolded
- Nov 2, 2026A blog post by MIT CSAIL authors describes "Weight-Space Programming," in which frontier LLMs natively read and write neural network weights.
- Nov 2, 2026Asked to improve Qwen3.5-0.8B's math without gradients, GPT-6 Astra identifies a weight issue causing reasoning loops that Qwen's model card warns about and fixes it by editing 0.016% of the weights.
- Nov 2, 2026Under the same sampling settings, MATH-500 rises from 7.9% to 48.1%, and the fix generalizes to instruction following, tripling IFEval.