Language Models Can Rewrite Their Own Weights
The gist
A 27B language model edited its own weights and gained 6.6 points on 490 unseen problems.
Gains come from one model family and mostly math, so outside checks are pending.
How it unfolded
- Nov 16, 2026The paper "Language Models Can Rewrite Their Own Weights" (arXiv 2611.21885) is published.
- Nov 18, 2026Jiajun Fan posts on X that the model's own edit made it 6.6 points more accurate on unseen problems, and that over eight sessions in a row its accuracy only went up.
- Nov 18, 2026Commentary reports one edit lifted accuracy on 490 unseen problems from 57.2% to 63.8%, another session produced +4.0 points, and a third change was not significant.
Key terms
- weights
- The numerical parameters of a neural network that store what it has learned and determine its behavior.
- language model
- A machine learning model trained on huge amounts of text to generate, summarize, translate, and analyze language.
- gradients
- A first-order iterative optimization method that minimizes a differentiable function by repeatedly stepping in the direction of the negative gradient.
Source: Wikipedia