GPT-6 Astra fixes model reasoning by editing 0.016% of weights without retraining OpenAI

Researchers at MIT CSAIL demonstrated Weight-Space Programming, using GPT-6 Astra to directly inspect and edit neural network weights in Python without fine-tuning, backpropagation, or training loops.

In the experiment, researchers gave GPT-6 Astra access to the weights of Qwen3.5-0.8B and permitted up to 50 test forward passes without gradient descent. Astra diagnosed that the small model frequently entered repetitive reasoning loops and hit the 32,768-token limit without producing an answer. By isolating the issue to MLP blocks in layers 15 and 19, Astra adjusted 131,072 parameters out of 800 million (0.016% of total weights), cutting looping failures and significantly improving math benchmark performance.

What it is

• Weight-Space Programming. A paradigm where a frontier large language model reads and writes neural network weight matrices directly using natural language and code, rather than optimizing them through gradient descent.
• Root cause isolation. Astra traced repetitive thinking loops to a specific direction written by the MLPs of layers 15 and 19, then subtracted that direction across 64 output columns in each layer.
• Zero gradient updates. The intervention required no training data passes, no backpropagation, and no optimizer updates like Adam or Muon.
• Targeted capability unlocking. The weight correction enabled the model to finish reasoning trajectories it already possessed the latent knowledge to solve.
• Limits on unlearned capabilities. When tested on Gemma 3 270M, which does not suffer from reasoning loops, similar direct weight edits did not improve math accuracy, showing that weight editing can unblock hidden latent capabilities but cannot impart entirely missing knowledge.

Benchmark results

Sources

• Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights