TL;DR
Get everyday essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A November 2 blog post by four researchers listed with MIT CSAIL reports that GPT-6 Astra edited two weight tensors in Qwen3.5-0.8B, raising its MATH-500 score from 7.9% to 48.1%. The edit reduced, but did not eliminate, long reasoning loops; the result comes from an early, unreviewed report and used an iterative search with substantial test feedback.
Four researchers listed with MIT CSAIL report that OpenAI’s GPT-6 Astra edited two layers of the open-weight model Qwen3.5-0.8B, raising its score on the MATH-500 benchmark from 7.9% to 48.1%. The reported change targets a known tendency to repeat reasoning until reaching a token limit; it is an early result published in a blog post, and the loops remained common after the edit.
The researchers published their post, Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, on November 2. They say Astra identified an internal direction associated with looping and subtracted it from output columns in the model’s multilayer perceptrons at layers 15 and 19. The edited tensors account for 0.016% of the model’s 0.8 billion weights; the post says the edit changed 131,072 weights.
The reported MATH-500 score rose from 7.9% to 48.1%, while the share of answers that reached the 32,768-token limit fell from 93.0% to 33.1%. The benchmark score and token-limit figures describe the researchers’ reported evaluation. Their post says the benchmark questions were withheld from Astra, which instead received practice problems from GSM8K and MATH.
The process did not use gradient descent, backward passes or fitting weights to a loss, according to the researchers. It did involve repeated model checks and code execution: the winning run used 158 turns, 80 Python runs and 50 forward passes over 20.3 hours. Each forward pass could evaluate up to 512 problems. The authors report that runs with 10 and 25 passes scored 12.3% and 14.1% on MATH-500, respectively.
AI Repairs AI: GPT-6 Astra Edited Another Model’s Weights. A Loop Fix, Not an IQ Jump
Researchers report that GPT-6 Astra changed a small part of Qwen3.5-0.8B to reduce repetitive reasoning. The reported math score rose, but loops remained and broader effects are unknown.
The reported change
Astra edited two layers in Qwen3.5-0.8B, targeting a direction associated with repetitive reasoning.
The researchers compared looping and non-looping reasoning texts.
Astra subtracted the associated direction at layers 15 and 19, changing 131,072 weights.
The iterative search used 158 turns, 80 Python runs and 50 forward passes.
What the numbers show
The reported evaluation links fewer token-limit answers with a higher MATH-500 score.
What it may mean for users
A targeted weight edit could help teams address a specific failure in an open-weight model.
- Target a narrow unwanted behavior.
- Explore repair without gradient-based training.
- Whether other tasks or outputs are affected.
- Whether independent teams can reproduce the result.
A Targeted Fix for Open Models
The result points to one possible use for models that can inspect and edit open model weights: diagnosing a narrow failure and changing the parameters linked to it. For organizations running open-weight systems, that could offer a way to repair certain behaviors without conducting conventional gradient-based training, though this report does not establish how reliably the approach works across models or tasks.
The change is best understood as a reported repair of a specific weakness, rather than evidence that Astra added general mathematical ability to Qwen. The model still reached its token limit on 33.1% of MATH-500 answers after editing. The researchers themselves caution that fixing loops is not the same as making a model better at math.
Any such weight change also produces a modified model that needs to be evaluated on its own. A patch that reduces one unwanted behavior could affect other outputs; the post’s reported benchmark results do not by themselves establish the full effect of the edit in other uses.
Popular right now
Ninja Flip Toaster Oven & Air Fryer, Large Capacity, Stainless, SP151
Breville BES870BTR Barista Express Espresso Machine, Black Truffle
Breville BES870XL Barista Express Espresso Machine, Brushed Stainless Steel
As an affiliate, we earn on qualifying purchases.
The Looping Problem in Qwen
Qwen’s model card warns that Qwen3.5-0.8B is prone to thinking loops. In the researchers’ initial practice checks, 85 of the first 96 answers ran to the model’s 32,768-token limit. Their experiment compared looping and non-looping reasoning texts, then used Astra to find a direction in the weights associated with the repetition.
The post describes eight experiments exploring whether a language model can read and edit neural network weights. The authors call the work “very early results” and say the capabilities remain limited. The post is not a peer-reviewed paper, according to the source material. It also reports a failed experiment in which Astra could not determine from weights alone which of two fine-tuned Gemma models had been trained to flatter users.
The post cites earlier work on removing refusal behavior from open chat models by deleting a direction from their weights. That provides a precedent for the kind of intervention described, but it does not establish that the Qwen math result will generalize. The source says OpenAI was not an author of the post and had made no statement about the experiment.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
How Far the Edit Generalizes
The reported figures do not show whether the edit improves performance on other math benchmarks, or how it affects the model’s behavior beyond the tests described. It also remains unclear whether independent researchers can reproduce the result using the published logs and edited model.
The source material identifies the post as an early report rather than a peer-reviewed paper. It does not provide independent validation of the score increase, a full account of possible side effects across other tasks, or a statement from OpenAI about Astra’s role.
Replication and Broader Evaluations
The next useful evidence would be independent replication and evaluations of the edited model on additional math and non-math tasks. Those checks could show whether reducing loops preserves other behaviors and whether the improvement holds beyond the reported MATH-500 evaluation.
The researchers have published logs and an edited model on Hugging Face, according to the source material. The post does not announce a follow-up paper, review date or further milestone, so the timing of additional results is unknown.
Key Questions
What did GPT-6 Astra change?
The researchers say Astra subtracted a direction associated with looping from output columns in the model’s multilayer perceptrons at layers 15 and 19. They report changes to 131,072 weights, or 0.016% of Qwen3.5-0.8B’s parameters.
How did the reported MATH-500 score change?
The blog post reports an increase from 7.9% to 48.1% after the edit. It also reports that the share of answers hitting the 32,768-token limit fell from 93.0% to 33.1%.
Was this done without training?
The researchers say they did not use gradient descent, backward passes or loss-based fitting. The winning result still involved an iterative search with 158 turns, 80 Python runs and 50 forward passes over 20.3 hours.
Are the loops gone?
No. The reported share of MATH-500 answers that hit the token limit fell to 33.1%, so the behavior remained in a substantial portion of answers.
Has the result been peer reviewed?
The source material describes it as a blog post and says it has not been peer reviewed. The authors call the findings very early; independent replication is not reported.
Source: Thorsten Meyer AI
