GPT-6 Astra Rewrites Qwen3.5 and Boosts Math Scores by 40%
Rikiya Takehi released an edited Qwen3.5-0.8B on Hugging Face, modified by GPT-6 Astra without any training. Astra changed 0.016% of weights (131,072 values) across two MLP down-projection tensors in layers 15 and 19.
Takeaways
- Rikiya Takehi released an edited Qwen3.5-0.8B on Hugging Face, modified by GPT-6 Astra without any training.
- Astra changed 0.016% of weights (131,072 values) across two MLP down-projection tensors in layers 15 and 19.
- MATH-500 jumped from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, IFEval from 21.0% to 62.3%.
- The real fix was removing a repetition direction causing reasoning loops, not adding math knowledge.
- Produced over 158 conversational turns, 80 Python runs and 20.3 hours of autonomous debugging.
- Full writeup in Sparks of Weight-Space Programming by Takehi, Teoh, Zhang, and Khattab.
Researchers Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab have released a modified Qwen3.5-0.8B checkpoint on Hugging Face. According to the team, OpenAI’s GPT-6 Astra selected the edits by inspecting the model’s parameters, running diagnostics, and evaluating candidate patches. The checkpoint accompanies their project post , Sparks of Weight-Space Programming .
The authors call the method weight-space programming. A weight tensor is a multidimensional array containing model parameters; the workflow gives an LLM access to those arrays, describes a goal in plain English, and provides tools for testing possible changes. Gradient-based training played no role in generating the reported patch.
The score jump tracks fewer reasoning loops
The strongest reported experiment targeted Qwen3.5-0.8B’s math performance. Astra’s intervention changed about 0.016% of the model’s weights and produced the following results:
| Metric | Base model | Edited model | Change |
|---|---|---|---|
| MATH-500 | 7.9% | 48.1% | +40.2 percentage points |
| GSM8K | 13.8% | 67.1% | +53.3 percentage points |
| IFEval | 21.0% | 62.3% | +41.3 percentage points |
| Responses hitting the token limit | 93.0% | 33.1% | 59.9 percentage points lower |
Before the edit, the base model frequently repeated intermediate reasoning until it exhausted its context window. The authors traced that behavior to a numerical direction associated with repetition and subtracted the pattern from selected weight matrices. Under their diagnosis, the benchmark gains reflect abilities that looping had prevented the model from expressing, including producing a final answer and following output instructions.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
