Gürkan Özkan

AI

I changed 2,048 numbers in a 0.8B model and it stopped looping

Claude Opus 5.5 edited the weights of Qwen3.5-0.8B by hand, no training; two neurons took MATH-500 from 4.5 to 55.5 by getting the model to finish its thinking, and the honest half of the story is that it adds no knowledge.

Gürkan Özkan8 min read

For agentsView .mdOpen in ClaudeOpen in ChatGPT
For agentsView .mdOpen in ClaudeOpen in ChatGPT
Contents
MATH-500 accuracy after each editOn 100 MATH-500 problems, two samples each: the original scores 4.5, the authors' edit 50.0, Claude's second session 55.5. The second session beats the original by 51.0 points, with a 95% cluster bootstrap interval of 42.5 to 59.5 drawn on the difference.The original4.5The authors' edit50.0Claude's second edit55.50255075100MATH-500 accuracy, percent+51.0 points over the original, 95% interval +42.5 to +59.5
MATH-500 accuracy after each editOn 100 MATH-500 problems, two samples each: the original scores 4.5, the authors' edit 50.0, Claude's second session 55.5. The second session beats the original by 51.0 points, with a 95% cluster bootstrap interval of 42.5 to 59.5 drawn on the difference.The original4.5The authors'edit50.0Claude'ssecond edit55.50255075100MATH-500 accuracy, percent+51.0 points over the original,95% interval +42.5 to +59.5
Two columns of numbers took MATH-500 from 4.5 to 55.5, past the published edit's 50.0; the interval is on the difference to the original, not on the score.Source: my vLLM runs on 100 MATH-500 problems, two samples each; the interval is my own cluster bootstrap over the 100 problems, 10,000 resamples, not the source post's.
Data
MATH-500 accuracy, percent (100 problems, two samples each)
ModelAccuracy (%)Interval lowInterval high
Original4.5
Authors' edit50.0
Claude second session55.542.559.5

Source: my saved run logs and per-answer result files (res_*.jsonl, scored_*.json); the interval on the second session is my own cluster bootstrap over the 100 problems, 10,000 resamples, not the source post's

Here is a school-level question. Amber, Micah and Ahito ran 52 miles in total. Amber ran 8 miles. Micah ran 3.5 times what Amber ran. How far did Ahito run?

Qwen3.5-0.8B works it out early: Micah ran 28, so Ahito ran 52 − 8 − 28 = 16. Then it writes “I’ll write the final answer.” and writes it again. It is a GSM8K problem, one of 100 the model ran, and in the saved answer the sentence appears 2,741 times, until the 32,768-token budget runs out. No answer ever appears. Under Claude’s second edit the same question finishes in 1,771 tokens with 16 as the answer.

The claim behind this post is that an LLM read the model’s weights, guessed what caused the loop, tested its guesses and rewrote two columns of numbers. Nobody trained anything. I ran the checks. What follows is what changed, what I verified myself, and what I cannot claim.

The bug, measured

Qwen3.5-0.8B thinks before it answers: it reasons between <think> and </think> tags, then writes a final answer. Its own model card warns that it can get stuck repeating itself while thinking. I measured how often on 100 MATH-500 problems, two samples each: 96% of the original model’s answers hit the 32,768-token limit without finishing. Of the 8 that did finish, 7 were right. The model often knows the answer. It just cannot stop.

The idea of fixing this by editing weights is called weight-space programming, from Rikiya Takehi and colleagues at MIT CSAIL. Their blog post (a blog write-up, not a peer-reviewed paper, as far as I can tell) reports a model they call GPT-6 Astra raising Qwen3.5-0.8B on MATH-500 from 7.9% to 48.1% by editing 0.016% of its weights. I did not test their other claims. I tested the Qwen result.

The published edit exists, and I wanted to see whether it works. I downloaded the original and the authors’ edited weights and compared them value by value: of 488 tensors, exactly two differ, the down_proj matrices in layers 15 and 19, 64 columns each, 130,188 values. Then I ran both models on the same 100 problems with the same settings: accuracy went from 4.5% to 50.0%. The authors report 7.9% to 48.1%; my baseline sits below theirs and I have not tested why. The result holds up.

One answer, before and after the second editThe original writes its reasoning, then repeats "I'll write the final answer." 2,741 times until the 32,768-token budget runs out; no answer ever appears. Under Claude's second edit the same question closes its thinking with </think> and writes 16 as the answer in 1,771 tokens; the authors' edit finishes the same item at 7,540 tokens.The original, one GSM8K answer32,768 tokensreasoning<think>I’ll write the final answer.× 2,741no answercut at 32,768The sentence repeats until the token budget runs out. No answer ever appears.Claude’s second editThe same question after the edit1,771 tokensreasoning<think>repeat</think>16The exit opens only while the model repeats. It writes 16 in 1,771 tokens (the authors’ edit: 7,540).
One answer, before and after the second editThe original writes its reasoning, then repeats "I'll write the final answer." 2,741 times until the 32,768-token budget runs out; no answer ever appears. Under Claude's second edit the same question closes its thinking with </think> and writes 16 as the answer in 1,771 tokens; the authors' edit finishes the same item at 7,540 tokens.The original, one GSM8K answer32,768 tokensreasoning<think>I’ll write the final answer.× 2,741no answercut at 32,768The sentence repeats until the token budget runs out. No answerever appears.Claude’s second editThe same question after the edit1,771 tokensreasoning<think>repeat</think>16The exit opens only while the model repeats. It writes 16 in 1,771tokens (the authors’ edit: 7,540).
The loop itself: a GSM8K answer that repeats one sentence 2,741 times and dies at the 32,768-token limit, and the edited model's stream that closes the thinking and answers in 1,771 tokens.Source: the saved run of the one GSM8K item in my result files (suite_base_gsm8k.jsonl, suite_opus2_gsm8k.jsonl); a schematic of one answer.

The workbench

Then I gave Claude Opus 5.5 (the Claude CLI reports claude-opus-5-5) a locked-down workbench: three tools and nothing else, a Python notebook with the weights loaded, a tool that runs the model, a status tool. No shell, no files, no internet. The budget was 50 runs of the model, at most 14 of them generation runs. The kernel server enforced rules in code: gradients raise, optimisers are unimportable, edits to the embedding, output layer and final norm are refused, and so is scaling a whole tensor by one constant. Two more rules lived only in the prompt: no other way of fitting weights to a loss, no simulating the network by hand. I scanned Claude’s logs for training code and found none, but I did not audit every cell for hand-run forward passes.

Claude saw practice problems from the GSM8K and MATH training sets. It never saw the benchmark questions I scored on.

The first session changed 32 tensors and 92,273,111 values, about 10.6% of the file. Its MATH-500 score matched the authors’ at 50.0%. On HumanEval, a coding test, it scored 27% against 30% for the unedited model: better at math, worse at code.

The second session was the same setup with one prompt change. I told Claude that coding, instruction following and general knowledge would also be checked, and asked it to write its own small checks and use them on every candidate. It came back with two neurons.

What the two edits do

Each MLP layer of the model has a block of 3,584 neurons. A neuron reads the layer’s state, decides how strongly to fire, and writes something back: one column of a matrix called down_proj. Change the column, change what that neuron does. Claude changed exactly two columns, 1,024 numbers each.

Edit 1: layer 22, neuron 573, column multiplied by 2. This neuron goes strongly negative where the word “Wait” is likely, and acts as a brake on it. Doubling the column doubles the brake. I checked the diff myself: the ratio is exactly 2.0 for all 1,024 values, and every other column in that matrix is untouched.

Edit 2: layer 19, neuron 11, one vector added. This neuron counts repetition; Claude measured it around −0.3 on fresh text and around −3.4 after a line has repeated eight times. The added vector (norm 0.45) points away from the </think> token, cosine −0.95, which I computed from the saved tensors. Because the neuron fires negative during a loop, the two negatives multiply into a push towards closing the thinking. In Claude’s tests it works as an exit door that opens only while the model is repeating itself. Claude’s own report is explicit that this half adds a new connection rather than repairing something broken.

I measured the first edit’s effect myself rather than trust Claude’s number. I took 30 truncated GSM8K answers from the original run, found the 375 spots where the model wrote “Wait”, and asked each model how likely it would be to write “Wait” at exactly those spots. The original gives it 34%, the authors’ edit 26%, Claude’s edit 1.7%. At other positions the probability is near zero for every model (means about 0.3%, 0.2% and 0.03%). The two routes reach the same goal differently: the authors describe their edit as subtracting a repetition direction from 64 columns in each of two layers (my diff confirms the 64 columns, not what was subtracted); Claude’s takes away the habit of saying “Wait” and adds an exit.

Where the two edits sitClaude changed one down_proj column in layer 22 (neuron 573, multiplied by 2: the brake on "Wait") and added one vector to the column of neuron 11 in layer 19 (norm 0.45, cosine −0.95 to </think>): 1,024 numbers each, 2,048 in total. The authors' edit changed 64 output columns in each of layers 15 and 19, 130,188 values.Each MLP layer holds 3,584 neurons; a neuron is one column of down_proj.one neuronEdit 1 · layer 22, neuron 573The column is multiplied by 2: the brake on “Wait” doubles (34% to 1.7% in the check).Edit 2 · layer 19, neuron 11A vector (norm 0.45) is added to the column: it opens the exit toward </think> while the model repeats.1,024 numbers each, 2,048 in totalthe authors changed 130,188 values
Where the two edits sitClaude changed one down_proj column in layer 22 (neuron 573, multiplied by 2: the brake on "Wait") and added one vector to the column of neuron 11 in layer 19 (norm 0.45, cosine −0.95 to </think>): 1,024 numbers each, 2,048 in total. The authors' edit changed 64 output columns in each of layers 15 and 19, 130,188 values.Each MLP layer holds 3,584 neurons; a neuron is one columnof down_proj.one neuronEdit 1 · layer 22, neuron 573The column is multiplied by 2: the brake on “Wait” doubles(34% to 1.7% in the check).Edit 2 · layer 19, neuron 11A vector (norm 0.45) is added to the column: it opens theexit toward </think> while the model repeats.1,024 numbers each, 2,048 in totalthe authors changed 130,188 values
The two edits in the layer stack: a doubled brake column in layer 22 and an added exit vector in layer 19, 2,048 numbers against the authors' 130,188.Source: my tensor diffs of the saved checkpoints, re-checked by an independent audit.
Data
Values changed, per tensor and in total
EditValues per tensorValues in total
Claude's two columns1,0242,048
The authors' edit65,260130,188

Source: my saved run logs and per-answer result files (res_*.jsonl, scored_*.json); the diff of the authors' checkpoint

Results

All numbers use the same prompts and sampling settings (thinking on, temperature 0.6, top_p 0.95, top_k 20, presence penalty 1.5, up to 32,768 new tokens) and the benchmarks’ own answer checkers. Raw outputs are saved.

On MATH-500 (100 problems, two samples), Claude’s second edit scores 55.5% against the original’s 4.5%. I computed a cluster bootstrap over the 100 problems, 10,000 resamples: +51.0 points, 95% interval +42.5 to +59.5. Against the authors’ edit it is ahead by 5.5 points with an interval of −2.0 to +13.0 (the upper end moves to +13.5 under some bootstrap seeds), which is not a difference I can call real. What I can say is that it matches the published fix while changing about 60 times fewer numbers: 2,048 against 130,188.

The gain comes from finishing, not from new knowledge. The share of answers that never finished fell on every task; on MATH-500 it dropped from 96% to 24.5%, and on GSM8K from 87% to 4%.

How 200 MATH-500 answers ended, per modelOne square per answer, of 200 per model: the original finished 8 and got 7 of them right, the authors' edit finished 133 and got 99 right, Claude's second edit finished 151 and got 111 right. The rest were cut at 32,768 tokens: 192, 67 and 49.finished and rightfinished, wrongcut at 32,768 tokensOriginal7 right · 1 wrong · 192 cutAuthors' edit99 right · 34 wrong · 67 cutClaude second edit111 right · 40 wrong · 49 cut
How 200 MATH-500 answers ended, per modelOne square per answer, of 200 per model: the original finished 8 and got 7 of them right, the authors' edit finished 133 and got 99 right, Claude's second edit finished 151 and got 111 right. The rest were cut at 32,768 tokens: 192, 67 and 49.finished and rightfinished, wrongcut at 32,768 tokensOriginal7 right · 1 wrong · 192 cutAuthors' edit99 right · 34 wrong · 67 cutClaude second edit111 right · 40 wrong · 49 cut
Each row is all 200 answers: the gain is in finishing — cut answers fell from 192 to 49 while most of what finished was right.Source: my saved per-answer result files: my saved run logs and per-answer result files (res_*.jsonl, scored_*.json).
Data
MATH-500 answers of 200, by how they ended
ModelFinishedRight among finishedCut at 32,768 tokens
Original87192
Authors' edit1339967
Claude second edit15111149

Source: my saved run logs and per-answer result files (res_*.jsonl, scored_*.json)

The base model finished 8 of 200 answers and 7 of those were right; Claude’s second edit finished 151 and got 111 of them right, 74% of what finished. The edited model itself got no smarter at solving; it just stopped running out of room. The knowledge was very likely in there already, stuck behind the loop, though I did not test that directly.

On the other four benchmarks, 100 items each, one sample: GSM8K 14 to 74, IFEval 24 to 65, MMLU-Pro 24 to 39, HumanEval 30 to 36. The chart draws both ends of that before-and-after; with one sample per item the standard error is about five points, so the HumanEval gap sits inside the noise, and the first session’s coding drop (27% against 30%) sits inside it too, so I read it as a warning and not a finding: only the smaller edit is known to have kept both, and I cannot say the coarse edit cost any coding ability.

The four other benchmarks, original to second editPercent of 100 items, one sample each, from the original to Claude's second edit: GSM8K 14 to 74, IFEval 24 to 65, MMLU-Pro 24 to 39, HumanEval 30 to 36.the originalsecond editGSM8K1474IFEval2465MMLU-Pro2439HumanEval30360255075100percent of 100 items, one sample eachHumanEval, 30 to 36, sits inside the noise: the standard error is about five points.
The four other benchmarks, original to second editPercent of 100 items, one sample each, from the original to Claude's second edit: GSM8K 14 to 74, IFEval 24 to 65, MMLU-Pro 24 to 39, HumanEval 30 to 36.the originalsecond editGSM8K1474IFEval2465MMLU-Pro2439HumanEval30360255075100percent of 100 items, one sample eachHumanEval, 30 to 36, sits inside the noise: the standard error isabout five points.
Everything rises with the smaller edit; only HumanEval, 30 to 36, stays inside the noise.Source: my scored result files, one sample per item; the standard error is about five points.
Data
Four other benchmarks, percent of 100 items, one sample each
TaskOriginal (%)Second edit (%)
GSM8K1474
IFEval2465
MMLU-Pro2439
HumanEval3036

Source: my saved run logs and per-answer result files (res_*.jsonl, scored_*.json); before the original model, after Claude's second edit

What I cannot claim

One run per condition, small samples, one machine (a DGX Spark, vLLM). The second editing session was not blind: I told it which kinds of tasks would be checked, and I cannot separate the effect of that instruction from luck. A third session started from the second edit and tried to improve it; the session’s own report lists six candidate edits and it saved five candidates, and none beat the starting edit, so it changed nothing. The best candidate cut the unfinished share from 32% to 22% but did not raise accuracy (55.1% against 54.7% on 256 problems, standard error 2.6): forcing a stuck problem to end mostly produced wrong answers.

With “Wait” suppressed, self-correction is mostly gone. Claude reported that its exit neuron also fires during some legitimate long lists, so it can close the thinking early; my benchmarks were not built to measure that cost. About a quarter of MATH-500 answers still hit the limit, and many look like hard problems the small model is genuinely working on. This fixes a behaviour, not knowledge.

It is one model, and one with a mechanical, obvious failure. The authors tried the method on a small Gemma that rarely loops and found nothing to fix. I measured Qwen3.8 on 24 MATH-500 questions with thinking on and it got all 24, so there is no headroom there to show anything either.

Why it is interesting anyway

A training run produces a new set of weights and nobody can say which of the millions of numbers matters. This edit is two columns in two layers, each with a stated reason, and you can read the diff. You can undo either half separately. Claude tried exactly that and recorded the dose-response on a 64-problem dev set: brake at ×1, ×1.5, ×2, ×3 gives 48.4%, 65.6%, 68.8%, 64.1%.

The first session took about 7.3 hours and 27 forward passes; the second, 4.3 hours and 31; the third, about 6.1 hours and 24. The CLI reported roughly 5.94and5.94 and 6.88 of API-equivalent usage for the two editing sessions; the third cost $7.96. For a 0.8-billion-parameter model that could not finish its thoughts, that was enough to find a fix as good as the published one. Whether the same hours buy anything on a model without such an obvious bug is the open question, and I have not tested it.

Claude wrote its own probe code inside the sandbox; the harness, prompts and evaluation were mine.