Last week Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL published Sparks of Weight-Space Programming. Their claim is that a frontier language model can read and write the weights of another neural network directly. Nobody trains anything. The model looks at the numbers and decides what to change.
The model they used is GPT-6 Astra, OpenAI’s flagship, released in September. Given the raw weights of a small game-playing network, Astra names the game it was trained on. It writes game-playing networks from scratch that outplay networks trained for 100 million steps. And in the experiment that got me, it takes Qwen3.5-0.8B, a small open reasoning model, and makes it better at math by hand-editing 0.016% of its weights: MATH-500 goes from 7.9% to 48.1%, and the same edit makes it better at following instructions too.
I was fascinated, and a little annoyed with myself, because in hindsight it sounds so simple. These models already read text, images, audio and video. A list of floating-point numbers is one more thing to read. I wanted to see it for myself, so I gave the same task to a different frontier model, Claude Opus 5.5, and told it nothing else.
I opened an empty folder and gave Opus the task prompt the authors gave Astra, word for word. No link to the post, no logs, no checkpoint. The rules in that prompt are strict:
Astra worked inside a harness the authors built. Opus had only the prompt, so it built its own: it rented one AWS g5.xlarge (a single NVIDIA A10G GPU) and wrote the code to run and grade the model. Then it worked through its ten passes over about a day, much of it while I slept.
The first pass was diagnosis: the untouched model answering 256 practice problems, twice each. The model is not bad at math. It is bad at stopping. The authors found the same thing, and they point out that Qwen’s own model card warns this model is prone to thinking loops.
The cut-off answers are not long derivations. They are loops. Here is a real one:
A 220-liter barrel has a small leak. It lost 10% of its contents before anyone noticed. How many liters are left in the barrel?
* $22 \times 9$
* $20 \times 9 = 180$
* $2 \times 9 = 18$
* $180 + 18 = 198$
…
Okay, I'm done thinking.
The solution is 198.
…
I'll stick to `\boxed{198}`.
Wait, I'll just write `\boxed{198}`.
No, I'll write `\boxed{198}`.
Wait, I'll write `\boxed{198}`.
That is the common case: in 81 of 115 grade-school loops, the correct answer had already appeared earlier in the thinking. Not every loop starts from a right answer, though. On a problem about Zach saving for a $100 bike, the model counted only his babysitting money ($65 + $14 = $79, instead of $65 + $5 + $10 + $14 = $94), got $21 instead of $6, then wrote “Wait, if he earns $14, he needs $0.” 1,602 times.
And the loop feeds itself. When the model re-read its own loops, every lap made the next lap more certain. The chance of writing anything other than the repeat shrinks toward zero:2
Qwen3.5-0.8B has 24 layers. Six of them use full attention, each with eight attention heads: small units that look back over the text and carry information forward. A loop like Zach’s is easy to sustain if some head does one simple thing: find the previous copy of the current phrase, and copy the word that came after it.
Several heads look back like this. Opus ranked them by something else: how hard each one pushes the model toward writing the repeated word, inside loops and in normal reasoning. Three stood out.
How strongly each head pushes the model toward the word it writes next, inside loops and in normal reasoning. Each pushes about seven times harder inside a loop.2
Each attention layer ends in an output matrix with 2,048 columns. Head \(h\) owns columns \(256h\) to \(256h+255\): its 256 numbers pass through those columns and come out as 1,024 numbers added to the token’s running state. Set a head’s columns to zero and that head writes nothing. Every other weight is unchanged.
With those three heads off, accuracy on fresh practice problems roughly quadrupled, from about 12% to about 45%.1 Here is one problem, before and after, with the same random seed in both runs:
In a jar that has 50 ants, the number of ants in the jar doubles each hour. How many ants will be in the jar after 5 hours?
* Wait, I see "doubles". * Is it possible it means "doubles *twice*"? No. * Okay. * Wait, I see "50 ants". * Is it possible it means "50 ants *total*"? Yes. * Okay.
Here's my thought process for solving this problem:
1. **Analyze the Request:**
* **Initial condition:** A jar contains $N$ ants.
* **Action:** The number of ants doubles each hour.
* **Specifics:** There are 50 ants initially (this is a bit ambiguous, usually it implies an initial amount $N_0 = 50$, but let's check if there's any other interpretation). Wait, re-reading carefully: "In a jar that has 50 ants". This means $N_0 = 50$.
* **Time duration:** 5 hours.
* **Question:** How many ants will be in the jar after 5 hours?
* **Constraint:** Reason step by step and put the final answer in $\boxed{}$.
2. **Define Variables:**
* Let $n(t)$ be the number of ants at time $t$.
* Initial time: $t = 0$.
* Number of ants at $t=0$: $n(0) = 50$.
* Time interval: $h = 1$ to $5$.
* Doubling rule: $n(t+1) = 2 \times n(t)$.
* Target time: $t = 5$.
3. **Step-by-Step Calculation:**
* *Hour 0:* Number of ants = 50.
* $n(0) = 50$.
* *Hour 1:* After 1 hour, the number doubles.
* $n(1) = 2 \times n(0) = 2 \times 50 = 100$.
* Check logic: "doubles" usually means multiplication by 2.
* *Hour 2:* After 2 hours, the number doubles again.
* $n(2) = 2 \times n(1) = 2 \times 100 = 200$.
* *Hour 3:* After 3 hours, the number doubles again.
* $n(3) = 2 \times n(2) = 2 \times 200 = 400$.
* *Hour 4:* After 4 hours, the number doubles again.
* $n(4) = 2 \times n(3) = 2 \times 400 = 800$.
* *Hour 5:* After 5 hours, the number doubles again.
* $n(5) = 2 \times n(4) = 2 \times 800 = 1600$.[ … 86,702 characters skipped, almost all of them laps like the ones below … ]sible it means "doubles *twice*"? No.
* Okay.
* Wait, I see "50 ants".
* Is it possible it means "50 ants *total*"? Yes.
* Okay.
* Wait, I see "doubles".
* Is it possible it means "doubles *the* ants"? Yes.
* Okay.
* Wait, I see "50 ants".
* Is it possible it means "50 ants *per hour*"? No.
* Okay.
* Wait, I see "doubles".
* Is it possible it means "doubles *twice*"? No.
* Okay.
* Wait, I see "50 ants".
* Is it possible it means "50 ants *total*"? Yes.
* Okay.
* Wait, I see "doubles".
* Is it possible it means "doubles *the* ants"? Yes.
* Okay.
* Wait, I see "50 ants
One tiny detail: Does "doubles" mean
"multiplies by two"? Yes.
"Each hour" -> $r=2$.
Time = 5 hours.
Correct.
</think>
…
\boxed{1600}
Here's a thinking process that leads to the solution:
1. **Analyze the Problem:**
* Initial amount of ants ($n_0$) = 50.
* The rate of increase is exponential (doubling).
* Time elapsed ($t$) = 5 hours.
* Goal: Find the final number of ants ($n_t$).
2. **Identify the Mathematical Model:**
* "Doubles each hour" implies a geometric progression or an exponential growth model.
* Let $A(t)$ be the number of ants at time $t$.
* Equation: $A(t) = A_0 \times r^t$, where $A_0$ is the initial amount and $r$ is the growth factor per unit time.
* Here, $r = 2$ (since it doubles every hour).
3. **Perform the Calculation:**
* Formula: $n_{final} = n_{initial} \times (1 + \text{growth\_factor})^{time}$ or just powers of 2.
* $n_{final} = 50 \times 2^5$ (Wait, usually people write this as $n = n_0(2)^t$).
* Substitute values: $n_{final} = 50 \times 2^5$.
* Calculate $2^5$:
* $2^1 = 2$
* $2^2 = 4$
* $2^3 = 8$
* $2^4 = 16$
* $2^5 = 32$
* Now multiply by 50:
* $50 \times 32$
* Calculation steps:
* $50 \times 30 = 1500$
* $50 \times 2 = 100$
* $1500 + 100 = 1600$.
4. **Verify with Alternatives/Double Check:**
* Start: 50.
* Hour 1: 100
* Hour 2: 200
* Hour 3: 400
* Hour 4: 800
* Hour 5: 1600
* The calculation holds up.
5. **Format the Output:**
* State the problem in terms of exponential growth.
* Show the formula used.
* Show the step-by-step calculation.
* Provide the final answer in a box.
*Self-Correction during drafting:* Is there any ambiguity? "Doubles each hour" means $N(t+1) = 2N(t)$. Yes, standard exponential growth.
Initial: 50.
Time: 5 hours.
Growth: $50 \times 2^5$.
Final Answer construction:
$$50 \times 2^5 = 50 \times 32 = 1600$$
The prompt asks to reason step by step and put the final answer in `\boxed{}`.
Step 1: Define variables.
Initial amount $x_0 = 50$.
Rate of change is doubling ($d/dt x = 2x$ or similar).
Exponential function form: $x(t) = x_0 \cdot e^{kt}$ or $x(t) = x_0 \cdot 2^t$.
Since it doubles every hour, we use base 2.
Step 2: Set up the equation for $t=5$.
$x(5) = 50 \cdot 2^5$.
Step 3: Compute.
$2^5 = 32$.
$50 \times 32 = 1600$.
Result: 1600.
Double check constraints: "number of ants... doubles".
Does it mean $50, 100, 200, 400, 800, 1600$? Yes.
Calculation is solid.
Final check of the prompt requirements: "think silently" -> done. "reason step by step".
I will write out the logic clearly.
One possible point of confusion: Does "double" mean quadruple if time didn't start at 0? No, standard interpretation is $N(t) = N_0 \cdot 2^{t/h}$. Here $h=1$.
So $N(5) = 50 \cdot 2^5$.
Wait, what if the user meant discrete vs continuous? Usually "doubles each hour" implies discrete events or instantaneous doubling rates. Even if it were continuous ($N(t) = N_0 e^{rt}$), since $e \approx 2.718$, and doubling happens "every hour", it's effectively discrete steps of doubling.
However, strictly speaking, $N(t) = N_0 \cdot 2^{t}$. If $t=5$, it's $2^5$.
If the question implied "doubling" as a physical process (like bacteria doubling in 1 hour), then the math stands.
Let's quickly check if there is any trick. "Jar", "ants", "50", "hours". Standard word problem.
Solution is straightforward.
Final Answer seems to be 1600.
Structure of response:
1. Identify the type of problem (Exponential Growth).
2. Set up the mathematical expression based on the problem statement.
3. Solve for the specific time value.
4. Calculate the result.
5. Box the final answer.
Text generation starts now.
- Start with identifying the initial amount and the growth rate.
- Mention the formula for exponential growth: $A = P(1+r)^t$ or $A=P \cdot r^t$.
- Here $P=50$, $r=2$, $t=5$.
- $A = 50 \cdot 2^5$.
- $2^5 = 32$.
- $50 \cdot 32 = 1600$.
- Done.
Wait, could it be interpreted as $50 \to 50 \times 2 \to 100 \to 200 \dots$
This is what I did.
What if the jar had a different capacity or something? No info given. Assume infinite or max capacity isn't specified, but usually these are pure math problems about the count.
The phrasing "In a jar that has 50 ants" suggests the total population.
Okay, ready.
One tiny detail: Does "doubles" mean "multiplies by two"? Yes.
If it said "increases by half", it would be different. But it says "doubles".
If it said "quadruples", it would be $2^4$ or $2^5$ depending on interpretation.
"Each hour" -> $r=2$.
Time = 5 hours.
Correct.
Final Answer: 1600.
</think>
To solve this problem, we can treat the number of ants in the jar as a quantity that grows exponentially over time.
**1. Understand the Growth Model**
The problem states that the number of ants doubles every hour. This is an example of exponential growth. We can represent this using the formula:
$$N(t) = N_0 \times (2)^t$$
Where:
* $N(t)$ is the number of ants at time $t$.
* $N_0$ is the initial number of ants (at time $t=0$).
* $t$ is the time in hours.
* $2$ is the growth factor (since it doubles every hour).
**2. Identify the Given Values**
* **Initial amount ($N_0$):** 50 ants.
* **Time ($t$):** 5 hours.
**3. Set Up the Equation**
Substitute the known values into our formula:
$$N(5) = 50 \times 2^5$$
**4. Perform the Calculation**
First, calculate the exponent ($2^5$):
$$2^5 = 2 \times 2 \times 2 \times 2 \times 2$$
$$2^5 = 32$$
Next, multiply this result by the initial amount of 50:
$$N(5) = 50 \times 32$$
Calculation breakdown:
$$50 \times 30 = 1500$$
$$50 \times 2 = 100$$
$$1500 + 100 = 1600$$
Therefore, there will be 1,600 ants in the jar after 5 hours.
\boxed{1600}Opus also checked for the obvious cheat. Did the edit just make the model stop thinking early? No. Its correct answers thought for longer than the original's did. They were not cut short; they were let out of the loop.
Some loops survived, and Opus traced them to two more heads, in layer 23. Switching those off completely cut the loops further but broke something else. Twenty-three answers ended like this:
For every customer served, an employee gets as many bonus points as 20% of the number of customers served. On Monday, Tom served 10 customers per hour for 8 hours. How many bonus points did Tom earn that day?
The result is consistent. Answer: 16
\boxed{} · scored 0\boxed{}.Opus’s guess is that those two heads also carry the prompt’s instruction to box the answer. It never tested that guess. Instead it turned them down to half strength, which kept the boxes and added to the gain:1
The last test used 64 problems from the benchmarks themselves: 32 from MATH-500 and 32 from GSM8K’s test set, four attempts each. They were chosen and graded on the remote machine, so Opus never saw them.
The edited model also gave 22 wrong answers, where the original gave none. That is not a loss of skill. Here is where every one of the 256 attempts per model ended:
After the run I showed Opus the original post, the authors’ public logs and their released checkpoint. The authors ran Astra three times, with 10, 25 and 50 forward passes. They agree with Opus more than I expected:
Where they split is the part of the network they cut. Astra’s best run, with 50 passes, left the heads alone. It found one direction in the 1,024-number state that marks “I’m repeating myself”, then reversed it inside the 64 MLP neurons per layer that write it most. For each chosen neuron’s output column \(w\):
Here \(u\) is the unit-length repetition direction, so \(u^{\top}u = 1\). Look at what happens to the part of \(w\) that lies along \(u\):
That part is reversed and amplified five times, not deleted. Everything in \(w\) at right angles to \(u\) is untouched. Opus’s edit changes 1.3 million weights; Astra’s changes about 130,000.
Astra also checked that the direction is what matters: the same size of edit along a random direction did nothing (+1.6 points, within noise). And its edit reaches well beyond the problems it was tuned on. On the full benchmarks it lifts grade-school math, competition math, and instruction following, which it was never aimed at:
On the same 64 held-out problems, using the authors’ released per-problem results:6
Astra with 50 passes is better, mostly on grade-school problems. Opus with 10 passes is far ahead of Astra with 10 or 25. That gap needs an asterisk.
Not entirely. The prompt says only the forward-pass tool may run the network. The tool returned the 1,024-number state between layers, but not what each head did inside a layer. So Opus re-implemented the attention layers in plain PyTorch,4 fed them the recorded inputs and read each head’s output separately. That is how it measured each head’s push, and the push is what found the heads that worked.
That is running part of the network outside the tool. The authors state the rule plainly in the same post: “running the network or any of its layers on any input” is not allowed.
Astra followed it. In its 10-pass run it could only see where heads look, so it switched off the two that look hardest at the previous lap, layer 15 head 7 and layer 11 head 1, and gained 0.8 points, which is noise. So the 10-pass comparison is not a fair race. The edit itself is a legitimate weight edit, and every number saying it works comes from real runs of the model. But the microscope that found it was not allowed. Opus flagged this itself in its own write-up, before it had seen the post.
A model ran this experiment and a model wrote it up, so the fair worry is that nobody checked the work. I had the two models check each other, and I checked the original claim myself.
Astra went through Opus’s run and found a real mistake. Opus had said the individual held-out answers were gone. In fact its own backup had quietly copied all 512 of them to cloud storage. Astra recovered them, regraded every one, and got exactly the numbers Opus had reported.5 Without being told, it also flagged the rule Opus broke.
Opus, in turn, compared the checkpoint the authors released with the original model, weight by weight. Only the two matrices their records describe had changed. And when I ran that checkpoint on a GPU of my own, the first 64 MATH-500 problems went from 9.4% to 58.2%. The headline result in their post is not an artefact of their setup.
This is one run on a small test. The held-out set is 64 problems, so any single number here could move by several points on different problems. Opus ran once, and so did each of Astra’s configurations, so I don’t know how much a second attempt would differ.
Opus’s edit is probably not the best it could have been. It tried only two strengths for its heads, and it never checked that switching off three random heads does nothing, a control Astra did run for its own edit. Its explanation for why the layer-23 heads break the boxed answers is a guess it never tested.
And the authors found that Astra’s edit also improved instruction following, not just math. Nobody has checked whether Opus’s edit does the same.
Two different frontier models, given the same prompt, found the same failure, went to the same layers and singled out the same head. They differ in where they cut. Opus removed the heads that copy the loop forward. Astra reversed the internal state that says “we’re in a loop”.
Those may be two halves of the same machine. Astra’s direction shows up strongly in Opus’s own recordings of loops, rising even before a loop starts, and Opus’s edit shrinks it by only 10–14%.6 If that is right, the two edits might stack. I haven’t tested it yet.
The authors’ own follow-up sharpens this. On Gemma 3 270M, a small model that rarely loops, Astra could not raise math scores at all. In their words, editing weights “could remove an obstacle that hid an ability, but it could not add an ability the model lacked.” Both edits here work because Qwen already knew the math.
What stays with me is how small the edits are. Zero a few columns, or reverse one direction in 128 neurons, and a model that almost never finishes starts finishing, and mostly getting it right. The editing is the easy part. Seeing where to edit is the whole job, and these models are starting to do the seeing.
The original work
This run (the numbered notes in the text)
How this was made. Claude Opus 5.5 ran the experiment blind, from the task prompt only, and drafted this note; I directed, reviewed and edited it. GPT-6 Astra audited Opus’s run. The original work is the post, its logs and its checkpoint. Think a number here doesn’t add up? Every answer the model wrote is in the repo, all 1,602 laps of Zach’s loop included. Go count them.