Towards First Principles

Technical explainers built the long way round: start at the smallest case you can hold in your head, count it upward with a pencil, and only then go and check whether the machine agrees. Every number here is measured, and every figure is one you can poke at.

Technical Articles

We trained a small model to query a graph with RL. It got better, and it stopped listening.

Reinforcement learning on Qwen3.5-9B as a Cypher agent over a 650,000-node graph, rewarded only by executing its own queries. It gains on question shapes it never saw, gains on human questions only in answer form, and — measured by removing the hints — learns to get about half as much from a hint as before. Three pre-registered predictions; two failed.

11 Oct 2026research note6 figuresTinker

RL post-training a graph-query agent on Fireworks: a 6-point gain, and what it is made of

The same recipe on Qwen3.8-27B through Fireworks' serverless training API: a 6.2-point gain on human-written questions against a same-route untrained control, all of it on questions the model could already sometimes answer.

11 Oct 2026research note6 figuresFireworks

A Forking-Tokens Experiment with Qwen3.5-9B

Single tokens flipped GPT-3.5’s answers in 2024. Do they still flip a 2026 model’s? On a question Qwen is confident about, nothing moves. On a question it is torn about, six alternate tokens move the final answer — and being torn at the very first token turns out to predict it.

22 Sep 2026research note2 figuresvLLM + one rented A40

FlashAttention, From First Principles

Why attention runs out of memory before it runs out of maths. Online softmax derived by hand from a 3×2 matrix, then the tiled kernel that never writes the N×N matrix down — ending in a measured benchmark, including the receipt I did not get.

17 Aug 20267 interactive figuresnumpy + Triton

Towards Free Normalization, From First Principles

A gentle on-ramp to the PyTorch “free normalization” work — derived from counting a 2×2 matrix upward, arriving at the roofline and the memory hierarchy that every article since has borrowed its vocabulary from.

28 Jul 20266 interactive figures