All articles.
The arithmetic of attention: why FlashAttention still matters
Inference & ServingMemory bandwidth, not FLOPs, is what bounds modern inference. A walk through the numbers behind a kernel that quietly reshaped the field.
Continuous batching, revisited
Inference & ServingThree years after the original paper, what does state-of-the-art serving actually look like? A field report from a team running 12B tokens a day.
What we've been getting wrong about MoE routing
ArchitectureTop-k routing has become a default. It shouldn't be. A look at the tradeoffs nobody's measuring and the experiments that change my mind.
Quantization-aware training, end-to-end
QuantizationFP4 is here, and the gap between PTQ and QAT has widened. What's actually working in production today, and why the recipe is messier than it looks.
A research-grade trainer in 400 lines
Training SystemsMost training frameworks are 50,000 lines of code in a trench coat. This is what falls out when you start from FSDP and a will to delete.
Embeddings as compression: the bitter lesson, retold
Retrieval & RAGWe've stopped treating embeddings like first-class data. A case for revisiting them, with measurements from a 200M-document corpus.
World models and the cost of imagination
ModelsLatent rollouts are cheap. World rollouts are not. What we learned trying to scale a JEPA-style world model on robotics data.
The hidden latency in agent loops
AgentsEvery tool call is a round trip. Every round trip is a context append. Why naive agent loops compound latency faster than you expect, and what to do about it.
RAG is not retrieval-augmented generation
Retrieval & RAGIt is a feedback loop, an index policy, and a re-ranker pretending to be a system. Why most RAG postmortems mistake the symptom for the disease.
Speculative decoding without the speculation
Inference & ServingDraft models work. They also fail in ways the original papers didn't surface. A small bag of tricks for keeping acceptance rates high in real workloads.
FSDP vs DeepSpeed, 2026 edition
Distributed TrainingThe choice used to be obvious. It isn't anymore. A side-by-side on training a 30B model across three clusters and four hardware generations.
The four evals that matter (and the dozen that don't)
EvaluationWe have too many benchmarks and too few signals. A framework for choosing evaluations that correlate with the thing you actually care about.
Neural Networks From Zero: From a Single Number to a Billion Parameters
ArchitectureA neural network never sees a word, an image, or a sound — only a list of numbers. Starting from that one fact and a single neuron, this day-zero guide builds the whole machine: how any input becomes numbers, why weights, biases, and activations each exist, how neurons stack into layers and layers into a model, and how to compute a model's size and running cost by hand.
Notes on KV cache paging at scale
Inference & ServingPagedAttention is a good idea poorly understood. A primer, plus the second-order effects you only see at 10,000 concurrent requests.