Tag

#KV-cache

1 article tagged KV-cache.

Notes on KV cache paging at scale

Inference & Serving

PagedAttention is a good idea poorly understood. A primer, plus the second-order effects you only see at 10,000 concurrent requests.

Priya Raghavan Jul 11, 2026