Tag

#paging

1 article tagged paging.

Notes on KV cache paging at scale

Inference & Serving

PagedAttention is a good idea poorly understood. A primer, plus the second-order effects you only see at 10,000 concurrent requests.

Priya Raghavan Jul 11, 2026