PagedAttention is a way of storing a language model's memory of past tokens in small fixed-size blocks, so a server wastes less memory and can handle more requests at once.
In plain words
While generating, a model keeps a KV cache of every earlier token for each request. It is large and grows unpredictably, so reserving one big chunk per request wastes space. PagedAttention borrows the paging idea from operating systems: the cache is split into blocks that can sit anywhere in memory, and requests can share blocks. Less waste means bigger batches and higher throughput. It was introduced with the vLLM serving engine.
A quick example
Imagine a server answering many chats that happen to share the same long system prompt. With PagedAttention, the cache blocks for that shared prompt can be stored once and shared, leaving room for more conversations.
Where you'll meet it
- The vLLM serving engine.
- High-throughput LLM serving.
Related
Sources
- PaperEfficient Memory Management for Large Language Model Serving with PagedAttention, Kwon et al. (arXiv) · read 29 Sept 2026
- DocsvLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention, vLLM blog · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.