重温 vLLM 中的 PagedAttention
vLLM introduced PagedAttention, which borrows the paging abstraction that operating systems use for RAM and applies it to the GPU’s KV cache. During LLM inference, the KV cache – the stored key and value tensors for all previous tokens – is the dominant…
2026/7/20 16:50:15