Tiered KV Cache Offloading in vLLM
Tiered KV Cache Offloading in vLLM
摘要
A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers — reducing recomputation and increasing serving capacity.
A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers — reducing recomputation and increasing serving capacity.
译自 vLLM · 官方博客 · 录于 二〇二六年九月十一日