KVGuard: A KV Cache Safety Checker for LLM Serving Systems

Researchers introduce KVGuard, a systematic memory-safety abstraction designed for KV cache management in modern LLM serving systems like vLLM and SGLang. By maintaining compact shadow blocks for each physical KV block, KVGuard performs five safety checks across the PagedAttention workflow to detect bugs such as use-after-free and out-of-bounds access. The system covers historical bugs with minimal memory and performance overheads and no false positives. Open article →