KV cache sizing for DeepSeek V3 on one GPU

pub.towardsai.net5 min read

The author explains how to calculate large language model KV cache sizes from configuration files to determine how many concurrent users fit on a single GPU. The article details a specific formula, compares cache memory costs across models like Llama and DeepSeek V3, and examines architectural tricks such as grouped-query attention, latent attention, and sliding windows. Open article →