331 output tokens/s per GPU on AgentX: Serving Kimi K3 on GB300 with TokenSpeed | LightSeek Foundation

TokenSpeed serves the Kimi K3 MoE model at 331 output tokens per second per GPU on the AgentX workload using 32 NVIDIA GB300 GPUs. The architecture combines Distributed Weight Data Parallelism for prefill, Decode Context Parallelism, expert parallelism, a shared host-memory L2 cache pool, and independent request admission to maximize throughput and cache reuse. Open article →