Moonshot Kimi K3.1 Pro served via llm-d on TPUs and GPUs

Infrastructure teams have released Day-0 distributed serving recipes and a Kubernetes-native inference control plane using llm-d to deploy Moonshot AI's Kimi K3.1 Pro across Google Cloud TPUs and NVIDIA GPUs. The configuration utilizes vLLM and SGLang base engines to handle state-aware KV routing, hierarchical memory offloading, and prefill-decode disgeneration across multi-trillion-parameter Mixture-of-Experts architectures. Open article →