Optimizing Muse Spark Open on TPU v7 with SGLang-TorchTPU

Engineers from SGLang-TorchTPU served Meta's 816B-parameter Muse Spark Open mixture-of-experts model on a single TPU v7 host using a 62-layer megakernel and specialized pipelining techniques. The optimizations reduced token generation time, increased KV cache capacity via sequence-on-lane layouts and session-aware caching, and improved prefill performance using fused kernels and pair exchanges. Open article →