Serving GLM-5.3-Flash on Trainium2 with NKI Layer Megakernels

Researchers served the 320-billion-parameter GLM-5.3-Flash model on a single AWS Trainium2 instance by rewriting the model's decode layers into NKI megakernels and building a prefix cache. Tested on SemiAnalysis's AgentX benchmark, the system achieves up to 28.9 agent requests per minute at a cost of $0.021 per request while significantly reducing time-to-first-token compared to baseline engines. Open article →