vLLM serves GLM-5.3 across two NVIDIA DGX Stations

This tutorial explains how to serve the 753-billion-parameter NVIDIA GLM-5.3 NVFP4 model across two NVIDIA DGX Stations using vLLM and tensor parallelism. The setup connects the machines via RoCE RDMA, splits the model weights across both GPUs, and utilizes speculative decoding and an FP8 KV cache to optimize performance. Benchmarking with AIPerf shows an overall throughput peak of 607.8 tokens per second at a concurrency of 16. Open article →