AMD releases online quantization for vLLM ATOM SGLang

AMD has introduced online quantization, a feature that quantizes model weights layer by layer during the model-loading stage inside serving engines rather than ahead of time through a separate export job. This approach eliminates the need for intermediate disk checkpoints and enables fast exploration of quantization schemes, dynamic requantization, and reinforcement learning rollouts. Integrations are available in inference engines including vLLM, ATOM, and SGLang, with benchmarks showing that online quantization matches the accuracy and latency of traditional offline methods. Open article →