NVIDIA releases open-source Snapshot checkpoint restore

NVIDIA Snapshot is a new open-source checkpoint and restore system for GPU workloads on Kubernetes that makes initialized workloads reusable across inference replicas. Designed to bypass the lengthy initialization process known as a cold start, Snapshot captures CPU and GPU memory as portable artifacts to bring inference capacity online in seconds instead of minutes. Benchmarks across vLLM, SGLang, and NVIDIA TensorRT LLM demonstrate that Snapshot accelerates replica startup by 8x to 36x across various model sizes and parallelism configurations. Open article →