End‑to‑End 4‑Bit Training - Thinking Machines Lab

Thinking Machines Lab describes its end-to-end 4-bit training recipe for large language models, utilizing NVFP4 precision for forward passes and stochastic rounding for backward passes on NVIDIA Grace Blackwell Ultra systems. The authors report that this approach yields notable speedups in pretraining and reinforcement learning while maintaining model accuracy. Open article →