NVIDIA Nemotron 3.5 Super and Lightning Hybrid Mamba-Transformer Models

arxiv.org1 min read

NVIDIA researchers introduced Nemotron 3.5 Super and Nemotron 3.5 Lightning, open Mixture-of-Experts Hybrid Mamba-Transformer models built for reasoning, long-context understanding, and agentic workloads. The 120B Super model incorporates multimodal capabilities, Indic continued pretraining, and agentic software-engineering mid-training, while the 30B Lightning model focuses on efficient text tasks. According to the researchers, these models combine large-scale distillation, long-context training, and native speculative decoding to optimize both performance and deployment efficiency. Open article →

More on this topic