Researchers introduce CISA to speed up Nemotron 3 Nano attention

Researchers introduce Chunk-Indexed Sparse Attention (CISA), a technique that retrofits pretrained dense attention models without gradient updates by selecting fixed-size key-value chunks using compact summaries. Evaluated on NVIDIA Nemotron and Inkling models, CISA accelerates core attention up to 13x during decode while maintaining aggregate scores within small margins of dense attention across several benchmarks. Serving million-token prompts with CISA achieves up to 2.29x the end-to-end throughput of dense attention. Open article →