Researchers introduce Fused-Softmax kernels in AMD AITER

Researchers introduced Fused-Softmax Attention, a training-free algorithm that reformulates and approximates softmax to move dominant operations into attention GEMMs and utilize low-precision matrix throughput on modern GPUs. Implemented for the AMD CDNA 4 architecture, the MXFP8, MXFP6, and MXFP4 kernels achieved up to 4.6 PFLOP/s on an AMD Instinct MI355X GPU with speedups ranging from 2.1x to 3.2x over BF16 attention. The kernels have been shipped in AMD's open-source AITER library. Open article →