Researchers enable MXFP4 models on Apple M5 INT8 GEMM

Researchers demonstrated that MXFP4 language models can run on hardware with INT8 matrix datapaths, such as Apple's M5 GPUs, by using integer-exact microscaling. The method maps a row of MXFP4 weights to a single INT8 vector and a power-of-two scale when block exponents span at most three. This technique allows a linear layer to run as a single INT8 GEMM with high accuracy, competitive performance, and bit-identical outputs on tested models. Open article →