MiniMax open-sources attention library for Blackwell, M3 weights due Friday

MiniMax developer relations lead Ryan Lee announced that the high-performance attention library MiniMax Sparse Attention (MSA), designed for NVIDIA Blackwell (SM100) GPUs, has been officially open-sourced under the MIT license. Ryan Lee also indicated that the MiniMax-M3 model weights are expected to be released this Friday.

MSA has been applied to MiniMax-M3's million-token context inference, selecting the most relevant KV blocks within each GQA group and performing attention computation only on those selected blocks. According to the paper, at 1 million token context, compared to a dense GQA baseline, MSA reduces attention computation by 28.4x and achieves a 14.2x prefill speedup and a 7.6x decoding speedup on H800 GPUs.

The open-source version integrates both C++ JIT and CuTe-DSL implementations in the same Python package, and provides dense FlashAttention and sparse Top-k attention kernels supporting BF16, FP8, NVFP4, and FP4 precisions. It is currently targeted for deployment on NVIDIA Blackwell (SM100) GPUs.