How MiniMax M3 Was Built: Sparse Attention and Native Multimodality — Olive Song

Olive from MiniMax walks through MiniMax M3, an open-weight model built for agentic coding, 1M token context, and native multimodality from the first training step. The talk centers on MSA, a GQA-aware block-level sparse attention design co-developed with GPU infrastructure in mind, and on why training text and vision together from scratch beats adding vision later.