Sparse attention

Is a Faster Model Actually Faster? Test the Whole Job — Morgan Suo, MiniMax

Is a Faster Model Actually Faster? Test the Whole Job — Morgan Suo, MiniMax

Morgan Suo of MiniMax tests the changes inside M3.1 Flash on a realistic analyst research task about circuit-board suppliers and AI-driven cash flow. She examines sparse attention, 4-bit KV caches, low-precision MoE experts, speculative decoding, serving concurrency, and reasoning budgets, showing how each saves compute while potentially costing accuracy or latency. She closes with a framework for judging a model switch by verifiable output, the full cost of a usable report, and repeatable results.

How MiniMax M3 Was Built: Sparse Attention and Native Multimodality — Olive Song

How MiniMax M3 Was Built: Sparse Attention and Native Multimodality — Olive Song

Olive from MiniMax walks through MiniMax M3, an open-weight model built for agentic coding, 1M token context, and native multimodality from the first training step. The talk centers on MSA, a GQA-aware block-level sparse attention design co-developed with GPU infrastructure in mind, and on why training text and vision together from scratch beats adding vision later.