MiniMax Sparse Attention: New Method Enables Efficient Processing of Million-Token Contexts in Large Language Models
Researchers from MiniMax have introduced MiniMax Sparse Attention (MSA), a blockwise sparse attention mechanism that reduces per-token attention compute by 28.4x at one-million-token context lengths while matching the performance of standard attention. MSA builds on Grouped Query Attention by using a lightweight Index Branch to select only the most relevant key-value blocks for each attention group, avoiding the quadratic cost scaling that makes long-context inference prohibitively expensive. The work is significant because ultra-long context is increasingly essential for agentic AI, code reasoning, and persistent memory applications, and MSA offers a practical path to deploying such capabilities at scale.
MiniMax Sparse Attention (MSA) is a new blockwise sparse attention architecture designed to make million-token context windows computationally feasible for large language models. The method introduces an Index Branch that scores key-value blocks and selects a top-k subset independently for each Grouped Query Attention (GQA) group, allowing group-specific sparse retrieval while preserving efficient block-level GPU execution. A co-designed GPU kernel employs exp-free top-k selection and KV-outer sparse attention to maximize tensor-core utilization under block-granular memory access patterns. On a 109-billion-parameter natively multimodal model, MSA matches standard GQA quality while delivering 14.2x prefill and 7.6x decoding wall-clock speedups on NVIDIA H800 GPUs at one-million-token context. The authors emphasize simplicity and scalability as core design principles, making MSA straightforward to deploy across a broad range of GPU hardware. A production-grade multimodal model powered by MSA has been publicly released, and the inference kernel is available as open source. The paper spans 30 pages with 14 figures and was submitted to arXiv on June 11, 2026.
What's missing
The paper does not report independent third-party benchmarking of the quality or speed claims; all evaluations appear to be conducted by the authors themselves. It is also unclear how MSA performs relative to other recent sparse or linear attention alternatives (e.g., FlashAttention variants, linear attention methods) beyond the GQA baseline, and the generalizability of the speedup figures to GPU architectures other than the H800 is not established.
What different sources said
- arXiv cs.AICenter
MiniMax Sparse Attention
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.