SHAPE: New Method for Pruning Mixture-of-Experts Language Models Using Coalition-Aware Expert Scoring
Researchers have proposed TENP, a structured pruning framework that reduces the parameter footprint of Mixture-of-Experts large language models by selectively pruning neurons in less important experts in a trapezoidal pattern across layers. Existing compression methods either remove whole experts—disrupting routing—or apply unstructured weight pruning with limited efficiency gains. TENP achieves roughly 36% parameter reduction on DeepSeek models with only a 1-point accuracy drop, and even surpasses the full model by 10% on code generation tasks.
TENP (Trapezoidal Expert Neuron Pruning) is a structured compression framework targeting Mixture-of-Experts (MoE) LLMs, which scale efficiently via sparse activation but carry a large static memory footprint that hinders deployment. The method uses a small number of calibration samples to identify important experts, which are kept intact, while applying Expert Neuron Pruning (ENP) to less important experts—removing neurons based on their projected contribution to expert output. Crucially, the pruning follows a trapezoidal pattern, retaining more parameters in shallow layers and progressively pruning deeper ones. Expert importance is evaluated jointly by the magnitude of expert output and its ability to rotate the input vector's direction, a dual criterion intended to capture both scale and representational influence. Experiments on Qwen and DeepSeek model families show that at 40% routing expert sparsity and 63.76% average activated expert parameters, DeepSeek loses only 1 accuracy point versus the dense baseline while outperforming it by 10% on code generation benchmarks. The work was submitted to arXiv on June 3, 2026, and has not yet undergone formal peer review.
What's missing
The paper has not yet been peer-reviewed. Key open questions include: whether the trapezoidal pruning pattern generalizes beyond Qwen and DeepSeek architectures; what calibration dataset size and composition are needed for reliable importance estimation; whether the 10% code generation improvement is robust across multiple benchmarks or specific to a narrow evaluation set; and what the wall-clock inference speedup and memory savings are in practice on real hardware.
What different sources said
- arXiv cs.LGCenter
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.