New Activation Function Improves Robustness and Interpretability in Machine Learning Models
Researchers have introduced the Similarity-Distance-Magnitude (SDM) activation function, a proposed replacement for the standard softmax used in neural network outputs. SDM adds awareness of how similar new inputs are to training data and how far they fall from the training distribution, alongside the existing confidence-score output. The work, accepted to ACL 2026 Findings, targets a known weakness in deployed language models: unreliable confidence estimates when inputs differ from training data.
The SDM activation function is presented as a more robust and interpretable alternative to softmax, the near-universal choice for final-layer classification in neural networks. Where softmax produces only a magnitude-based confidence score tied to decision boundaries, SDM incorporates two additional signals: similarity (how well a new input matches correctly predicted training examples) and distance (how far the input lies from the training distribution). Built on top of these activations, the SDM estimator uses data-driven partitioning of class-wise empirical cumulative distribution functions to control accuracy under selective classification — a setting where the model can abstain rather than guess. Experiments apply the method as a drop-in final layer over pre-trained language models, finding improved robustness to covariate shifts and out-of-distribution inputs compared to existing softmax-based calibration approaches, while preserving informativeness on in-distribution data. The paper also claims interpretability benefits through 'interpretability-by-exemplar,' allowing predictions to be explained by reference to similar training instances. The work spans 21 pages with 8 tables and has been accepted to the Findings of ACL 2026.
What's missing
Empirical results are reported only for language model tasks, leaving open whether SDM generalizes to vision or other modalities. Computational overhead relative to softmax is not addressed in the available abstract.
What different sources said
- arXiv cs.LGCenter
Similarity-Distance-Magnitude Activations
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.