New Method Reveals Interpretable Dimensions in Neural and Behavioral Representations
Two new studies present computational approaches for extracting interpretable features from complex representational data: one introduces Similarity-Based Representation Factorization (SRF) for recovering low-dimensional embeddings from similarity matrices, while the other demonstrates that transformer neural networks' standard basis dimensions already encode semantic information through sign patterns without requiring training. These methods advance mechanistic interpretability across neuroscience, psychology, and artificial intelligence by making high-dimensional representations more understandable and analyzable. The findings have implications for understanding how biological and artificial systems organize information.
Two complementary studies address the challenge of interpreting high-dimensional representations in neural and computational systems. The first introduces Similarity-Based Representation Factorization (SRF), a general method that recovers low-dimensional, non-negative, interpretable embeddings from similarity matrices across neural, behavioral, and computational datasets. SRF demonstrates superior performance compared to direct similarity matrix comparison for hypothesis testing and exploratory analysis, even with sparse or incomplete data. The second study, 'Bag of Dims,' reveals that transformer language models' standard basis dimensions function as semantic feature detectors through their sign patterns and magnitudes, enabling feature discovery with zero training across multiple model families (Qwen, Gemma, Mistral). Remarkably, sign patterns alone preserve 72-93% of language modeling accuracy, and unsupervised discovery scales to 1,500 features with 99% sparsity. Together, these findings establish that interpretable structure exists in representations across biological and artificial systems, accessible through appropriate analytical frameworks.
What's missing
Both studies are preprints on arXiv and have not undergone peer review at a traditional journal venue. The SRF study's performance on real neural data compared to established neuroscience methods (e.g., principal component analysis, independent component analysis) is not quantitatively detailed. The Bag of Dims study focuses on relatively small language models (3.5-7B parameters); generalization to larger models remains unexplored.
What different sources said
- arXiv q-bioCenter
Similarity-based matrix factorization for revealing interpretable dimensions in representational data
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.