Study Finds MLP Residual Networks Implement Renormalization Group-Like Coarse-Graining
A new preprint on arXiv reports the first quantitative, position-level evidence that MLP residual networks perform a selective coarse-graining procedure analogous to the renormalization group (RG) framework from statistical physics. Researchers trained a pure MLP residual stack on synthetic Markov chain sequences and measured how the effective rank of internal representations changes with network depth. The findings matter because they move a long-standing qualitative analogy between deep learning and RG flows onto empirically testable, measurable ground.
Researchers have submitted a preprint to arXiv arguing that MLP residual networks implement a process structurally analogous to the renormalization group, a mathematical framework used in physics to systematically eliminate irrelevant degrees of freedom across scales. Using a controlled experimental setup — a pure MLP residual stack trained on masked token prediction over synthetic Markov chain sequences with known spectral properties — the team identified three key findings. First, the effective rank of the residual stream decreases monotonically with depth after training, consistent with progressive elimination of irrelevant information. Second, this rank collapse is selective: it occurs for input sequences with short correlation lengths (approximately 1) but is absent for sequences with long correlation lengths (approximately 7), suggesting the network preserves exactly the degrees of freedom relevant to the prediction task. Third, inter-layer kernel drift is concentrated at one or two specific depth transitions, with the rest of the network near a fixed point, consistent with a discrete fixed-point plateau predicted by RG theory. The authors claim these results constitute the first quantitative, position-level empirical verification of the RG analogy in neural networks, going beyond prior qualitative descriptions. The work is 16 pages with 9 figures and has been submitted to both the Machine Learning and Statistical Mechanics communities.
What's missing
The study uses only synthetic Markov chain sequences with known spectral properties; it is unclear whether the reported RG-like behavior generalizes to real-world datasets or more complex architectures such as transformers. The paper is an unreviewed preprint and has not yet undergone peer review. The authors do not address whether the observed rank collapse could be explained by simpler mechanisms unrelated to RG theory, nor do they provide ablations varying architecture depth or width systematically. The causal relationship between input spectral structure and rank collapse is correlational in the current design.
What different sources said
- arXiv cs.LGCenter
Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.