Study Identifies Critical Branching Dynamics in Small LSTM Neural Networks
Researchers analyzing trained LSTM networks found that small models near optimal training exhibit scale-free avalanche statistics and near-critical branching parameters, while larger models remain subcritical. The study proposes that criticality, a key principle in biological neural systems, emerges as a capacity-dependent dynamical regime in artificial networks. The findings suggest a novel connection between network size, training dynamics, and the emergence of critical-like behavior in deep learning systems.
A new arXiv preprint examines hidden-state dynamics in long short-term memory (LSTM) networks to investigate whether artificial neural networks exhibit criticality—an organizing principle observed in biological brains. The researchers found that smaller LSTM networks at optimal training epochs display scale-free avalanche statistics and branching parameters near unity, indicating near-critical dynamics, whereas larger models exhibit subcritical behavior. To reconcile the coexistence of subcritical branching with robust 1/f^β noise patterns, the authors introduce a mixture branching process framework that links heterogeneous branching dynamics to long-range temporal correlations. The work identifies critical-like behavior as an emergent property dependent on network capacity, suggesting that the relationship between network architecture and dynamical regimes may be more nuanced than previously understood.
Limitations & open questions
The preprint does not discuss potential implications for improving neural network training, generalization performance, or practical applications. Additionally, the study's limitations regarding the specific architectures tested, dataset characteristics, and whether findings generalize to other recurrent or transformer-based architectures are not detailed in the abstract.
What different sources said
- arXiv cs.AICenter
Towards Critical Branching Mechanism in Recurrent Neural Networks
Related
Researchers Identify D-Retro-Inverso Peptide Candidates for Treating Cardiac Amyloidosis
Researchers have designed and computationally evaluated four peptide candidates made from D-amino acids that may inhibit the formation of Serum Amyloid A (SAA) fibrils associated with cardiac complications following heart attacks. The study builds on mouse model evidence suggesting SAA aggregates contribute to long-term myocardial infarction complications and may operate similarly in humans. Two candidates, DRI-R5S and DRI-H6A, were identified as particularly promising for potential drug development.
Study Identifies D-Retro-Inverso Peptides as Potential Treatments for Cardiac Amyloidosis
Researchers designed four peptide candidates using D-amino acids to inhibit Serum Amyloid A (SAA) fibril formation, which may contribute to complications following heart attacks. The study, conducted using molecular dynamics simulations in a mouse model, identified two peptides—DRI-R5S and DRI-H6A—as promising drug candidates. This work could lead to new therapeutic approaches for cardiac amyloidosis, a serious post-infarction complication.
Scientists Discover Previously Unknown Branch of Tryptophan Metabolism in Humans
Researchers identified a new enzymatic step in human tryptophan catabolism, showing that the protein ASPDH acts as a 2-aminomuconate reductase to produce a previously unknown amino acid. This discovery fills a gap in the kynurenine pathway, one of the body's major metabolic routes for processing the amino acid tryptophan. The finding expands understanding of human metabolism and may have implications for understanding NAD cofactor production and related metabolic diseases.