Contrastive Learning Methods Struggle to Distinguish Slow Noise from True Dynamics, Study Shows
Researchers have identified a systematic failure mode in self-supervised predictive representation learning methods, such as JEPA, where slowly varying noise is encoded instead of the true dynamical signals. This occurs because contrastive objectives that sample negative examples across trajectories inadvertently reward encoding trajectory-specific noise rather than the underlying physics. The findings suggest a simple fix—sampling negatives within a single trajectory—and offer design principles for more robust representation learning in physical and experimental systems.
A preprint submitted to arXiv on June 5, 2026 identifies a fundamental flaw in a class of self-supervised learning methods that learn representations and predict dynamics in latent space, including the widely studied JEPA framework. When noise features remain approximately constant within a trajectory, contrastive predictive objectives that draw negative samples across different trajectories preferentially encode that slow noise rather than the true latent variables governing the system. As a result, learned representations become dominated by trajectory-specific noise artifacts, causing downstream task performance to degrade with noise strength and fail to improve with more or longer training trajectories. The authors demonstrate this failure mode in two settings: a synthetic moving-dot dataset using a SimCLR-style JEPA, and a rigid-body pendulum movie dataset using DySIB, a method designed for physically interpretable dynamics. Their proposed remedy—sampling negatives within a single trajectory—removes the shortcut by ensuring slow noise cannot distinguish frames within that trajectory, forcing the encoder to capture dynamically relevant variables instead. The study argues this failure is a general property of the contrastive predictive objective class rather than any specific implementation, and that longer within-trajectory training yields progressively better representations even under strong noise conditions.
What's missing
As a preprint, this work has not yet undergone peer review. The experiments are limited to two relatively controlled synthetic or semi-synthetic settings (a moving-dot dataset and a pendulum movie), and it remains an open question how well the proposed within-trajectory negative sampling remedy scales to more complex real-world dynamical systems or high-dimensional experimental data. The authors do not provide theoretical guarantees for the proposed fix, nor do they benchmark against a broad range of alternative contrastive or non-contrastive representation learning methods.
What different sources said
- arXiv cs.LGCenter
Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.