Kalman Linear Attention: A Parallel Bayesian Filtering Approach for Language Models
Researchers have developed Kalman Linear Attention (KLA), a new sequence-mixing layer for language models that reformulates sequence processing as exact Bayesian filtering using the Kalman filter. By reparameterizing the classical Kalman filter in information form, the authors show its updates become an associative scan, enabling temporal parallelism despite a non-linear per-token update. The work addresses a key expressivity limitation in existing linear state-space models and demonstrates stronger state-tracking capabilities than current alternatives at comparable computational cost.
Kalman Linear Attention (KLA) is a proposed drop-in sequence mixer for language models that grounds sequence processing in probabilistic inference, specifically exact Bayesian filtering via the Kalman filter. Existing linear-complexity alternatives to transformers—such as Mamba and gated linear attention (GLA)—rely on linear state updates that limit their expressivity and state-tracking robustness. The key technical contribution is showing that recasting the Kalman filter in information form transforms its updates into an associative scan, making the model both temporally parallel and strictly more expressive than GLA-style updates. This expressivity gain is demonstrated concretely: KLA solves permutation-composition tasks on the A5 group, which linear SSMs and standard attention mechanisms cannot. The model also carries an explicit belief-state uncertainty estimate, a feature absent from most competing architectures. Empirically, KLA matches or outperforms modern SSMs and GLAs on synthetic token-manipulation tasks and zero-shot commonsense benchmarks, and is among the first stacked Bayesian-filtering architectures trained at the billion-token scale. The paper has been accepted at ICML 2026, with an earlier version presented at the EIML workshop at EurIPS 2025.
What's missing
The paper does not report wall-clock training or inference speed benchmarks comparing KLA to transformer baselines at scale, making it difficult to assess practical deployment costs beyond asymptotic complexity. Additionally, evaluations are limited to synthetic tasks and zero-shot commonsense benchmarks; performance on long-context generation, retrieval-augmented tasks, or fine-tuning scenarios remains an open question.
What different sources said
- arXiv cs.LGCenter
Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.