GEASS: New Method Reduces Hallucinations in Vision-Language Models by Selectively Trusting Generated Captions
Researchers have developed GEASS, a training-free, logit-level module that dynamically controls how much a Vision-Language Model (VLM) relies on its own generated captions when answering visual questions. The work identifies that naively appending captions to VLM inputs can actually hurt performance — dropping accuracy on HallusionBench by nearly ten points — because captions compete with image features for the model's attention. GEASS addresses this by gating caption trust on a per-query basis, improving results across four VLMs and two benchmarks without adding any model parameters.
Vision-Language Models are known to hallucinate objects not present in images, and a common mitigation strategy involves feeding the model its own generated captions as auxiliary evidence. However, researchers from this study demonstrate that this approach can backfire: appending captions indiscriminately reduced Qwen2.5-VL-3B's accuracy on HallusionBench by nearly ten percentage points. To diagnose the problem, the team created GD-Probe, a diagnostic dataset pairing global and detail questions about the same image, revealing that caption utility is a per-query property — captions help with broad, global questions but hurt performance on fine-grained detail questions. The underlying mechanism is that an embedded caption competes with the image for the model's attention, and whether this is beneficial or harmful depends on whether the caption actually covers the content being queried. GEASS (Gated Evidence-Adaptive Selective Caption Trust) operationalizes this insight as a plug-in module that gates caption influence using the model's own confidence, weights it by the entropy reduction the caption induces, and raises the evidence threshold when the captioned and uncaptioned inference paths disagree. Evaluated across four VLMs on POPE and HallusionBench benchmarks, GEASS consistently outperforms both vanilla inference and contrastive decoding under a single fixed configuration, requiring only two forward passes and no additional trainable parameters.
What's missing
The two-forward-pass overhead is noted as minimal, but wall-clock latency comparisons against baseline inference are not provided. It is also unclear how GEASS performs on larger-scale VLMs (e.g., 70B+ parameter models) or in multilingual and multimodal settings beyond standard English visual QA.
What different sources said
- arXiv cs.AICenter
GEASS: Gated Evidence-Adaptive Selective Caption Trust for Vision-Language Models
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.