Counterexample-Guided Learning Improves LLM Performance on Regular Expression Tasks
Researchers have developed a counterexample-guided learning framework that significantly boosts large language model (LLM) performance on regular-expression induction tasks. The approach uses a verifier to return targeted counterexamples when an LLM's proposed solution is incorrect, rather than simply providing more labeled training data. On the hardest task groups, success rates improved from 3.2% to 38.1% and from 38.9% to 74.1%, suggesting structured feedback can unlock more robust LLM reasoning.
A new preprint from arXiv proposes a counterexample-guided learning framework in which an LLM acts as a learner proposing candidate regular expressions, while a symbolic verifier acts as a teacher returning precise counterexamples that highlight differences between the candidate and target languages. The researchers introduced novel refinement strategies including regularization and symbolic counterexample clustering, as well as agentic techniques such as reflection and repair loops. Empirical results show that verifier feedback substantially reduces the number of labeled examples needed and enables learning of complex expressions where standard prompting fails entirely. On two distinct regex domains, the hardest task groups saw success rates jump from 3.2% to 38.1% and from 38.9% to 74.1% respectively. The authors argue these findings demonstrate that LLMs can leverage rich structured feedback in ways that go beyond simply treating it as additional training data. The work opens potential pathways for verifier-guided methods in LLM-based program synthesis and formal reasoning more broadly. Code, data, and resources have been made publicly available for research purposes.
What's missing
The study focuses exclusively on regular-expression induction as a testbed; it remains an open question how well counterexample-guided refinement generalizes to other formal reasoning or program synthesis domains. Computational cost and latency of the verifier-in-the-loop setup relative to standard prompting are not discussed in the abstract.
What different sources said
- arXiv cs.LGCenter
Counterexample Guided Learning in the Large using Reasoning Agents
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.