OrderDP: New Framework Achieves Lossless Data Pruning with Theoretical Guarantees
Researchers have proposed OrderDP, a data pruning framework that reduces training dataset size by over 40% while maintaining near-lossless model performance with theoretical guarantees. Existing data pruning methods tend to introduce biased gradient estimates by favoring highly informative samples, a problem OrderDP addresses through a two-stage random-then-top-q selection process tied to a surrogate loss. The work is significant because it offers both theoretical convergence and generalization guarantees alongside empirical validation on standard benchmarks, potentially lowering the computational cost of deep learning training substantially.
OrderDP is a plug-and-play data pruning framework introduced by Shengze Xu and colleagues, accepted at ICLR 2026, that targets the inefficiency of training large neural networks on full datasets. The core insight is that common data pruning strategies—which select the most informative samples—introduce systematic bias in gradient estimation relative to full-dataset training, and prior work has not rigorously characterized this bias or its downstream effect on model quality. OrderDP counters this by first drawing a random subset and then selecting the top-q samples within it, a procedure shown to be unbiased with respect to a surrogate loss function. The authors provide formal convergence and generalization analyses, giving practitioners theoretical control over the trade-off between training acceleration and final performance. Empirical evaluations on CIFAR-10, CIFAR-100, and ImageNet-1K show competitive accuracy and stable convergence compared to a range of baselines. The method reduces training cost by more than 40% while being simpler to implement and faster at runtime than competing approaches. Code has been made publicly available alongside the paper.
What's missing
It is unclear how OrderDP performs on modalities beyond image classification (e.g., language or multimodal tasks), whether the surrogate loss unbiasedness guarantee holds under distribution shift or continual learning settings, and how sensitive results are to the choice of the top-q hyperparameter across different dataset scales. Long-term effects on model robustness and fairness under aggressive pruning are not addressed.
What different sources said
- arXiv cs.LGCenter
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.