iOSWorld: New Benchmark Tests AI Agents' Ability to Understand Personal Data on iPhones
Researchers have released iOSWorld, the first interactive native iOS simulator benchmark designed to evaluate AI phone agents on personalized, user-specific tasks across 26 custom-built apps. Unlike existing mobile agent benchmarks, iOSWorld incorporates a persistent user identity with connected data such as messages, transactions, and social relationships. The benchmark reveals significant limitations in current AI models, with the best-performing system achieving only 52% overall accuracy and dropping to 37% on complex multi-app tasks.
iOSWorld is an open-source benchmark introduced by researchers to address a gap in mobile AI agent evaluation: the ability to reason over a user's personal identity, history, and preferences rather than executing isolated instructions in a generic environment. The benchmark spans 26 newly built iOS apps containing interconnected personal data including financial activity, travel records, messages, and social relationships. It includes 133 tasks divided into three difficulty tiers — 27 single-app tasks, 60 multi-app tasks spanning up to 8 apps, and 46 memory-and-personalization tasks requiring inference from personal data patterns. Evaluations of both frontier and open-source computer-use models show that the best configuration, using privileged vision combined with accessibility-tree (XML) input, reaches 52% overall accuracy, while multi-app tasks remain particularly challenging at 37%. Notably, providing accessibility-tree data improved frontier models by up to 26 percentage points, but smaller models showed no meaningful benefit from this additional input. The benchmark, including all apps, seeded data, tasks, rubrics, and evaluation code, has been released publicly to support further research.
What's missing
It is unclear how the seeded personal data was constructed for realism or privacy considerations, and whether the benchmark's closed iOS simulation environment limits transferability to real-world deployment scenarios.
What different sources said
- arXiv cs.LGCenter
iOSWorld: A Benchmark for Personally Intelligent Phone Agents
Related
Gut Bacteria Enzyme Found to Break Down Heat-Processed Food Compounds, Producing Novel Biogenic Amines
Researchers have discovered that an enzyme in common gut bacteria can degrade N-epsilon-carboxymethyllysine (CML), a compound formed during thermal food processing, producing previously unknown biogenic amines. The enzyme, ornithine decarboxylase SpeC from enterobacteria, acts on CML and related modified lysine derivatives through a low-level 'underground' catalytic activity. This finding suggests a previously unrecognized communication axis between thermally processed dietary compounds and gut microbial physiology, with potential implications for host health.
Full-Length Gene Sequencing Reveals Two Distinct Bacterial Communities in Black-Legged Ticks Expanding Into Canada
Researchers used Oxford Nanopore full-length 16S rRNA gene sequencing to characterize the microbiome of Ixodes scapularis black-legged ticks collected in Nova Scotia, Canada, distinguishing between tick-adapted bacteria and environmentally acquired bacteria. The study comes as I. scapularis — the primary vector of Lyme disease — is rapidly expanding northward into Canada due to climate change. The findings suggest that environmentally derived bacteria in tick microbiomes are not mere contamination, which has implications for how tick microbiome data is collected and interpreted across surveillance studies.
Study Identifies Metabolic Link Between Cell Envelope Stress and Biofilm Formation in Bacteria
Researchers have discovered that the metabolite acetyl-CoA directly inhibits enzymes that degrade the bacterial signaling molecule c-di-GMP, connecting cell envelope biosynthesis stress to biofilm formation in Pseudomonas aeruginosa. The study found that sub-inhibitory concentrations of antibiotics targeting early peptidoglycan biosynthesis — but not other antibiotic classes — elevate c-di-GMP levels by reducing phosphodiesterase activity, with acetyl-CoA competing for the enzyme active site. Because the relevant enzyme domain is broadly conserved across bacterial species, this checkpoint mechanism may be widespread and could have implications for understanding antibiotic-induced biofilm responses.