Researchers Introduce AraSEG, a Diverse Corpus for Improving Arabic Sentence Segmentation

A team of researchers has released AraSEG, a new benchmark corpus for Arabic sentence segmentation that covers eight distinct genres and a broad range of punctuation and document structure conditions. The dataset addresses a significant gap in existing approaches, which tend to rely heavily on punctuation cues and are evaluated primarily on well-formed text — conditions that do not reflect the diversity of real-world Arabic. Using AraSEG, the authors systematically evaluated large language models (LLMs), lightweight encoder models, and dependency parser-based models across increasingly challenging segmentation settings. Counterintuitively, lightweight encoders and even dependency parsers outperformed LLMs in the hardest conditions, suggesting that scale alone does not guarantee robustness for this task. The study also found that model performance eventually saturates with more training data, and that cross-genre generalization remains a persistent challenge. Notably, accurate sentence segmentation was shown to substantially improve downstream dependency parsing quality. All code, data, and models have been made publicly available.