Richard Rutmann

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models

2025 · arXiv (Cornell University)

High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets remains limited. Existing state-of-the-art datasets mostly rely on heuristic filtering methods, restricting both their …