Orion Weller

MegaWika: Millions of reports and their sources across 50 diverse languages

2023 · arXiv (Cornell University)

To foster the development of new models for collaborative AI-assisted report generation, we introduce MegaWika, consisting of 13 million Wikipedia articles in 50 diverse languages, along with their 71 million referenced source materials. We process …

Dated Data: Tracing Knowledge Cutoffs in Large Language Models

2024 · arXiv (Cornell University)

Released Large Language Models (LLMs) are often paired with a claimed knowledge cutoff date, or the dates at which training data was gathered. Such information is crucial for applications where the LLM must provide up …