Noam Levi

Pretraining Scaling Laws for Generative Evaluations of Language Models

2025 · arXiv (Cornell University)

Neural scaling laws have driven the field's ever-expanding exponential growth in parameters, data and compute. While scaling behaviors for pretraining losses and discriminative benchmarks are well established, generative benchmarks such as mathematical problem-solving or software …