Sanmi Koyejo

SLSGD: Secure and Efficient Distributed On-device Machine Learning

2019 · arXiv (Cornell University)

We consider distributed on-device learning with limited communication and security requirements. We propose a new robust distributed optimization algorithm with efficient communication and attack tolerance. The proposed algorithm has provable convergence and robustness under non-IID …

Pretraining Scaling Laws for Generative Evaluations of Language Models

2025 · arXiv (Cornell University)

Neural scaling laws have driven the field's ever-expanding exponential growth in parameters, data and compute. While scaling behaviors for pretraining losses and discriminative benchmarks are well established, generative benchmarks such as mathematical problem-solving or software …

Are Emergent Abilities of Large Language Models a Mirage?

2023 · arXiv (Cornell University)

Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from …