Alethea Power

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

2022 · arXiv (Cornell University)

In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. …