Vedant Misra

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

2022 · arXiv (Cornell University)

In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. …

PaLM: Scaling Language Modeling with Pathways

2022 · arXiv (Cornell University)

Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to …

Solving Quantitative Reasoning Problems with Language Models

2022 · arXiv (Cornell University)

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and …