Zachary C. Lipton · PaperMetrix

Leveraging Unlabeled Data to Predict Out-of-Distribution Performance

2022 · arXiv (Cornell University)

Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this work, we investigate methods for predicting the target domain accuracy using only …

Characterizing Datapoints via Second-Split Forgetting

2022 · arXiv (Cornell University)

Researchers investigating example hardness have increasingly focused on the dynamics by which neural networks learn and forget examples throughout training. Popular metrics derived from these dynamics include (i) the epoch at which examples are first …

Downstream Datasets Make Surprisingly Good Pretraining Corpora

2023

For most natural language processing tasks, the dominant practice is to finetune large pretrained transformer models (e.g., BERT) using smaller downstream datasets.Despite the success of this approach, it remains unclear to what extent these gainsare …

Scaling Laws for Data Filtering—Data Curation Cannot be Compute Agnostic

2024

Vision-language models (VLMs) are trained for thousands of GPU hours on carefully selected subsets of massive web scrapes. For instance, the LAION public dataset retained only about 10% of the total crawled data. In recent …

Deep Active Learning for Named Entity Recognition

2017

Deep neural networks have advanced the state of the art in named entity recognition. However, under typical training procedures, advantages over classical methods emerge only with large datasets. As a result, deep learning is employed …

Learning The Difference That Makes A Difference With Counterfactually-Augmented Data

2020 · International Conference on Learning Representations