Samet Oymak

Transformers as Algorithms: Generalization and Stability in In-context Learning

2023 · arXiv (Cornell University)

In-context learning (ICL) is a type of prompting where a transformer model operates on a sequence of (input, output) examples and performs inference on-the-fly. In this work, we formalize in-context learning as an algorithm learning …

Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective

2024 · arXiv (Cornell University)

Modern classification problems exhibit heterogeneities across individual classes: Each class may have unique attributes, such as sample size, label quality, or predictability (easy vs difficult), and variable importance at test-time. Without care, these heterogeneities impede …