Wassim Bouaziz

A Mechanistic Study of Transformers Training Dynamics

2024 · arXiv (Cornell University)

Large-scale pretraining of transformers has been central to the success of foundation models. However, the scale of those models limits our understanding of the mechanisms at play during optimization. In this work, we study the …

On Monotonicity in AI Alignment

2025 · arXiv (Cornell University)

Comparison-based preference learning has become central to the alignment of AI models with human preferences. However, these methods may behave counterintuitively. After empirically observing that, when accounting for a preference for response $y$ over $z$, …