Ollie Liu

How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

2023 · arXiv (Cornell University)

Pre-trained language models can be surprisingly adept at tasks they were not explicitly trained on, but how they implement these capabilities is poorly understood. In this paper, we investigate the basic mathematical abilities often acquired …

Resa: Transparent Reasoning Models via SAEs

2025 · arXiv (Cornell University)

How cost-effectively can we elicit strong reasoning in language models by leveraging their underlying representations? We answer this question with Resa, a family of 1.5B reasoning models trained via a novel and efficient sparse autoencoder …