Zhiying Jiang

Investigating the Limitations of Transformers with Simple Arithmetic Tasks

2021 · arXiv (Cornell University)

The ability to perform arithmetic tasks is a remarkable trait of human intelligence and might form a critical component of more complex reasoning tasks. In this work, we investigate if the surface form of a …

“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors

2023

Deep neural networks (DNNs) are often used for text classification due to their high accuracy. However, DNNs can be computationally intensive, requiring millions of parameters and large amounts of labeled data, which can make them …

Approximating Human-Like Few-shot Learning with GPT-based Compression

2023 · arXiv (Cornell University)

In this work, we conceptualize the learning process as information compression. We seek to equip generative pre-trained models with human-like learning capabilities that enable data compression during inference. We present a novel approach that utilizes …

Document Ranking with a Pretrained Sequence-to-Sequence Model

2020

This work proposes the use of a pretrained sequence-to-sequence model for document ranking. Our approach is fundamentally different from a commonly adopted classificationbased formulation based on encoder-only pretrained transformer architectures such as BERT. We show …