Davis Liang

Embedding-based Zero-shot Retrieval through Query Generation

2020 · arXiv (Cornell University)

Passage retrieval addresses the problem of locating relevant passages, usually from a large corpus, given a query. In practice, lexical term-matching algorithms like BM25 are popular choices for retrieval owing to their efficiency. However, term-based …

Co-training and Co-distillation for Quality Improvement and Compression of Language Models

2023 · arXiv (Cornell University)

Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resource-constrained or real-time settings. However, most smaller models fail to surpass the performance of …