Yujia Qin

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

2023 · arXiv (Cornell University)

Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading …

Exploring Mode Connectivity for Pre-trained Language Models

2022

Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found. Although plenty of …

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub

2023 · arXiv (Cornell University)

Large Language Models (LLMs) excel in traditional natural language processing tasks but struggle with problems that require complex domain-specific calculations or simulations. While equipping LLMs with external tools to build LLM-based agents can enhance their …

UniMem: Towards a Unified View of Long-Context Large Language Models

2024 · arXiv (Cornell University)

Long-context processing is a critical ability that constrains the applicability of large language models (LLMs). Although there exist various methods devoted to enhancing the long-context processing ability of LLMs, they are developed in an isolated …

Parameter-efficient fine-tuning of large-scale pre-trained language models

2023 · Nature Machine Intelligence

Abstract With the prevalence of pre-trained language models (PLMs) and the pre-training–fine-tuning paradigm, it has been continuously shown that larger models tend to yield better performance. However, as PLMs scale up, fine-tuning and storing all …