Yong Dai

Creating and Evaluating Resources for Sentiment Analysis in the Low-resource Language: Sindhi.

2021

In this paper, we develop Sindhi subjective lexicon using a merger of existing English resources: NRC lexicon, list of opinion words, SentiWordNet, Sindhi-English bilingual dictionary, and collection of Sindhi modifiers. The positive or negative sentiment …

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

2022 · arXiv (Cornell University)

With increasing privacy concerns on data, recent studies have made significant progress using federated learning (FL) on privacy-sensitive natural language processing (NLP) tasks. Much literature suggests fully fine-tuning pre-trained language models (PLMs) in the FL …

TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs

2023 · arXiv (Cornell University)

Large language models (LLMs) have shown impressive capabilities across various natural language tasks. However, evaluating their alignment with human preferences remains a challenge. To this end, we propose a comprehensive human evaluation framework to assess …

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game

2023 · arXiv (Cornell University)

Human preference alignment is essential to improve the interaction quality of large language models (LLMs). Existing alignment methods depend on manually annotated preference data to guide the LLM optimization directions. However, continuously updating LLMs for …

Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers

2024

Although dominant in natural language processing, transformer-based models still struggle with long-sequence processing, due to the computational costs of their self-attention operations, which increase exponentially as the length of the input sequence grows.To address this …