Yunhai Tong

Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

2021 · arXiv (Cornell University)

Pre-trained language models like BERT achieve superior performances in various NLP tasks without explicit consideration of syntactic information. Meanwhile, syntactic information has been proved to be crucial for the success of NLP applications. However, how …

Mitigating Semantic Confusion from Hostile Neighborhood for Graph Active Learning

2023

Graph Active Learning (GAL), which aims to find the most informative nodes in graphs for annotation to maximize the Graph Neural Networks (GNNs) performance, has attracted many research efforts but remains non-trivial challenges. One major …

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning

2024 · arXiv (Cornell University)

Parameter-efficient fine-tuning (PEFT) has been widely employed for domain adaptation, with LoRA being one of the most prominent methods due to its simplicity and effectiveness. However, in multi-task learning (MTL) scenarios, LoRA tends to obscure …