Xiang Ren

Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

2018 · arXiv (Cornell University)

Motivation: State-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases. Although recent studies explored using neural network models for BioNER to free …

Learning Named Entity Tagger using Domain-Specific Dictionary

2018 · arXiv (Cornell University)

Recent advances in deep neural models allow us to build reliable named entity recognition (NER) systems without handcrafting features. However, such methods require large amounts of manually-labeled training data. There have been efforts on replacing …

LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation

2020 · arXiv (Cornell University)

Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from and collecting the requisite number of labels involves massive human effort. In …

Pre-training Text-to-Text Transformers for Concept-centric Common Sense

2021 · arXiv (Cornell University)

Pretrained language models (PTLM) have achieved impressive results in a range of natural language understanding (NLU) and generation (NLG) tasks that require a syntactic and semantic understanding of the text. However, current pre-training objectives such …

Learning to Generate Task-Specific Adapters from Task Description

2021

Qinyuan Ye, Xiang Ren. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 2021.

KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.

Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources

2021 · arXiv (Cornell University)

Warning: this paper contains content that may be offensive or upsetting. Numerous natural language processing models have tried injecting commonsense by using the ConceptNet knowledge base to improve performance on different tasks. ConceptNet, however, is …

L-BGNN: Layerwise Trained Bipartite Graph Neural Networks

2022 · IEEE Transactions on Neural Networks and Learning Systems

Learning low-dimensional representations of bipartite graphs enables e-commerce applications, such as recommendation, classification, and link prediction. A layerwise-trained bipartite graph neural network (L-BGNN) embedding method, which is unsupervised, efficient, and scalable, is proposed in this …

FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks

2021 · arXiv (Cornell University)

Increasing concerns and regulations about data privacy and sparsity necessitate the study of privacy-preserving, decentralized learning methods for natural language processing (NLP) tasks. Federated learning (FL) provides promising approaches for a large number of clients …

On the Robustness of Reading Comprehension Models to Entity Renaming

2021 · arXiv (Cornell University)

We study the robustness of machine reading comprehension (MRC) models to entity renaming -- do models make more wrong predictions when the same questions are asked about an entity whose name has been changed? Such …

PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning

2023 · arXiv (Cornell University)

Procedural planning, which entails decomposing a high-level goal into a sequence of temporally ordered steps, is an important yet intricate task for machines. It involves integrating common-sense knowledge to reason about complex and often contextualized …

Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step

2023 · arXiv (Cornell University)

Chain-of-thought prompting (e.g., "Let's think step-by-step") primes large language models to verbalize rationalization for their predictions. While chain-of-thought can lead to dramatic performance gains, benefits appear to emerge only for sufficiently large models (beyond 50B …

LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

2023

We present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs). Our framework consists of two modules: PairRanker and GenFuser, addressing the …

APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning

2023

Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

CoType

2017

Extracting entities and relations for types of interest from text is important for understanding massive text corpora. Traditionally, systems of entity relation extraction have relied on human-annotated corpora for training and adopted an incremental pipeline. …

Empower Sequence Labeling with Task-Aware Neural Language Model

2018 · Proceedings of the AAAI Conference on Artificial Intelligence

Linguistic sequence labeling is a general approach encompassing a variety of problems, such as part-of-speech tagging and named entity recognition. Recent advances in neural networks (NNs) make it possible to build reliable models without handcrafted …

Jointly Learning Explainable Rules for Recommendation with Knowledge Graph

2019

Explainability and effectiveness are two key aspects for building recommender systems. Prior efforts mostly focus on incorporating side information to achieve better recommendation performance. However, these methods have some weaknesses: (1) prediction of neural network-based …

CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases

2016 · arXiv (Cornell University)

Extracting entities and relations for types of interest from text is important for understanding massive text corpora. Traditionally, systems of entity relation extraction have relied on human-annotated corpora for training and adopted an incremental pipeline. …

Hierarchical Graph Representation Learning with Differentiable Pooling

2018 · arXiv (Cornell University)

Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction. However, current GNN methods …

Empower Sequence Labeling with Task-Aware Neural Language Model

2017 · arXiv (Cornell University)

Linguistic sequence labeling is a general modeling approach that encompasses a variety of problems, such as part-of-speech tagging and named entity recognition. Recent advances in neural networks (NNs) make it possible to build reliable models …

KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

2019

Bill Yuchen Lin, Xinyue Chen, Jamin Chen, Xiang Ren. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge

2021

Question: I have five fingers but I am not alive. What am I? Answer: a glove