Li Dong

Fault-tolerant distribution of GHZ states and controlled DSQC based on parity analyses

2017 · Optics Express

Based on the circuit including linear optical elements, a fault-tolerant distribution of GHZ states against collective noise among three parties is proposed. Additionally, two controlled DSQC protocols using the shared GHZ states as quantum channels …

Unified Language Model Pre-training for Natural Language Understanding and Generation

2019 · arXiv (Cornell University)

This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, …

Investigating Learning Dynamics of BERT Fine-Tuning

2020

Yaru Hao, Li Dong, Furu Wei, Ke Xu. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing. 2020.

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

2021 · arXiv (Cornell University)

The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences. In this paper, we introduce denoising word alignment as a new cross-lingual pre-training task. Specifically, the model first …

Interactive Continual Learning: Fast and Slow Thinking

2024

Advanced life forms, sustained by the synergistic interaction of neural cognitive mechanisms, continually acquire and transfer knowledge throughout their lifespan. In contrast, contemporary machine learning paradigms exhibit limitations in emulating the facets of continual learning …

Language Models as Inductive Reasoners

2024

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). …

Language to Logical Form with Neural Attention

2016 · arXiv (Cornell University)

Semantic parsing aims at mapping natural language to machine interpretable meaning representations. Traditional approaches rely on high-quality lexicons, manually-built templates, and linguistic features which are either domain- or representation-specific. In this paper we present a …

Question Answering over Freebase with Multi-Column Convolutional Neural Networks

2015

Li Dong, Furu Wei, Ming Zhou, Ke Xu. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Long Short-Term Memory-Networks for Machine Reading

2016 · arXiv (Cornell University)

In this paper we address the question of how to render sequence-level networks better at handling structured input. We propose a machine reading simulator which processes text incrementally from left to right and performs shallow …

Coarse-to-Fine Decoding for Neural Semantic Parsing

2018

Semantic parsing aims at mapping natural language utterances into structured meaning representations. In this work, we propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages. Given an input utterance, we …

Data-to-Text Generation with Content Selection and Planning

2019

Recent advances in data-to-text generation have led to the use of large-scale datasets and neural network models which are trained end-to-end, without explicitly modeling what to say and in what order. In this work, we …

Learning to Paraphrase for Question Answering

2017

Question answering (QA) systems are sensitive to the many different ways natural language expresses the same information need. In this paper we turn to paraphrases as a means of capturing this knowledge and present a …

Visualizing and Understanding the Effectiveness of BERT

2019

Yaru Hao, Li Dong, Furu Wei, Ke Xu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Ranking with Recursive Neural Networks and Its Application to Multi-Document Summarization

2015 · Proceedings of the AAAI Conference on Artificial Intelligence

We develop a Ranking framework upon Recursive Neural Networks (R2N2) to rank sentences for multi-document summarization. It formulates the sentence ranking task as a hierarchical regression process, which simultaneously measures the salience of a sentence …