Ming Zhou

Enhancing the optical cross section of quantum antenna

2017 · Physical Review A

The classical radio-frequency antenna theory indicates that large cross sections can be realized through directional radiation. In this paper, a similar principle is applied in quantum systems, in which quantum antennas, constructed by a cluster …

Improving Question Answering by Commonsense-Based Pre-Training

2018 · arXiv (Cornell University)

Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge. We believe the main reason is the lack of commonsense \mbox{connections} …

Unified Language Model Pre-training for Natural Language Understanding and Generation

2019 · arXiv (Cornell University)

This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, …

Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study

2019

Sequence-to-sequence (seq2seq) models have achieved tremendous success in text generation tasks. However, there is no guarantee that they can always generate sentences without grammatical errors. In this paper, we present a preliminary empirical study on …

Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks

2019 · arXiv (Cornell University)

Haoyang Huang, Yaobo Liang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Ming Zhou. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language …

At Which Level Should We Extract? An Empirical Study on Extractive Document Summarization

2020 · arXiv (Cornell University)

Extractive methods have been proven effective in automatic document summarization. Previous works perform this task by identifying informative contents at sentence level. However, it is unclear whether performing extraction at sentence level is the best …

An Efficient Bayesian Neural Network for Multiple Data Streams

2021

Spatial and temporal data such as multiple data streams often have concept drift problems, which refers to changes of the data distributions over time. Once concept drift occurs, a stationary machine learning predictor will probably …

Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese

2021 · arXiv (Cornell University)

Although pre-trained models (PLMs) have achieved remarkable improvements in a wide range of NLP tasks, they are expensive in terms of time and resources. This calls for the study of training more efficient models with …

BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation

2022 · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Yuchen Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, Ming Zhou. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational …

Text Morphing

2018 · arXiv (Cornell University)

In this paper, we introduce a novel natural language generation task, termed as text morphing, which targets at generating the intermediate sentences that are fluency and smooth with the two input sentences. We propose the …

SVFLC: Secure and Verifiable Federated Learning With Chain Aggregation

2023 · IEEE Internet of Things Journal

As many countries have promulgated laws to protect users’ data privacy, how to legally use users’ data has become a hot topic. With the emergence of federated learning (FL) (also known as collaborative learning), multiple …

MT2: Towards a Multi-Task Machine Translation Model with Translation-Specific In-Context Learning

2023

Sentence-level translation, document-level translation, translation memory, and terminology constrained translation play an important role in machine translation. Most of the previous work uses separate models or methods to solve these tasks, which is not conducive …

On Realization of Intelligent Decision Making in the Real World: A Foundation Decision Model Perspective

2023 · CAAI Artificial Intelligence Research

The pervasive uncertainty and dynamic nature of real-world environments present significant challenges for the widespread implementation of machine-driven Intelligent Decision-Making (IDM) systems. Consequently, IDM should possess the ability to continuously acquire new skills and effectively …

Generating Chinese Classical Poems with Statistical Machine Translation Models

2021 · Proceedings of the AAAI Conference on Artificial Intelligence

This paper describes a statistical approach to generation of Chinese classical poetry and proposes a novel method to automatically evaluate poems. The system accepts a set of keywords representing the writing intents from a writer …

Question Answering over Freebase with Multi-Column Convolutional Neural Networks

2015

Li Dong, Furu Wei, Ming Zhou, Ke Xu. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Constraint-Based Question Answering with Knowledge Graph

2016 · International Conference on Computational Linguistics

WebQuestions and SimpleQuestions are two benchmark data-sets commonly used in recent knowledge-based question answering (KBQA) work. Most questions in them are ‘simple’ questions which can be answered based on a single relation in the knowledge …

Hierarchical Recurrent Attention Network for Response Generation

2018 · Proceedings of the AAAI Conference on Artificial Intelligence

We study multi-turn response generation in chatbots where a response is generated according to a conversation context. Existing work has modeled the hierarchy of the context, but does not pay enough attention to the fact …

Selective Encoding for Abstractive Sentence Summarization

2017

We propose a selective encoding model to extend the sequence-to-sequence framework for abstractive sentence summarization. It consists of a sentence encoder, a selective gate network, and an attention equipped decoder. The sentence encoder and decoder …

Question Answering and Question Generation as Dual Tasks

2017 · arXiv (Cornell University)

We study the problem of joint question answering (QA) and question generation (QG) in this paper. Our intuition is that QA and QG have intrinsic connections and these two tasks could improve each other. On …

S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension

2017 · arXiv (Cornell University)

In this paper, we present a novel approach to machine reading comprehension for the MS-MARCO dataset. Unlike the SQuAD dataset that aims to answer a question with exact text spans in a passage, the MS-MARCO …

Gated Self-Matching Networks for Reading Comprehension and Question Answering

2017

In this paper, we present the gated selfmatching networks for reading comprehension style question answering, which aims to answer questions from a given passage. We first match the question and passage with gated attention-based recurrent …

Question Generation for Question Answering

2017

This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data. The contribution of the …

Achieving Human Parity on Automatic Chinese to English News Translation

2018 · arXiv (Cornell University)

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether …

Fluency Boost Learning and Inference for Neural Grammatical Error Correction

2018

Most of the neural sequence-to-sequence (seq2seq) models for grammatical error correction (GEC) have two limitations: (1) a seq2seq model may not be well generalized with only limited error-corrected data; (2) a seq2seq model may fail …

Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study

2018 · arXiv (Cornell University)

Neural sequence-to-sequence (seq2seq) approaches have proven to be successful in grammatical error correction (GEC). Based on the seq2seq framework, we propose a novel fluency boost learning and inference mechanism. Fluency boosting learning generates diverse error-corrected …

Style Transfer as Unsupervised Machine Translation

2018 · arXiv (Cornell University)

Language style transferring rephrases text with specific stylistic attributes while preserving the original attribute-independent content. One main challenge in learning a style transfer system is a lack of parallel data where the source sentence is …

Neural Latent Extractive Document Summarization

2018

Extractive summarization models require sentence-level labels, which are usually created heuristically (e.g., with rule-based methods) given that most summarization datasets only have document-summary pairs. Since these labels might be suboptimal, we propose a latent variable …

Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots

2017

We study response selection for multiturn conversation in retrieval-based chatbots. Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or …

Dialog-to-action: conversational question answering over a large-scale knowledge base

2018 · Neural Information Processing Systems

We present an approach to map utterances in conversation to logical forms, which will be executed on a large-scale knowledge base. To handle enormous ellipsis phenomena in conversation, we introduce dialog memory management to manipulate …

Neural Document Summarization by Jointly Learning to Score and Select Sentences

2018

Sentence scoring and sentence selection are two main steps in extractive document summarization systems. However, previous works treat them as two separated subtasks. In this paper, we present a novel end-to-end neural network framework for …

HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization

2019

Neural extractive summarization models usually employ a hierarchical encoder for document encoding and they are trained using sentence-level labels, which are created heuristically using rule-based methods. Training the hierarchical encoder with these inaccurate labels is …

A Dependency-Based Neural Network for Relation Classification

2015

Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, Houfeng Wang. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume …

Attention-Guided Answer Distillation for Machine Reading Comprehension

2018

Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of numerous models. Besides, existing approaches are also vulnerable to adversarial attacks. …

Response Generation by Context-Aware Prototype Editing

2019 · Proceedings of the AAAI Conference on Artificial Intelligence

Open domain response generation has achieved remarkable progress in recent years, but sometimes yields short and uninformative responses. We propose a new paradigm, prototypethen-edit for response generation, that first retrieves a prototype response from a …

Asking Clarification Questions in Knowledge-Based Question Answering

2019

Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, Xu Sun. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on …

Graph Neural News Recommendation with Unsupervised Preference Disentanglement

2020

With the explosion of news information, personalized news recommendation has become very important for users to quickly find their interested contents. Most existing methods usually learn the representations of users and news from news contents …

MIND: A Large-scale Dataset for News Recommendation

2020

Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, Ming Zhou. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.

CodeBERT: A Pre-Trained Model for Programming and Natural Languages

2020

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou. Findings of the Association for Computational Linguistics: EMNLP 2020. 2020.

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation

2020

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung …

K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters

2021

We study the problem of injecting knowledge into large pre-trained models like BERT and RoBERTa. Existing methods typically update the original parameters of pre-trained models when injecting knowledge. However, when multiple kinds of knowledge are …

Ranking with Recursive Neural Networks and Its Application to Multi-Document Summarization

2015 · Proceedings of the AAAI Conference on Artificial Intelligence

We develop a Ranking framework upon Recursive Neural Networks (R2N2) to rank sentences for multi-document summarization. It formulates the sentence ranking task as a hierarchical regression process, which simultaneously measures the salience of a sentence …