Xu Tan

Task-Agnostic and Adaptive-Size BERT Compression

2021

While pre-trained language models such as BERT and RoBERTa have achieved impressive results on various natural language processing tasks, they have huge numbers of parameters and suffer from huge computational and memory costs, which make …

MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition

2021 · arXiv (Cornell University)

In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR). MixSpeech trains an ASR model by taking a weighted combination of two different speech …

Prompttts: Controllable Text-To-Speech With Text Descriptions

2023

Using a text description as prompt to guide the generation of text or images (e.g., GPT-3 or DALLE-2) has drawn wide attention recently. Beyond text and image generation, in this work, we explore the possibility …

DiffusionNER: Boundary Diffusion for Named Entity Recognition

2023 · arXiv (Cornell University)

In this paper, we propose DiffusionNER, which formulates the named entity recognition task as a boundary-denoising diffusion process and thus generates named entities from noisy spans. During training, DiffusionNER gradually adds noises to the golden …

A Knowledge Graph Construction Method for Substation Infrared Operation and Maintenance Data Retrieval and Statistical Analysis

2024

The infrared operation and maintenance report of electrical equipment contains a large amount of equipment fault data, such as text, images and so on. However, these heterogeneous data are difficult to be analyzed and utilized …

Achieving Human Parity on Automatic Chinese to English News Translation

2018 · arXiv (Cornell University)

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether …

Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation

2018 · Neural Information Processing Systems

Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures. In this paper, we propose the concept of layer-wise coordination for NMT, which explicitly coordinates the learning of hidden representations …

Multilingual Neural Machine Translation with Knowledge Distillation

2019 · arXiv (Cornell University)

Multilingual machine translation, which translates multiple languages with a single model, has attracted much attention due to its efficiency of offline training and online serving. However, traditional multilingual translation usually yields inferior accuracy compared with …

MASS: Masked Sequence to Sequence Pre-training for Language Generation

2019 · arXiv (Cornell University)

Pre-training and fine-tuning, e.g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resource downstream tasks. Inspired by the success of BERT, we propose MAsked Sequence to …

Espnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit

2020

This paper introduces a new end-to-end text-to-speech (E2E-TTS) toolkit named ESPnet-TTS, which is an extension of the open-source speech processing toolkit ESPnet. The toolkit supports state-of- the-art E2E-TTS models, including Tacotron 2, Transformer TTS, and …

MPNet: Masked and Permuted Pre-training for Language Understanding

2020 · arXiv (Cornell University)

BERT adopts masked language modeling (MLM) for pre-training and is one of the most successful pre-training models. Since BERT neglects dependency among predicted tokens, XLNet introduces permuted language modeling (PLM) for pre-training to address this …