Changhan Wang

Multilingual Speech Translation with Efficient Finetuning of Pretrained Models

2020 · arXiv (Cornell University)

We present a simple yet effective approach to build multilingual speech-to-text (ST) translation by efficient transfer learning from pretrained speech encoder and text decoder. Our key finding is that a minimalistic LNA (LayerNorm and Attention) …

A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

2021

Attention-based sequence-to-sequence modeling provides a powerful and elegant solution for applications that need to map one sequence to a different sequence. Its success heavily relies on the availability of large amounts of training data. This …

Pre-training for Speech Translation: CTC Meets Optimal Transport (2)

2023 · Zenodo (CERN European Organization for Nuclear Research)

Pre-trained models for the paper: Pre-training for Speech Translation: CTC Meets Optimal Transport. - MT models for MuST-C and CoVoST-2 - ASR and ST models for CoVoST (one-to-many and many-to-one)

Enhancing Speech-To-Speech Translation with Multiple TTS Targets

2023

It has been known that direct speech-to-speech translation (S2ST) models usually suffer from the data scarcity issue because of the limited existing parallel materials for both source and target speech. Therefore to train a direct …

Code-Switched Named Entity Recognition with Embedding Attention

2018

We describe our work for the CALCS 2018 shared task on named entity recognition on code-switched data. Our system ranked first place for MS Arabic-Egyptian named entity recognition and third place for English-Spanish.

Dynamic Meta-Embeddings for Improved Sentence Representations

2018

While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. …

Levenshtein Transformer

2019 · Neural Information Processing Systems

Modern neural sequence generation models are built to either generate tokens step-by-step from scratch or (iteratively) modify a sequence of tokens bounded by a fixed length. In this work, we develop Levenshtein Transformer, a new …

Neural Machine Translation with Byte-Level Subwords

2020 · Proceedings of the AAAI Conference on Artificial Intelligence

Almost all existing machine translation models are built on top of character-based vocabularies: characters, subwords or words. Rare characters from noisy text or character-rich languages such as Japanese and Chinese however can unnecessarily take up …

XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale

2022 · Interspeech 2022

This paper presents XLS-R, a large-scale model for cross-lingual speech representation learning based on wav2vec 2.0.We train models with up to 2B parameters on nearly half a million hours of publicly available speech audio in …