Jared Casper

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

2015 · arXiv (Cornell University)

We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning …

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

2019 · arXiv (Cornell University)

Recent work in language modeling demonstrates that training large transformer models advances the state of the art in Natural Language Processing applications. However, very large models can be quite difficult to train due to memory …

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

2022 · arXiv (Cornell University)

Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few-shot and fine-tuning techniques. Because of their success, the size of these models has …