Hany Hassan Awadalla

Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers

2022 · arXiv (Cornell University)

Sparsely activated transformers, such as Mixture of Experts (MoE), have received great interest due to their outrageous scaling capability which enables dramatical increases in model size without significant increases in computational cost. To achieve this, …

How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation

2023 · arXiv (Cornell University)

Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a comprehensive evaluation of GPT models …