Alexandre Muzio

Resource Central

2017

Cloud research to date has lacked data on the characteristics of the production virtual machine (VM) workloads of large cloud providers. A thorough understanding of these characteristics can inform the providers' resource management systems, e.g. …

Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers

2022 · arXiv (Cornell University)

Sparsely activated transformers, such as Mixture of Experts (MoE), have received great interest due to their outrageous scaling capability which enables dramatical increases in model size without significant increases in computational cost. To achieve this, …