Ferdinand Kossmann

Optimizing Mixture of Experts using Dynamic Recompilations

2022 · arXiv (Cornell University)

The Mixture of Experts architecture allows for outrageously large neural networks by scaling model parameter size independently from computational demand (FLOPs). However, current DNN frameworks cannot effectively support the dynamic data flow in Mixture of …