Robert Mullins

Wide Attention Is The Way Forward For Transformers?

2022 · arXiv (Cornell University)

The Transformer is an extremely powerful and prominent deep learning architecture. In this work, we challenge the commonly held belief in deep learning that going deeper is better, and show an alternative design approach that …