Tomohiro Hayase

Understanding MLP-Mixer as a Wide and Sparse MLP

2023 · arXiv (Cornell University)

Multi-layer perceptron (MLP) is a fundamental component of deep learning, and recent MLP-based architectures, especially the MLP-Mixer, have achieved significant empirical success. Nevertheless, our understanding of why and how the MLP-Mixer outperforms conventional MLPs remains …