Tian Lin

Doubly Sparse: Sparse Mixture of Sparse Experts for Efficient Softmax Inference

2019 · arXiv (Cornell University)

Computations for the softmax function are significantly expensive when the number of output classes is large. In this paper, we present a novel softmax inference speedup method, Doubly Sparse Softmax (DS-Softmax), that leverages sparse mixture …