Moustapha Cissé

Efficient softmax approximation for GPUs

2017 · International Conference on Machine Learning

We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies. Our approach, called adaptive softmax, circumvents the linear dependency on the vocabulary size by exploiting the unbalanced word …