Samuel Smith

A Bayesian Perspective on Generalization and Stochastic Gradient Descent

2017 · arXiv (Cornell University)

We consider two questions at the heart of machine learning; how can we predict if a minimum will generalize to the test set, and why does stochastic gradient descent find minima that generalize well? Our …

Offline bilingual word vectors, orthogonal transformations and the inverted softmax

2017 · arXiv (Cornell University)

Usually bilingual word vectors are trained "online". Mikolov et al. showed they can also be found "offline", whereby two pre-trained embeddings are aligned with a linear transformation, using dictionaries compiled from expert knowledge. In this …