Adams Wei Yu

Normalized Gradient with Adaptive Stepsize Method for Deep Neural Network Training.

2017 · arXiv (Cornell University)

In this paper, we propose a generic and simple algorithmic framework for first order optimization. The framework essentially contains two consecutive steps in each iteration: 1) computing and normalizing the mini-batch stochastic gradient; 2) selecting …

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

2018 · arXiv (Cornell University)

Current end-to-end machine reading and question answering (Q\&A) models are primarily based on recurrent neural networks (RNNs) with attention. Despite their success, these models are often slow for both training and inference due to the …