Yuhuai Wu

Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations

2016 · arXiv (Cornell University)

We investigate the parameter-space geometry of recurrent neural networks (RNNs), and develop an adaptation of path-SGD optimization method, attuned to this geometry, that can learn plain RNNs with ReLU activations. On several datasets that require …

STaR: Bootstrapping Reasoning With Reasoning

2022 · arXiv (Cornell University)

Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using …

Solving Quantitative Reasoning Problems with Language Models

2022 · arXiv (Cornell University)

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and …