Yoon Kim

Sequence-Level Mixed Sample Data Augmentation

2020

Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. This work proposes a simple data augmentation approach to encourage compositional behavior in neural models for sequence-to-sequence problems. Our approach, …

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models

2023 · arXiv (Cornell University)

Despite their impressive capabilities, large language models (LLMs) are prone to hallucinations, i.e., generating content that deviates from facts seen during pretraining. We propose a simple decoding strategy for reducing hallucinations with pretrained LLMs that …

Value Augmented Sampling for Language Model Alignment and Personalization

2024 · ArXiv.org

Aligning Large Language Models (LLMs) to cater to different human preferences, learning new skills, and unlearning harmful behavior is an important problem. Search-based methods, such as Best-of-N or Monte-Carlo Tree Search, are performant, but impractical …

Character-Aware Neural Language Models

2015 · arXiv (Cornell University)

We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway network over characters, whose …

Sequence-Level Knowledge Distillation

2016 · arXiv (Cornell University)

Neural machine translation (NMT) offers a novel alternative formulation of translation that is potentially simpler than statistical approaches. However to reach competitive performance, NMT models need to be exceedingly large. In this paper we consider …

OpenNMT: Open-Source Toolkit for Neural Machine Translation

2017

We describe an open-source toolkit for neural machine translation (NMT). The toolkit prioritizes efficiency, modularity, and extensibility with the goal of supporting NMT research into model architectures, feature representations, and source modalities, while maintaining competitive …

OpenNMT: Neural Machine Translation Toolkit

2018 · arXiv (Cornell University)

OpenNMT is an open-source toolkit for neural machine translation (NMT). The system prioritizes efficiency, modularity, and extensibility with the goal of supporting NMT research into model architectures, feature representations, and source modalities, while maintaining competitive …

Compound Probabilistic Context-Free Grammars for Grammar Induction

2019

We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context free grammar. In contrast to traditional formulations which learn a single stochastic grammar, our context-free …

Character-Aware Neural Language Models

2016 · Proceedings of the AAAI Conference on Artificial Intelligence

We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway net work over characters, …