Tatsunori Hashimoto
11 papers in the PaperMetrix corpus
Papers by this author
-
Distributionally Robust Models with Parametric Likelihood Ratios
2022 · arXiv (Cornell University)
As machine learning models are deployed ever more broadly, it becomes increasingly important that they are not only able to perform well on their training distribution, but also yield accurate predictions when confronted with distribution …
-
Spurious Correlations in Reference-Free Evaluation of Text Generation
2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Model-based, reference-free evaluation metrics have been proposed as a fast and cost-effective approach to evaluate Natural Language Generation (NLG) systems. Despite promising recent results, we find evidence that reference-free evaluation metrics of summarization and dialog …
-
Diffusion-LM Improves Controllable Text Generation
2022 · arXiv (Cornell University)
Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation. While recent works have demonstrated successes on controlling simple sentence attributes (e.g., sentiment), there has been little …
-
Emergent Abilities of Large Language Models
2022 · arXiv (Cornell University)
Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities …
-
Contrastive Error Attribution for Finetuned Language Models
2022 · arXiv (Cornell University)
Recent work has identified noisy and misannotated data as a core cause of hallucinations and unfaithful outputs in Natural Language Generation (NLG) tasks. Consequently, identifying and removing these examples is a key open challenge in …
-
Likelihood-Based Diffusion Language Models
2023 · arXiv (Cornell University)
Despite a growing interest in diffusion-based language models, existing work has not shown that these models can attain nontrivial likelihoods on standard language modeling benchmarks. In this work, we take the first steps towards closing …
-
Generating Sentences by Editing Prototypes
2018 · Transactions of the Association for Computational Linguistics
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from …
-
Unifying Human and Statistical Evaluation for Natural Language Generation
2019
Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
-
Distributionally Robust Language Modeling
2019
Yonatan Oren, Shiori Sagawa, Tatsunori B. Hashimoto, Percy Liang. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
-
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
2021
Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, …
-
Benchmarking Large Language Models for News Summarization
2024 · Transactions of the Association for Computational Linguistics
Abstract Large language models (LLMs) have shown promise for automatic summarization but the reasons behind their successes are poorly understood. By conducting a human evaluation on ten LLMs across different pretraining methods, prompts, and model …