Nanyun Peng

The Woman Worked as a Babysitter: On Biases in Language Generation

2019 · arXiv (Cornell University)

We present a systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups. In this work, we introduce the notion of the regard …

"Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses

2020 · arXiv (Cornell University)

Ad hominem attacks are those that target some feature of a person's character instead of the position the person is maintaining. These attacks are harmful because they propagate implicit biases and diminish a person's credibility. …

What do Bias Measures Measure

2021 · arXiv (Cornell University)

Natural Language Processing (NLP) models propagate social biases about protected attributes such as gender, race, and nationality. To create interventions and mitigate these biases and associated harms, it is vital to be able to detect …

On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark

2022 · Findings of the Association for Computational Linguistics: ACL 2022

Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corresponding dataset is scarce. We propose a taxonomy …

Evaluating Large Language Models on Controlled Generation Tasks

2023 · arXiv (Cornell University)

While recent studies have looked into the abilities of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc, there have been few studies looking into the controllability of large …

MERCY: Multiple Response Ranking Concurrently in Realistic Open-Domain Conversational Systems

2023

Automatic Evaluation (AE) and Response Selection (RS) models assign quality scores to various candidate responses and rank them in conversational setups. Prior response ranking research compares various models’ performance on synthetically generated test sets. In …

MacGyver: Are Large Language Models Creative Problem Solvers?

2024

Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas Griffiths, Faeze Brahman. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: …

Vulnerability of Large Language Models to Output Prefix Jailbreaks: Impact of Positions on Safety

2024

Previous research on jailbreak attacks has mainly focused on optimizing the adversarial snippet content injected into input prompts to expose LLM security vulnerabilities. A significant portion of this research focuses on developing more complex, less …

DRS: Deep Question Reformulation With Structured Output

2025

Question answering represents a core capability of large language models (LLMs).However, when individuals encounter unfamiliar knowledge in texts, they often formulate questions that the text itself cannot answer due to insufficient understanding of the underlying …

Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings

2015

We consider the task of named entity recognition for Chinese social media. The long line of work in Chinese NER has fo-cused on formal domains, and NER for social media has been largely restricted to …

Cross-Sentence N-ary Relation Extraction with Graph LSTMs

2017 · arXiv (Cornell University)

Past work in relation extraction has focused on binary relations in single sentences. Recent NLP inroads in high-value domains have sparked interest in the more general setting of extracting n-ary relations that span multiple sentences. …

On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

2019

Wasi Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, Nanyun Peng. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and …

Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings

2019

Despite advances in open-domain dialogue systems, automatic evaluation of such systems is still a challenging problem. Traditional reference-based metrics such as BLEU are ineffective because there could be many valid responses for a given context …

Cross-Sentence <i>N</i>-ary Relation Extraction with Graph LSTMs

2017 · Transactions of the Association for Computational Linguistics

Past work in relation extraction has focused on binary relations in single sentences. Recent NLP inroads in high-value domains have sparked interest in the more general setting of extracting n-ary relations that span multiple sentences. …

Improving Named Entity Recognition for Chinese Social Media with Word Segmentation Representation Learning

2016

Named entity recognition, and other information extraction tasks, frequently use linguistic features such as part of speech tags or chunkings. For languages where word boundaries are not readily identified in text, word segmentation is a …

Plan-and-Write: Towards Better Automatic Storytelling

2019 · Proceedings of the AAAI Conference on Artificial Intelligence

Automatic storytelling is challenging since it requires generating long, coherent natural language to describes a sensible sequence of events. Despite considerable efforts on automatic story generation in the past, prior work either is restricted in …

Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction

2019

Rujun Han, Qiang Ning, Nanyun Peng. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.