Siyan Li

Reducing Non-Normative Text Generation from Language Models

2020 · arXiv (Cornell University)

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet. Consequently, they are prone to generating non-normative text (i.e. in violation of social norms). We introduce a technique for …

Systematicity in GPT-3’s Interpretation of Novel English Noun Compounds

2022

Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or …