Spencer Frazier

Reducing Non-Normative Text Generation from Language Models

2020 · arXiv (Cornell University)

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet. Consequently, they are prone to generating non-normative text (i.e. in violation of social norms). We introduce a technique for …