Darcey Riley

A Continuum of Generation Tasks for Investigating Length Bias and Degenerate Repetition

2022 · arXiv (Cornell University)

Language models suffer from various degenerate behaviors. These differ between tasks: machine translation (MT) exhibits length bias, while tasks like story generation exhibit excessive repetition. Recent work has attributed the difference to task constrainedness, but …