Yao Dou

Scarecrow: A Framework for Scrutinizing Machine Text.

2021 · arXiv (Cornell University)

Modern neural text generation systems can produce remarkably fluent and grammatical texts. While earlier language models suffered from repetition and syntactic errors, the errors made by contemporary models are often semantic, narrative, or discourse failures. …

Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluation

2023 · arXiv (Cornell University)

Fine-grained, span-level human evaluation has emerged as a reliable and robust method for evaluating text generation tasks such as summarization, simplification, machine translation and news generation, and the derived annotations have been useful for training …