Albert Gatt

Conceptualisation in reference production: Probabilistic modelling and experimental testing

2019

In psycholinguistics, there has been relatively little work investigating conceptualisation –how speakers decide which concepts to express. This contrasts with work in natural language generation (NLG), a subfield of AI, where much research has explored …

A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences

2024 · arXiv (Cornell University)

The reasoning abilities of Large Language Models (LLMs) are becoming a central focus of study in NLP. In this paper, we consider the case of syllogistic reasoning, an area of deductive reasoning studied extensively in …

Evaluating LLM-Generated Versus Human-Authored Responses in Role-Play Dialogues

2025 · arXiv (Cornell University)

Evaluating large language models (LLMs) in long-form, knowledge-grounded role-play dialogues remains challenging. This study compares LLM-generated and human-authored responses in multi-turn professional training simulations through human evaluation ($N=38$) and automated LLM-as-a-judge assessment. Human evaluation revealed …

References Matter: Investigating the Impact of Reference Set Variation on Summarization Evaluation

2025 · arXiv (Cornell University)

Human language production exhibits remarkable richness and variation, reflecting diverse communication styles and intents. However, this variation is often overlooked in summarization evaluation. While having multiple reference summaries is known to improve correlation with human …

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

2018 · Journal of Artificial Intelligence Research

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of …

Best practices for the human evaluation of automatically generated text

2019

Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated, with a particularly high degree of variation in the way that human evaluation is carried out. This paper provides …