Johan Jeuring

Evaluating LLM-Generated Versus Human-Authored Responses in Role-Play Dialogues

2025 · arXiv (Cornell University)

Evaluating large language models (LLMs) in long-form, knowledge-grounded role-play dialogues remains challenging. This study compares LLM-generated and human-authored responses in multi-turn professional training simulations through human evaluation ($N=38$) and automated LLM-as-a-judge assessment. Human evaluation revealed …