I love your chain mail! Making knights smile in a fantasy game world:\n Open-domain goal-oriented dialogue agents

Dialogue research tends to distinguish between chit-chat and goal-oriented\ntasks. While the former is arguably more naturalistic and has a wider use of\nlanguage, the latter has clearer metrics and a straightforward learning signal.\nHumans effortlessly combine the two, for example engaging in chit-chat with the\ngoal of exchanging information or eliciting a specific response. Here, we\nbridge the divide between these two domains in the setting of a rich\nmulti-player text-based fantasy environment where agents and humans engage in\nboth actions and dialogue. Specifically, we train a goal-oriented model with\nreinforcement learning against an imitation-learned ``chit-chat'' model with\ntwo approaches: the policy either learns to pick a topic or learns to pick an\nutterance given the top-K utterances from the chit-chat model. We show that\nboth models outperform an inverse model baseline and can converse naturally\nwith their dialogue partner in order to achieve goals.\n