Training human-like bots with Imitation Learning based on provenance data

Believable NonPlayer Characters in video games are one of the most challenging problems in the game industry over the last years. Players demand to expect to perceive the NPCs as other human-based player. Modeling NPC behavior manually is not always a good choice, mainly due to the number of NPCs a game can have and the difficulty of modeling a large number of actions that they can take. Our main goal is create a believable NPC acting like a real player. This work proposes an approach to training an NPC using Imitation Learning so that it is as similar as possible to a human player. Through this strategy, NPCs are trained from various types of players, avoiding predefined behaviors. Our proposal trains agents with the use of provenance data sets, tackling cause-effects data mining possibilities, and use Generative Adversarial Imitation Learning framework to take actions similar to what a player would take. The model proposed was create to be generic and applicable to various games. We validate our presented model with the DodgeBall environment inside Unity ML-Agents Toolkit for Unity Engine. Some players was asked to play against our agent and they validated the believability of our trained NPCs.