Nonparametric Bayesian Learning of Other Agents? Policies in Interactive POMDPs

We consider an autonomous POMDP agent facing a multiagent environment with unknown opponents, that are modeled as finite state controllers. The agent first learns the models from (imperfectly) observed behavior, and subsequently exploits them in planning for its own optimal policy by constructing an interactive POMDP. In the learning phase, Bayesian nonparametric methods are used to sample from the posterior distribution over the infinite-dimensional space of all possible controllers, resulting in models whose size scales with the complexity of observed behavior. Experimental results show that learning improves the agent's performance, which increases with the amount of data collected during the learning phase.