Ask Your Humans: Using Human Instructions to Improve Generalization in\n Reinforcement Learning

Complex, multi-task problems have proven to be difficult to solve efficiently\nin a sparse-reward reinforcement learning setting. In order to be sample\nefficient, multi-task learning requires reuse and sharing of low-level\npolicies. To facilitate the automatic decomposition of hierarchical tasks, we\npropose the use of step-by-step human demonstrations in the form of natural\nlanguage instructions and action trajectories. We introduce a dataset of such\ndemonstrations in a crafting-based grid world. Our model consists of a\nhigh-level language generator and low-level policy, conditioned on language. We\nfind that human demonstrations help solve the most complex tasks. We also find\nthat incorporating natural language allows the model to generalize to unseen\ntasks in a zero-shot setting and to learn quickly from a few demonstrations.\nGeneralization is not only reflected in the actions of the agent, but also in\nthe generated natural language instructions in unseen tasks. Our approach also\ngives our trained agent interpretable behaviors because it is able to generate\na sequence of high-level descriptions of its actions.\n