Exploration in Approximate Hyper-State Space for Meta Reinforcement\n Learning

To rapidly learn a new task, it is often essential for agents to explore\nefficiently -- especially when performance matters from the first timestep. One\nway to learn such behaviour is via meta-learning. Many existing methods however\nrely on dense rewards for meta-training, and can fail catastrophically if the\nrewards are sparse. Without a suitable reward signal, the need for exploration\nduring meta-training is exacerbated. To address this, we propose HyperX, which\nuses novel reward bonuses for meta-training to explore in approximate\nhyper-state space (where hyper-states represent the environment state and the\nagent's task belief). We show empirically that HyperX meta-learns better\ntask-exploration and adapts more successfully to new tasks than existing\nmethods.\n