Self-Attention for Deep Reinforcement Learning

Reinforcement learning is concerned with how software agents ought to take actions according to the state of the environment so as to maximize some notion of cumulative reward. Therefore, in-depth study and mining of the state of the environment will be more conducive to the agent to make better decisions. Motivated by the advantages of self-attention mechanism in machine translation, this paper presents a new scheme. In this scheme, the state in deep reinforcement learning algorithms can be combined with self-attention mechanism. After that agents will pay more attention to the internal structure of state especially in a complex game environment, like real-time strategy game StarCraft. StarCraft is a huge challenge platform for AI researchers because of its huge state spaces and action spaces. Some baseline agents of reinforcement learning provided by DeepMind for mini-games in StarCraft II have not reached the level of an amateur player. Our agents use fewer features than DeepMind's baseline agents and have made significant improvement.