A deep structure for option discovery in reinforcement learning

Hierarchical learning as another way to scale up reinforcement learning and enable its applications to very hard learning problems. Hierarchical learning is a divide-and-conquer technique a complex learning problem is decomposed into small pieces so that they can be easily solved. The option framework is one way to using hierarchical learning in reinforcement learning. In this paper we have used the free-energy based function approximation (FE-RBM) to determine the option initiation set. Our proposed method calculates the output for each of the input (including state and subgoal) according to negative free energy of an RBM. Learning is done by stochastic gradient descent and mean-squared error. The experimental results showed that this method has efficient functionality to create options. Moreover, it has a reasonable generalization ability for unvisited states.