Transformer-based On-Offline Hybrid Reinforcement Learning for Locomotion Tasks

In general, for a learner robot to learn an optimized behavior policy through reinforcement learning in locomotion tasks with continuous state and action spaces, a lot of trial and error experiences are required in the environment. To overcome the low data efficiency problem of online reinforcement learning, offline reinforcement learning methods using offline experience datasets are being actively studied in recent years. In this study, we propose a hybrid reinforcement learning framework that can effectively utilize online experience data in addition to offline datasets and then a Transformer-based policy network that reflects the temporal contextual information inherent in sequential experience data. In addition, to improve learning efficiency with the proposed hybrid reinforcement learning framework, a new priority sampling strategy is used to select a batch of training data from the trajectory replay buffer. Herein, we demonstrate the effectiveness and superiority of the proposed framework through various experiments on three different locomotion tasks provided by OpenAI Gym.