PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided\n Exploration

Model-based Reinforcement Learning (RL) is a popular learning paradigm due to\nits potential sample efficiency compared to model-free RL. However, existing\nempirical model-based RL approaches lack the ability to explore. This work\nstudies a computationally and statistically efficient model-based algorithm for\nboth Kernelized Nonlinear Regulators (KNR) and linear Markov Decision Processes\n(MDPs). For both models, our algorithm guarantees polynomial sample complexity\nand only uses access to a planning oracle. Experimentally, we first demonstrate\nthe flexibility and efficacy of our algorithm on a set of exploration\nchallenging control tasks where existing empirical model-based RL approaches\ncompletely fail. We then show that our approach retains excellent performance\neven in common dense reward control benchmarks that do not require heavy\nexploration. Finally, we demonstrate that our method can also perform\nreward-free exploration efficiently. Our code can be found at\nhttps://github.com/yudasong/PCMLP.\n