Multi-Agent Reinforcement Learning for Thermalling in Updrafts

View Video Presentation: https://doi.org/10.2514/6.2021-0864.vid Maximizing the energy intake by thermalling updrafts while avoiding collisions between two gliders or gliders with obstacles exceeds existing guidance strategies. Existing thermalling algorithms cannot independently incorporate multiple other gliders with an arbitrary number of updrafts and obstacles. Therefore, they would need to incorporate heuristics or be combined with other systems for this purpose. This leaves them vulnerable to unforeseen scenarios. In this research, we show how the outlined scenario can be formulated as an optimization problem and then propose to solve it with multi-agent reinforcement learning. We show the efficacy of this solution in different simulations using only the location of other gliders and a map of updrafts for the control strategy. This approach only requires the perception of other gliders, which is easier than also tracking heading or speed and is, therefore, applicable for future flight tests. Our approach has been successfully tested in simulations for up to a dozen gliders and also works on previously unseen problems like new environments and different sizes of the agent population.