Multi-UAV Automatic Dynamic Obstacle Avoidance with Experience-shared A2C

With the increasing usage of UAV in reconnaissance, agriculture, logistics and entertainment, it's necessary for multi-UAV to automatically avoid the dynamic obstacles in order to ensure the safety of drones and livings in environment. The automatic obstacle avoidance is a classic multiple agent decision-making problem. Traditional algorithms, limited in the method of state classification and policy selection, are not applicable in such a complex scene including randomly dynamic scene and cooperative decision-making. In this paper, Advantaged Actor-Critic Algorithm is introduced to train multi-UAVs to automatically avoid obstacles and optimize avoidance decision-making model. Deep Q Learning, Actor-Critic (AC) and Advantaged Actor-Critic (A2C) algorithm are compared. And to further maximize the performance, we specifically improved A2C algorithm towards the multi-UAV scene by sharing experiences between UAVs to expedite the training process. Our experimental result shows our Experience-shared A2C (ES-A2C) algorithm leads to a higher performance and a shorter training period.