Trajectory Optimization for 6G-UAV Based on Deep Reinforcement Learning

With more and more distributed deployment of Internet of Things (IoTs) in 5/6G networks, the unmanned aerial vehicles (UAVs) have attracted much interest for mobile edge computing service outside the coverage of terrestrial cellular systems. However, due to the interference and coverage hole issues of UAV-assisted aerial 3D communications, how to design a suitable UAV trajectory is crucial for the performance of future 6G-UAV. In order to achieve a balance between the communication quality and mission accomplishment, we define an environmental model with random interference and formulate a trajectory optimization problem to improve the communication coverage performance in high mobility conditions. By setting an interference point in the highly dynamic networks, we propose a novel trajectory optimization algorithm based on deep reinforcement learning (DRL) to minimize the overall task execution delay and expected outage duration. The received data information of UAVs is used for double deep$q$-network (DDQN) training while the prioritized empirical replay architecture in the prioritized replay deep$q$network (DQN) is utilized to select the best UAV action, avoid the UAV flying into the random interference points, and reduce the random interference. Simulation results demonstrate the superiority of the proposed algorithm.