Autonomous Driving: Integration of Segmentation and Depth Camera in a Curriculum Learning Approach

Autonomous Driving (AD) entails vehicles that can perceive their surroundings and navigate without human intervention. This involves utilising a combination of sensors and algorithms to recognise obstacles, interpret traffic signals, and make driving decisions. While AD holds promise for transforming transportation by enhancing safety, reducing congestion, minimising pollution, and optimising efficiency, it poses technical challenges also. This work extends a novel approach to building an autonomous vehicle agent using Deep Reinforcement Learning (DRL) with Proximal Policy Optimisation (PPO) to navigate urban environments simulated by the CAR Learning to Act (CARLA) Simulator. The agent aims to maintain lane integrity and avoid collisions, even in adverse weather conditions. The proposed architecture integrates a 180-degree environmental view and various multimodal data inputs (RGB, segmentation, and depth camera inputs), extensively tested through experimentation. Notably, the integration of segmentation and depth data results in a 13% reduction in the collision rate, with the proposed agent achieving a total reward of 2510. This approach demonstrates significant progress over the previous framework, showcasing improved obstacle detection and collision avoidance accuracy. Moreover, these findings contribute to ongoing autonomous vehicle research, offering insights into effective strategies for developing robust and dependable driving agents capable of navigating urban environments and interacting with road infrastructure, contributing to advancements in Augmented Intelligence of Things (AIoT)-enabled autonomous driving.