Training Autonomous Cyber Defense Agents: Challenges & Opportunities in Military Networks

This paper addresses the development and training of robust autonomous cyber defense (ACD) agents within military networks. We propose an architecture that integrates a hybrid AI model comprising Multi-Agent Reinforcement Learning (MARL), Large Language Models (LLMs), and a rule-based system into blue and red agent teams distributed across network devices. The primary goal is to automate key cybersecurity tasks such as monitoring, detection, and mitigation, thereby augmenting the capabilities of cybersecurity professionals in protecting critical military infrastructure. This architecture is designed to operate in modern network environments characterized by segmented clouds and software-defined controllers, which facilitate the deployment of ACD agents and other cybersecurity tools. The agent teams were evaluated in an Automated Cyber Operation (ACO) gym, which simulates NATO protected core networks and enables reproducible training and testing of autonomous agents. The paper concludes with an examination of the main challenges encountered in the training of ACD agents, with a particular focus on the security of the data and the robustness of the AI models during the training/testing phase.