Effective Communications: A Joint Learning and Communication Framework\n for Multi-Agent Reinforcement Learning over Noisy Channels

We propose a novel formulation of the "effectiveness problem" in\ncommunications, put forth by Shannon and Weaver in their seminal work [2], by\nconsidering multiple agents communicating over a noisy channel in order to\nachieve better coordination and cooperation in a multi-agent reinforcement\nlearning (MARL) framework. Specifically, we consider a multi-agent partially\nobservable Markov decision process (MA-POMDP), in which the agents, in addition\nto interacting with the environment can also communicate with each other over a\nnoisy communication channel. The noisy communication channel is considered\nexplicitly as part of the dynamics of the environment and the message each\nagent sends is part of the action that the agent can take. As a result, the\nagents learn not only to collaborate with each other but also to communicate\n"effectively" over a noisy channel. This framework generalizes both the\ntraditional communication problem, where the main goal is to convey a message\nreliably over a noisy channel, and the "learning to communicate" framework that\nhas received recent attention in the MARL literature, where the underlying\ncommunication channels are assumed to be error-free. We show via examples that\nthe joint policy learned using the proposed framework is superior to that where\nthe communication is considered separately from the underlying MA-POMDP. This\nis a very powerful framework, which has many real world applications, from\nautonomous vehicle planning to drone swarm control, and opens up the rich\ntoolbox of deep reinforcement learning for the design of multi-user\ncommunication systems.\n