Empirical Evaluation of Similarity Coefficients for Multiagent Fault Localization

Detecting and diagnosing unwanted behavior in multiagent systems (MASs) are crucial to ascertain correct operation of agents. Current techniques assume a priori knowledge to identify unexpected behavior. However, generation of MAS models is both error-prone and time-consuming, as it exponentially increases with the number of agents and their interactions. In this paper, we describe a light-weight, automatic debugging-based technique, coined extended spectrum-based fault localization for MAS (ESFL-MAS), that shortens the diagnostic process, while only relying on minimal information about the system. ESFL-MAS uses a heuristic that quantifies the suspiciousness of an agent to be faulty. Different heuristics may have a different impact on the diagnostic quality of ESFL-MAS. Our experimental evaluation shows that 10 out of 42 heuristics (namely accuracy, coverage, Jaccard, Laplace, least contradiction, Ochiai, Rogers and Tanimoto, simple-matching, Sorensen-dice, and support) yield the best diagnostic accuracy (96.26% on average) in the context of the MAS used in our experiments.