Anomaly Localization in Model Gradients Under Backdoor Attacks Against\n Federated Learning

Inserting a backdoor into the joint model in federated learning (FL) is a\nrecent threat raising concerns. Existing studies mostly focus on developing\neffective countermeasures against this threat, assuming that backdoored local\nmodels, if any, somehow reveal themselves by anomalies in their gradients.\nHowever, this assumption needs to be elaborated by identifying specifically\nwhich gradients are more likely to indicate an anomaly to what extent under\nwhich conditions. This is an important issue given that neural network models\nusually have huge parametric space and consist of a large number of weights. In\nthis study, we make a deep gradient-level analysis on the expected variations\nin model gradients under several backdoor attack scenarios against FL. Our main\nnovel finding is that backdoor-induced anomalies in local model updates\n(weights or gradients) appear in the final layer bias weights of the malicious\nlocal models. We support and validate our findings by both theoretical and\nexperimental analysis in various FL settings. We also investigate the impact of\nthe number of malicious clients, learning rate, and malicious data rate on the\nobserved anomaly. Our implementation is publicly available\\footnote{\\url{\nhttps://github.com/ArcelikAcikKaynak/Federated_Learning.git}}.\n