Abstract
Federated learning (FL) has made possible the collaborative training of machine learning models between aggregation server and clients without sharing their private data. With the massive volume of heterogeneous data from various clients, the server faces challenges such as data unbalance, data corruption, and/or data irrelevancy. As a result, the FL setting is exposed to numerous security risks that lead to performance deterioration of learning effectiveness. To tackle the issue, in this paper we propose the Heterogeneity Index Based Clustering (HIC) approach, which enables the dynamic categorization of clients into clusters. Particularly, the model weights are dynamically clustered based on their heterogeneity level using an affinity propagation method. The HIC approach uses a simple, but effective way of scaling data heterogeneity and dynamic clustering to create a resilient learning system against backdoor attacks that outperforms the existing works on FL robustness. Our experimental results demonstrate that the clustering client’s weight based on their heterogeneity level decreases data unbalance and reduces attack success rate, increasing model performance, and encouraging clients’ contribution in FL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The full description of the dataset and source can be found in Sect. 5.
- 2.
References
McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data, pp 1273–1282
Li Q, Wen Z, Wu Z, Hu S, Wang N, Li Y, Liu X, He B (2021) A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Trans Knowl Data Eng 1
Chen Z, Tian P, Liao W, Yu W (2021) Towards multi-party targeted model poisoning attacks against federated learning systems. High-Confid Comput 1(1):100002
Tian P, Liao W, Yu W, Blasch E (2022) WSCC: a weight-similarity-based client clustering approach for non-iid federated learning. IEEE Internet Things J 9(20):20 243–20 256
Chen Z, Liao W, Hua K, Lu C, Yu W (2021) Towards asynchronous federated learning for heterogeneous edge-powered internet of things. Digit Commun Netw 7(3):317–326
Xu H, Yu W, Griffith D, Golmie N (2018) A survey on industrial internet of things: a cyber-physical systems perspective. IEEE Access 6:78 238–78 259
Liang F, Yu W, Liu X, Griffith D, Golmie N (2020) Toward edge-based deep learning in industrial internet of things. IEEE Internet Things J 7(5):4329–4341
Liang Y, Cai Z, Yu J, Han Q, Li Y (2018) Deep learning based inference of private information using embedded sensors in smart devices. IEEE Netw 32(4):8–14
Hatcher WG, Yu W (2018) A survey of deep learning: platforms, applications and emerging research trends. IEEE Access 6:24 411–24 432
Liang F, Hatcher WG, Liao W, Gao W, Yu W (2019) Machine learning for security and the internet of things: the good, the bad, and the ugly. IEEE Access 7:158 126–158 147
Xu H, Liu X, Yu W, Griffith D, Golmie N (2020) Reinforcement learning-based control and networking co-design for industrial internet of things. IEEE J Sel Areas Commun 38(5):885–898
Liang F, Yu W, Liu X, Griffith D, Golmie N (2022) Toward deep q-network-based resource allocation in industrial internet of things. IEEE Internet Things J 9(12):9138–9150
Qian C, Yu W, Lu C, Griffith D, Golmie N (2022) Toward generative adversarial networks for the industrial internet of things. IEEE Internet Things J 9(19):19 147–19 159
Balkus SV, Wang H, Cornet BD, Mahabal C, Ngo H, Fang H (2022) A survey of collaborative machine learning using 5g vehicular communications. IEEE Commun Surv Tutor 24(2):1280–1303
Al-Garadi MA, Mohamed A, Al-Ali AK, Du X, Ali I, Guizani M (2020) A survey of machine and deep learning methods for internet of things (IoT) security. IEEE Commun Surv Tutor 22(3):1646–1685
Liang F, Qian C, Yu W, Griffith D, Golmie N (2022) Survey of graph neural networks and applications. Wirel Commun Mob Comput
Chen Z, Tian P, Liao W, Yu W (2021) Zero knowledge clustering based adversarial mitigation in heterogeneous federated learning. IEEE Trans Netw Sci Eng 8(2):1070–1083
**ong Z, Cai Z, Takabi D, Li W (2022) Privacy threat and defense for federated learning with non-i.i.d. data in aiot. IEEE Trans Ind Inform 18(2):1310–1321
Ozdayi MS, Kantarcioglu M, Gel YR (2021) Defending against backdoors in federated learning with robust learning rate. Proc AAAI Conf Artif Intell 35(10):9268–9276. https://ojs.aaai.org/index.php/AAAI/article/view/17118
Blanchard P, El Mhamdi EM, Guerraoui R, Stainer J (2017) Machine learning with adversaries: byzantine tolerant gradient descent. Adv Neural Inf Process Syst 30
Zawad S, Ali A, Chen P-Y, Anwar A, Zhou Y, Baracaldo N, Tian Y, Yan F (2021) Curse or redemption? How data heterogeneity affects the robustness of federated learning. In: Proc AAAI conference artificial intelligence
Dean J, Corrado G, Monga R, Chen K, Devin M, Mao M, Ranzato M, Senior A, Tucker P, Yang K et al (2012) Large scale distributed deep networks. Adv Neural Inf Process Syst 25
Chen Z, Liao W, Tian P, Wang Q, Yu W (2022) A fairness-aware peer-to-peer decentralized learning framework with heterogeneous devices. Future Internet 14(5). https://www.mdpi.com/1999-5903/14/5/138
Zhu L, Liu Z, Han S (2019) Deep leakage from gradients. Adv Neural Inf Process Syst 32
Guerraoui R, Rouault S et al (2018) The hidden vulnerability of distributed learning in byzantium. International conference on machine learning. PMLR 2018:3521–3530
Li T, Sanjabi M, Beirami A, Smith V (2020) Fair resource allocation in federated learning. In: International conference on learning representations. https://openreview.net/forum?id=ByexElSYDr
Sun Z, Kairouz P, Suresh AT, McMahan HB (2019) Can you really backdoor federated learning?. In: 2nd international workshop on federated learning for data privacy and confidentiality, NeurIPS
Geyer RC, Klein T, Nabi M (2017) Differentially private federated learning: a client level perspective. In: NIPS workshop
Fung C, Yoon CJ, Beschastnikh I (2018) Mitigating sybils in federated learning poisoning. ar**v:1808.04866
Bernstein J, Zhao J, Azizzadenesheli K, Anandkumar A (2019) signSGD with majority vote is communication efficient and fault tolerant. https://openreview.net/forum?id=BJxhijAcY7
Ghosh A, Chung J, Yin D, Ramchandran K (2020) An efficient framework for clustered federated learning. Adv Neural Inf Process Syst 33:19 586–19 597
Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020) Federated optimization in heterogeneous networks. Proc Mach Learn Syst 2:429–450
Fallah A, Mokhtari A, Ozdaglar A (2020) Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. Adv Neural Inf Process Syst 33:3557–3568
Wang J, Liu Q, Liang H, Joshi G, Poor HV (2020) Tackling the objective inconsistency problem in heterogeneous federated optimization. Adv Neural Inf Process Syst 33:7611–7623
Pang J, Huang Y, **e Z, Han Q, Cai Z (2021) Realizing the heterogeneity: a self-organized federated learning framework for iot. IEEE Internet Things J 8(5):3088–3098
Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-iid data. ar**v:1806.00582
Sattler F, Wiedemann S, Müller K-R, Samek W (2019) Robust and communication-efficient federated learning from non-iid data. IEEE Trans Neural Netw Learn Syst 31(9):3400–3413
Chen B, Carvalho W, Baracaldo N, Ludwig H, Edwards B, Lee T, Molloy I, Srivastava B (2018) Detecting backdoor attacks on deep neural networks by activation clustering. ar**v:1811.03728
**e C, Huang K, Chen P-Y, Li B (2019) Dba: distributed backdoor attacks against federated learning. In: International conference on learning representations
Bhagoji AN, Chakraborty S, Mittal P, Calo S (2019) Analyzing federated learning through an adversarial lens. In: International conference on machine learning. PMLR, pp 634–643
Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315(5814):972–976
Caldas S, Duddu SMK, Wu P, Li T, Konečnỳ J, McMahan HB, Smith V, Talwalkar A (2018) Leaf: a benchmark for federated settings. ar**v:1812.01097
Nagalapatti L, Narayanam R (2021) Game of gradients: mitigating irrelevant clients in federated learning. Proc AAAI Conf Artif Intell 35(10):9046–9054
Acknowledgements
This material is based upon work in part supported by the Air Force Office of Scientific Research under award number FA9550-20-1-0418. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pene, P., Tian, P., Liao, W., Wang, Q., Yu, W. (2024). Robust Federated Learning: A Heterogeneity Index Based Clustering Approach. In: Lee, R. (eds) Software Engineering and Management: Theory and Application. Studies in Computational Intelligence, vol 1137. Springer, Cham. https://doi.org/10.1007/978-3-031-55174-1_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-55174-1_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55173-4
Online ISBN: 978-3-031-55174-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)