SpringerLink
Log in

Trust-driven reinforcement selection strategy for federated learning on IoT devices

  • Special Issue Article
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Federated learning is a distributed machine learning approach that enables a large number of edge/end devices to perform on-device training for a single machine learning model, without having to share their own raw data. We consider in this paper a federated learning scenario wherein the local training is carried out on IoT devices and the global aggregation is done at the level of an edge server. One essential challenge in this emerging approach is IoT devices selection (also called scheduling), i.e., how to select the IoT devices to participate in the distributed training process. The existing approaches suggest to base the scheduling decision on the resource characteristics of the devices to guarantee that the selected devices would have enough resources to carry out the training. In this work, we argue that trust should be an integral part of the decision-making process and therefore design a trust establishment mechanism between the edge server and IoT devices. The trust mechanism aims to detect those IoT devices that over-utilize or under-utilize their resources during the local training. Thereafter, we introduce DDQN-Trust, a double deep Q learning-based selection algorithm that takes into account the trust scores and energy levels of the IoT devices to make appropriate scheduling decisions. Finally, we integrate our solution into four federated learning aggregation approaches, namely, FedAvg, FedProx, FedShare and FedSGD. Experiments conducted using a real-world dataset show that our DDQN-Trust solution always achieves better performance compared to two main benchmarks: the DQN and random scheduling algorithms. The results also reveal that FedProx outperforms the competitor aggregation models in terms of accuracy when integrated into our DDQN-Trust solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://www.cs.toronto.edu/~kriz/cifar.html.

  2. https://public.roboflow.com/object-detection/self-driving-car.

  3. https://github.com/gaith7/Trust_Fed.

References

  1. AbdulRahman S, Tout H, Mourad A, Talhi C (2020) FedMCCS: multicriteria client selection model for optimal IoT federated learning. IEEE Internet Things J 8(6):4723–4735

    Article  Google Scholar 

  2. Anh TT, Luong NC, Niyato D, Kim DI, Wang LC (2019) Efficient training management for mobile crowd-machine learning: a deep reinforcement learning approach. IEEE Wirel Commun Lett 8(5):1345–1348

    Article  Google Scholar 

  3. Bataineh AS, Bentahar J, Mizouni R, Abdel Wahab O, Rjoub G, El Barachi M (2021) Cloud computing as a platform for monetizing data services: a two-sided game business model. IEEE Trans Netw Serv Manag. https://doi.org/10.1109/TNSM.2021.3128160

    Article  Google Scholar 

  4. Bataineh AS, Bentahar J, Wahab OA, Mizouni R, Rjoub G (2020) A game-based secure trading of big data and IoT services: blockchain as a two-sided market. In: International conference on service-oriented computing. Springer, pp 85–100

  5. Bentahar J, Drawel N, Sadiki A (2022) Quantitative group trust: a two-stage verification approach. In: Faliszewski P, Mascardi V, Pelachaud C, Taylor ME (eds) Proceedings of the 21st international conference on autonomous agents and multiagent systems (AAMAS 2022), pp xx–xx

  6. Chen M, Yang Z, Saad W, Yin C, Poor HV, Cui S (2019) A joint learning and communications framework for federated learning over wireless networks. CoRR. ar**v:1909.07972

  7. Chen S, Shen C, Zhang L, Tang Y (2021) Dynamic aggregation for heterogeneous quantization in federated learning. IEEE Trans Wirel Commun 20:6804–6819

    Article  Google Scholar 

  8. Dai H, Zeng X, Yu Z, Wang T (2019) A scheduling algorithm for autonomous driving tasks on mobile edge computing servers. J Syst Archit 94:14–23

    Article  Google Scholar 

  9. Ding D, Fan X, Zhao Y, Kang K, Yin Q, Zeng J (2020) Q-learning based dynamic task scheduling for energy-efficient cloud computing. Future Gener Comput Syst 108:361–371

    Article  Google Scholar 

  10. Dinh CT, Tran NH, Nguyen MN, Hong CS, Bao W, Zomaya AY, Gramoli V (2020) Federated learning over wireless networks: convergence analysis and resource allocation. IEEE/ACM Trans Netw 29:398–409

    Article  Google Scholar 

  11. Drawel N, Bentahar J, Laarej A, Rjoub G (2020) Formalizing group and propagated trust in multi-agent systems. In: Bessiere C (ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020, pp 60–66. https://doi.org/10.24963/ijcai.2020/9

  12. Drawel N, Bentahar J, Laarej A, Rjoub G (2022) Formal verification of group and propagated trust in multi-agent systems. Auton Agents Multi-Agent Syst. https://doi.org/10.1007/s10458-021-09542-6 (in press)

    Article  Google Scholar 

  13. Hu C, Jiang J, Wang Z (2019) Decentralized federated learning: a segmented gossip approach. CoRR. ar**v:1908.07782

  14. Huang J, Li S, Chen Y (2020) Revenue-optimal task scheduling and resource management for IoT batch jobs in mobile edge computing. Peer Peer Netw Appl 13(5):1776–1787

    Article  Google Scholar 

  15. Iglewicz B, Hoaglin DC (1993) How to detect and handle outliers, vol 16. ASQ Press, Milwaukee

    Google Scholar 

  16. Khan LU, Saad W, Han Z, Hossain E, Hong CS (2021) Federated learning for internet of things: recent advances, taxonomy, and open challenges. IEEE Commun Surv Tutor 23:1759–1799

    Article  Google Scholar 

  17. Lei L, Tan Y, Zheng K, Liu S, Zhang K, Shen X (2020) Deep reinforcement learning for autonomous internet of things: model, applications and challenges. IEEE Commun Surv Tutor 22(3):1722–1760

    Article  Google Scholar 

  18. Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2018) Federated optimization in heterogeneous networks. ar**v preprint. ar**v:1812.06127

  19. Lin C, Deng D, Chih Y, Chiu H (2019) Smart manufacturing scheduling with edge computing using multiclass deep Q network. IEEE Trans Ind Inform 15(7):4276–4284

    Article  Google Scholar 

  20. Luo S (2020) Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput 91:106208

    Article  Google Scholar 

  21. Ma Z, Zhao M, Cai X, Jia Z (2021) Fast-convergent federated learning with class-weighted aggregation. J Syst Archit 117:102125

    Article  Google Scholar 

  22. Mao H, Alizadeh M, Menache I, Kandula S (2016) Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM workshop on hot topics in networks, pp 50–56

  23. McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. PMLR, pp 1273–1282

  24. Nesterov Y et al (2018) Lectures on convex optimization, vol 137. Springer, Berlin

    Google Scholar 

  25. Nguyen HT, Luong NC, Zhao J, Yuen C, Niyato D (2020) Resource allocation in mobility-aware federated learning networks: a deep reinforcement learning approach. In: 6th IEEE world forum on internet of things, WF-IoT 2020, New Orleans, LA, USA, June 2-16, 2020. IEEE, pp 1–6

  26. Nishio T, Yonetani R (2019) Client selection for federated learning with heterogeneous resources in mobile edge. In: IEEE international conference on communications, ICC 2019, Shanghai, China, May 20-24, 2019. IEEE, pp 1–7

  27. Rjoub G, Bentahar J, Abdel Wahab O, Saleh Bataineh A (2020) Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems. Pract Exp Concurr Comput 33:e5919

    Article  Google Scholar 

  28. Rjoub G, Bentahar J, Wahab OA (2020) Bigtrustscheduling: trust-aware big data task scheduling approach in cloud computing environments. Future Gener Comput Syst 110:1079–1097

    Article  Google Scholar 

  29. Rjoub G, Bentahar J, Wahab OA, Bataineh AS (2019) Deep smart scheduling: a deep learning approach for automated big data scheduling over the cloud. In: 7th International conference on future internet of things and cloud, FiCloud 2019, Istanbul, Turkey, August 26-28, 2019. IEEE, pp 189–196

  30. Rjoub G, Wahab OA, Bentahar J, Bataineh A (2020) A trust and energy-aware double deep reinforcement learning scheduling strategy for federated learning on IoT devices. In: International conference on service-oriented computing. Springer, pp 319–333

  31. Rjoub G, Wahab OA, Bentahar J, Bataineh AS (2021) Improving autonomous vehicles safety in snow weather using federated YOLO CNN learning. In: International conference on mobile web and intelligent information systems. Springer, pp 121–134

  32. Shamir O, Srebro N, Zhang T (2014) Communication-efficient distributed optimization using an approximate newton-type method. In: International conference on machine learning. PMLR, pp 1000–1008

  33. van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double Q-learning. In: Schuurmans D, Wellman MP (eds) Proceedings of the thirtieth AAAI conference on artificial intelligence, February 12-17, 2016, Phoenix, Arizona, USA. AAAI Press, pp 2094–2100

  34. Wahab OA, Bentahar J, Otrok H, Mourad A (2019) Resource-aware detection and defense system against multi-type attacks in the cloud: repeated Bayesian Stackelberg game. IEEE Trans Dependable Secur Comput 18:605–622

    Article  Google Scholar 

  35. Wahab OA, Cohen R, Bentahar J, Otrok H, Mourad A, Rjoub G (2020) An endorsement-based trust bootstrap** approach for newcomer cloud services. Inf Sci 527:159–175

    Article  Google Scholar 

  36. Wahab OA, Mourad A, Otrok H, Taleb T (2021) Federated machine learning: survey, multi-level classification, desirable criteria and future directions in communication and networking systems. IEEE Commun Surv Tutor 23:1342–1397

    Article  Google Scholar 

  37. Wang X, Han Y, Wang C, Zhao Q, Chen X, Chen M (2019) In-edge AI: intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Netw 33(5):156–165

    Article  Google Scholar 

  38. Yang HH, Liu Z, Quek TQS, Poor HV (2020) Scheduling policies for federated learning in wireless networks. IEEE Trans Commun 68(1):317–333

    Article  Google Scholar 

  39. Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-iid data. ar**v preprint. ar**v:1806.00582

  40. Zhou Z, Yang S, Pu L, Yu S (2020) CEFL: online admission control, data scheduling, and accuracy tuning for cost-efficient federated learning across edge nodes. IEEE Internet Things J 7(10):9341–9356

    Article  Google Scholar 

  41. Zhu G, Liu D, Du Y, You C, Zhang J, Huang K (2020) Toward an intelligent edge: wireless communication meets machine learning. IEEE Commun Mag 58(1):19–25

    Article  Google Scholar 

Download references

Funding

Funding was provided by Natural Sciences and Engineering Research Council of Canada (Discovery Grant), Defence Research and Development Canada (Innovation for Defence Excellence and Security (IDEaS)), Mitacs (Accelerate).

Author information

Authors and Affiliations

  1. Concordia University, Montreal, QC, Canada

    Gaith Rjoub, Jamal Bentahar & Ahmed Bataineh

  2. Université du Québec en Outaouais, Gatineau, QC, Canada

    Omar Abdel Wahab

Authors
  1. Gaith Rjoub
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Omar Abdel Wahab
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jamal Bentahar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmed Bataineh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaith Rjoub.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rjoub, G., Wahab, O.A., Bentahar, J. et al. Trust-driven reinforcement selection strategy for federated learning on IoT devices. Computing 106, 1273–1295 (2024). https://doi.org/10.1007/s00607-022-01078-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-022-01078-1

Keywords

Mathematics Subject Classification

Search

Navigation