SpringerLink
Log in

DoME: Dew computing based microservice execution in mobile edge using Q-learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The microservice approach unlatched a new window of distributed service-oriented architecture over the computational horizon. Microservice coordination in a heterogeneous and distributed manner among different edge computing nodes with minimum delay is the challenge of interest. Service migration to vantage locations accompanying bulk data hinge on bandwidth and internet connectivity. This paper proposes a Dew Computing-based Microservice Execution (DoME) scheme using reinforcement learning to alleviate the comprehensive service delay and provide seamless and real-time responses with optimized costs. With internet connectivity, DoME elects the edge server for service migration in exigency. During deficient internet connection, the service execution comes to pass at the dew server of the novel dew-cloud framework. This study imitates the Q-learning method comprehending the dew-edge-cloud framework and develops the Dew-Q algorithm to execute microservices on the move without continuous internet connectivity. The Dew-Q algorithm uses an agent that intelligently assimilates from historical data to envisage the service migration-node selection with the uphold connectivity. The proposed scheme is compared with existing microservice execution policies like FWS and SDGA schemes. The performance evaluations exhibit that the DoME scheme excels 41% to 68% compared with FWS and 12% to 86% with SDGA in terms of execution and migration cost. The proposed scheme dynamically controls microservice execution from the perspective of migration and timeliness with intermittent internet connectivity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Bhamare D, Samaka M, Erbad A, Jain R, Gupta L, Chan HA (2017) Multi-objective scheduling of micro-services for optimal service function chains. In: 2017 IEEE international conference on communications (ICC), IEEE. pp. 1–6

  2. ** W, Liu T, Cai Y, Kazman R, Mo R, Zheng Q (2019) Service candidate identification from monolithic systems based on execution traces. IEEE Trans Softw Eng 47(5):987–1007

    Article  Google Scholar 

  3. Niu Y, Liu F, Li Z (2018) Load balancing across microservices. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, IEEE. pp. 198–206

  4. Li J, Shen X, Chen L, Van DP, Jiannan O, Wosinska L, Chen J (2019) Service migration in fog computing enabled cellular networks to support real-time vehicular communications. IEEE Access 7:13704–13714

    Article  Google Scholar 

  5. Wang S, Guo Y, Zhang N, Yang P, Zhou A, Shen XS (2019) Delay-aware microservice coordination in mobile edge computing: A reinforcement learning approach. IEEE Trans Mob Comput 20(3):939–951

  6. Deng J, Li B, Wang J, Zhao Y (2021) Microservice Pre-Deployment Based on Mobility Prediction and Service Composition in Edge. In: 2021 IEEE International Conference on Web Services (ICWS), IEEE. pp. 569–578

  7. Wang Y, Zhao C, Yang S, Ren X, Wang L, Zhao P, Yang X (2020) Mpcsm: microservice placement for edge-cloud collaborative smart manufacturing. IEEE Trans Industr Inform 17(9):5898–5908

    Article  Google Scholar 

  8. Cao S, Wang Y, Xu C (2017) Service migrations in the cloud for mobile accesses: A reinforcement learning approach. In: 2017 International Conference on Networking, Architecture, and Storage (NAS), IEEE. pp. 1–10

  9. Wang Y, Pan Y (2015) Cloud-dew architecture: realizing the potential of distributed database systems in unreliable networks. WorldComp

  10. Wang Y (2016) Definition and categorization of dew computing. Open J Cloud Comput (OJCC) 3(1):1–7

    Google Scholar 

  11. Wang Y, Leblanc D (2016) "Integrating SaaS and SaaP with dew computing. In: 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE. pp. 590–594

  12. Rindos A, Wang Y (2016) Dew computing: The complementary piece of cloud computing. In: 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE. pp. 15–20

  13. Zhou Y, Zhang D, **ong N (2017) Post-cloud computing paradigms: a survey and comparison. Tsinghua Sci Technol 22(6):714–732

    Article  Google Scholar 

  14. Roy A, Midya S, Majumder K, Phadikar S (2020) Distributed resource management in dew based edge to cloud computing ecosystem: a hybrid adaptive evolutionary approach. Trans Emerg Telecommun Technol 31(8):e4018

    Google Scholar 

  15. Roy S, Sarkar D, De D (2021) DewMusic: crowdsourcing-based internet of music things in dew computing paradigm. J Ambient Intell Humaniz Comput 12(2):2103–2119

    Article  Google Scholar 

  16. Zou G, Qin Z, Deng S, Li K-C, Gan Y, Zhang B (2021) Towards the optimality of service instance selection in mobile edge computing. Knowl-Based Syst 217:106831

    Article  Google Scholar 

  17. Ray PP, Dash D, De D (2019) Internet of things-based real-time model study on e-healthcare: device, message service and dew computing. Comput Netw 149:226–239

    Article  Google Scholar 

  18. Longo M, Hirsch M, Mateos C, Zunino A (2019) Towards integrating mobile devices into dew computing: a model for hour-wise prediction of energy availability. Information 10(3):86

    Article  Google Scholar 

  19. Mukherjee A, De D, Ghosh SK, Buyya R (2021) Mobile edge computing. Springer Nature, Switzerland. https://www.springerprofessional.de/en/introduction-to-mobile-edge-computing/19880854

  20. Skala K, Davidovic D, Afgan E, Sovic I, Sojat Z (2015) Scalable distributed computing hierarchy: cloud, fog and dew computing. Open J Cloud Comput (OJCC) 2(1):16–24

    Google Scholar 

  21. Wang Y (2018) Post-cloud Computing Models: from Cloud to CDEF. In: Proceedings The 3rd International Workshop on Dew Computing. https://doi.org/10.13140/RG.2.2.34150.47688

  22. Jang B, Kim M, Harerimana G, Kim JW (2019) Q-learning algorithms: a comprehensive classification and applications. IEEE Access 7:133653–133667

    Article  Google Scholar 

  23. Abdoos M, Mozayani N, Bazzan ALC (2014) Hierarchical control of traffic signals using Q-learning with tile coding. Appl Intell 40(2):201–213

    Article  Google Scholar 

  24. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Article  Google Scholar 

  25. Fujita H, Selamat A, Lin JC-W, Ali M, eds. (2021) Advances and Trends in Artificial Intelligence. From Theory to Practice: 34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2021, Kuala Lumpur, Malaysia, July 26–29, 2021, Proceedings, Part II. Lecture Notes in Artificial Intelligence

  26. Asghari A, Sohrabi MK, Yaghmaee F (2020) Online scheduling of dependent tasks of cloud’s workflows to enhance resource utilization and reduce the makespan using multiple reinforcement learning-based agents. Soft Comput 24(21):16177–16199

    Article  Google Scholar 

  27. Chakraborty S, Mazumdar K, De D (2021) CBLM: Cluster-Based Location Management for 5G Small Cell Network Under Stochastic Environment. J Circuits Syst Comput 30(10):2150174

    Article  Google Scholar 

  28. Chakraborty S, Mazumdar K (2022) Sustainable task offloading decision using genetic algorithm in sensor mobile edge computing. J King Saud Univ-Comput Inf Sci 34:1552–1568

    Google Scholar 

  29. Melo FS (2001) Convergence of Q-learning: a simple proof. Institute of Systems and Robotics, Tech Rep:1–4. http://users.isr.ist.utl.pt/~mtjspaan/readingGroup/ProofQlearning.pdf

Download references

Author information

Authors and Affiliations

  1. Department of Electronics Engineering, Indian Institute of Technology (ISM), Dhanbad, Jharkhand, 826004, India

    Sheuli Chakraborty & Kaushik Mazumdar

  2. Centre of Mobile Cloud Computing, Department of Computer Science and Engineering, Maulana Abul Kalam Azad University of Technology, West Bengal, Nadia, West Bengal, 741249, India

    Debashis De

Authors
  1. Sheuli Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debashis De
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaushik Mazumdar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debashis De.

Ethics declarations

Conflicts of interest/competing interests

No conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, S., De, D. & Mazumdar, K. DoME: Dew computing based microservice execution in mobile edge using Q-learning. Appl Intell 53, 10917–10936 (2023). https://doi.org/10.1007/s10489-022-04087-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04087-x

Keywords

Search

Navigation