SpringerLink
Log in

Delay Constrained SFC Orchestration for Edge Intelligence-Enabled IIoT: A DRL Approach

  • Research
  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

The intelligent edge has accelerated the Internet of Things (IoT) revolution towards next-generation operational efficiency and massive connectivity. Supporting fast response, agility, and adaptive industrial IoT (IIoT) services, on the other hand, remains a challenge. In this paper, we investigate the dynamic service function chain (SFC) orchestration problem (i.e., SFC-DOP) in edge intelligence-empowered IIoT. By jointly considering the unique characteristics of IIoT service requests, i.e., specific delay constraints as well as the time-varying and heterogeneous natures of the IIoT, this optimization problem is modeled as a Markov decision process (MDP). The aforementioned MDP problem is then solved by the optimized soft actor-critic (SAC) deep reinforcement learning (DRL) method based on the maximum entropy framework. Simulation results demonstrate that compared to existing DRL-based methods (i.e., DDPG, TD3, and PPO), the optimized-SAC methods can achieve significant improvements in throughput and scalability with delay guarantee, and adapt to varying scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

Not applicable.

Notes

  1. https://github.com/RealVNF/DeepCoord.

References

  1. Aazam, M., Zeadally, S., Harras, K.A.: Deploying fog computing in industrial Internet of Things and industry 4.0. IEEE Trans. Ind. Inf. 14(10), 4674–4682 (2018)

    Article  Google Scholar 

  2. Chalapathi, G.S.S., Chamola, V., Vaish, A., Buyya, R.: Industrial Internet of Things (IIoT) applications of edge and fog computing: A review and future directions. Fog/Edge Comput. Secur. Privacy App. 293–325 (2021)

  3. Abbas, N., Zhang, Y., Taherkordi, A., Skeie, T.: Mobile edge computing: A survey. IEEE Internet Things J. 5(1), 450–465 (2017)

    Article  Google Scholar 

  4. Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., Zhang, J.: Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE 107(8), 1738–1762 (2019)

    Article  Google Scholar 

  5. Fu, X., Yu, F.R., Wang, J., Qi, Q., Liao, J.: Dynamic service function chain embedding for NFV-enabled IoT: A deep reinforcement learning approach. IEEE Trans. Wireless Commun. 19(1), 507–519 (2019)

    Article  Google Scholar 

  6. Cai, J., Huang, Z., Liao, L., Luo, J., Liu, W.-X.: APPM: Adaptive parallel processing mechanism for service function chains. IEEE Trans. Netw. Serv. Manage. 18(2), 1540–1555 (2021)

    Article  Google Scholar 

  7. Fu, F., Kang, Y., Zhang, Z., Yu, F.R., Wu, T.: Soft actor-critic DRL for live transcoding and streaming in vehicular fog-computing-enabled IOV. IEEE Internet Things J. 8(3), 1308–1321 (2020)

    Article  Google Scholar 

  8. Liu, Y., Lu, H., Li, X., Zhang, Y., **, L., Zhao, D.: Dynamic service function chain orchestration for NFV/MEC-enabled IoT networks: A deep reinforcement learning approach. IEEE Internet Things J. 8(9), 7450–7465 (2020)

    Article  Google Scholar 

  9. Xu, S., Li, Y., Guo, S., Lei, C., Liu, D., Qiu, X.: Cloud-edge collaborative SFC map** for industrial IoT using deep reinforcement learning. IEEE Trans. Industr. Inf. 18(6), 4158–4168 (2021)

    Article  Google Scholar 

  10. Chen, H., Wang, S., Li, G., Nie, L., Wang, X., Ning, Z.: Distributed orchestration of service function chains for edge intelligence in the industrial Internet of Things. IEEE Trans. Ind. Inf. 18(9), 6244–6254 (2021)

    Article  Google Scholar 

  11. Zhang, K., Zhu, Y., Maharjan, S., Zhang, Y.: Edge intelligence and blockchain empowered 5G beyond for the industrial Internet of Things. IEEE Netw. 33(5), 12–19 (2019)

    Article  Google Scholar 

  12. Hazra, A., Adhikari, M., Amgoth, T., Srirama, S.N.: Collaborative AI-enabled intelligent partial service provisioning in green industrial fog networks. IEEE Internet Things J. (2021)

  13. Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: A brief survey. IEEE Signal Process. Mag. 34(6), 26–38 (2017)

    Article  Google Scholar 

  14. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Proc. International Conference on Machine Learning, pp. 1861–1870 (2018). PMLR

  15. Sun, W., Liu, J., Yue, Y.: AI-enhanced offloading in edge computing: When machine learning meets industrial IoT. IEEE Netw. 33(5), 68–74 (2019)

    Article  Google Scholar 

  16. Amin, S.U., Hossain, M.S.: Edge intelligence and Internet of Things in healthcare: A survey. IEEE Access 9, 45–59 (2020)

    Article  Google Scholar 

  17. **a, W., Zhang, J., Quek, T.Q., **, S., Zhu, H.: Mobile edge cloud-based industrial Internet of Things: Improving edge intelligence with hierarchical SDN controllers. IEEE Veh. Technol. Mag. 15(1), 36–45 (2020)

    Article  Google Scholar 

  18. Bari, F., Chowdhury, S.R., Ahmed, R., Boutaba, R., Duarte, O.C.M.B.: Orchestrating virtualized network functions. IEEE Trans. Netw. Serv. Manage. 13(4), 725–739 (2016)

    Article  Google Scholar 

  19. Li, G., Zhou, H., Feng, B., Zhang, Y., Yu, S.: Efficient provision of service function chains in overlay networks using reinforcement learning. IEEE Trans. Cloud Comput. 10(1), 383–395 (2022)

    Article  Google Scholar 

  20. Guo, S., Dai, Y., Xu, S., Qiu, X., Qi, F.: Trusted cloud-edge network resource management: DRL-driven service function chain orchestration for IoT. IEEE Internet Things J. 7(7), 6010–6022 (2019)

    Article  Google Scholar 

  21. Schneider, S., Khalili, R., Manzoor, A., Qarawlus, H., Schellenberg, R., Karl, H., Hecker, A.: Self-learning multi-objective service coordination using deep reinforcement learning. IEEE Trans. Netw. Serv. Manage. 18(3), 3829–3842 (2021)

    Article  Google Scholar 

  22. Knight, S., Nguyen, H.X., Falkner, N., Bowden, R., Roughan, M.: The internet topology zoo. IEEE J. Sel. Areas Commun. 29(9), 1765–1775 (2011)

    Article  Google Scholar 

  23. Fischer, W., Meier-Hellstern, K.: The Markov-modulated Poisson process (MMPP) cookbook. Perform. Eval. 18(2), 149–171 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  24. Orlowski, S., Wessäly, R., Pióro, M., Tomaszewski, A.: SNDlib 1.0-survivable network design library. Netw.: Int. J 55(3), 276–286 (2010)

  25. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-baselines3: Reliable reinforcement learning implementations. J. Mach. Learn. Res. 268(22), 1–8 (2021)

    MATH  Google Scholar 

  26. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al.: Soft actor-critic algorithms and applications. ar**v preprint ar**v:1812.05905 (2018)

  27. Hieu, N.Q., Hoang, D.T., Niyato, D., Wang, P., Kim, D.I., Yuen, C.: Transferable deep reinforcement learning framework for autonomous vehicles with joint radar-data communications. IEEE Trans. Commun. 70(8), 5164–5180 (2022)

    Article  Google Scholar 

  28. Zhang, X., **, X., Tripp, C., Biagioni, D.J., Graf, P., Jiang, H.: Transferable reinforcement learning for smart homes. In: Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings & Cities, pp. 43–47 (2020)

Download references

Funding

This work was supported by the National Key R &D Program of China No. 2019YFB1804400, and MUST Faculty Research Grants No. FRG-21-031-IINGI.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Macau University of Science and Technology, Macao, 999078, China

    Zhongwei Huang & Weijian Zhong

  2. International Institute of Next Generation Internet, Macau University of Science and Technology, Macao, 999078, China

    Dagang Li

  3. Zhuhai-M.U.S.T. Science and Technology Research Institute, Zhuhai, 519031, China

    Dagang Li

  4. Guangdong Communication & Network Institute, Guangzhou, 510663, China

    Hua Lu

Authors
  1. Zhongwei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weijian Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dagang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hua Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZH: original draft writing and preparation, simulation implementation, and results analysis. ZH: Simulation & results analysis, Review & Editing. DL: methodology, review & editing manuscript, funding acquisition. HL: funding acquisition, supervision.

Corresponding author

Correspondence to Dagang Li.

Ethics declarations

Conflict of interest

The authors have no competing interests as defined by Springer.

Ethical Approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Z., Zhong, W., Li, D. et al. Delay Constrained SFC Orchestration for Edge Intelligence-Enabled IIoT: A DRL Approach. J Netw Syst Manage 31, 53 (2023). https://doi.org/10.1007/s10922-023-09743-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-023-09743-2

Keywords

Search

Navigation