SpringerLink
Log in

Mutation-driven and population grou** PRO algorithm for scheduling budget-constrained workflows in the cloud

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Benefiting from cloud computing’s elasticity, scalability, and pay-per-use model, more and more scientific applications are deployed in or migrated to the cloud. Workflow scheduling still faces many challenges due to the growing scales of workflows and the diversified user QoS requirements. In this work, we propose a Mutation-driven and population Grou** Poor and Rich Optimization algorithm (MG-PRO) for scheduling workflows in the cloud to minimize makespan while satisfying the budget constraints. Specifically, we first adopt the middle-class sub-population into the original Poor and Rich Optimization algorithm (PRO), and develop the update strategies for rich and middle-class sub-populations to increase the randomness and search diversity. Secondly, the update mechanism for rich individuals is enriched, and the middle-class sub-population is guided by elite rich individuals, which enhances the information exchange and sharing among sub-populations. Finally, an evolution-aware mutation strategy is designed, where the mutation probability is adjusted adaptively as the dynamic monitoring of the population update process, and the two-point and triangular crossover-based mutations are used alternately to intervene the evolution trajectory according to the degree of objective optimization, resulting in a better balance between exploration and exploration. Extensive experiments are conducted on well-known scientific workflows with different types and scales through WorkflowSim. The experimental results show that, in most cases, MG-PRO outperforms existing algorithms in terms of constraint satisfiability, solution quality and stability. It can generate near-optimal solutions with the different budget constraints satisfied in a relatively short time, for example, the makespan resulting from MG-PRO is at most 59.95% shorter than other meta-heuristic algorithms, and at least 7.33% shorter than all its peers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

The data used to support the findings of this study is available from the corresponding author upon request.

References

  1. Marozzo, F., Talia, D., Trunfio, P.: A workflow management system for scalable data mining on clouds. IEEE Trans. Serv. Comput. 11(3), 480–492 (2016)

    Article  Google Scholar 

  2. Bokhari, M.U., Makki, Q., Tamandani, Y.K.: A survey on cloud computing Big Data Analytics, pp. 149–164. Springer, Germany (2018)

    Google Scholar 

  3. Li, H., Wang, D., Cañizares Abreu, J.R., Zhao, Q., Bonilla Pineda, O.: PSO+LOA:hybrid constrained optimization for scheduling scientific workflows in the cloud. J. Supercomput. 77(11), 13139–13165 (2021)

    Article  Google Scholar 

  4. Zhang, F., Cao, J., Hwang, K., Li, K., Khan, S.U.: Adaptive workflow scheduling on cloud computing platforms with iterativeordinal optimization. IEEE Trans. Cloud Comput. 3(2), 156–168 (2015)

    Article  Google Scholar 

  5. Moosavi, S.H.S., Bardsiri, V.K.: Poor and rich optimization algorithm: A new human-based and multi populations algorithm. Eng. Appl. Artif. Intell. 86, 165–181 (2019)

    Article  Google Scholar 

  6. Rodriguez, M.A., Buyya, R.: Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)

    Article  Google Scholar 

  7. Mohammadzadeh, A., Masdari, M., Gharehchopogh, F.S., Jafarian, A.: Improved chaotic binary grey wolf optimization algorithm for workflow scheduling in green cloud computing. Evol. Intell. 14(4), 1997–2025 (2021)

    Article  Google Scholar 

  8. Etminani, K., Naghibzadeh, M.: A Min-Min Max-Min selective algorihtm for grid task scheduling. In: 2007 3rd IEEE/IFIP International Conference in Central Asia on Internet, pp. 1–7. IEEE (2007)

  9. Ilavarasan, E., Thambidurai, P.: Low complexity performance effective task scheduling algorithm for heterogeneous computing environments. J. Comput. Sci. 3(2), 94–103 (2007)

    Article  Google Scholar 

  10. Wu, Q., Zhou, M., Wen, J.: Endpoint communication contention-aware cloud workflow scheduling. IEEE Trans. Autom. Sci. Eng. 19(2), 1137–1150 (2022)

    Article  Google Scholar 

  11. Sih, G.C., Lee, E.A.: A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures. IEEE Trans. Parallel Distrib. Syst. 4(2), 175–187 (1993)

    Article  Google Scholar 

  12. Kwok, Y.K., Ahmad, I.: Dynamic critical-path scheduling: An effective technique for allocating task graphs to multiprocessors. IEEE Trans. Parallel Distrib. Syst. 7(5), 506–521 (1996)

    Article  Google Scholar 

  13. Faragardi, H.R., Sedghpour, M.R.S., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: A budget-constrained resource provisioning scheme for workflow scheduling in iaas clouds. IEEE Trans. Parallel Distrib. Syst. 31(6), 1239–1254 (2019)

    Article  Google Scholar 

  14. Arabnejad, V., Bubendorfer, K., Ng, B.: Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1), 29–44 (2018)

    Article  Google Scholar 

  15. Rizvi, N., Ramesh, D.: Design of a scheduling approach for budget-deadline constrained applications in heterogeneous clouds. In: International Conference on Distributed Computing and Internet Technology, pp. 198–213. Springer (2020)

  16. Hu, B., Cao, Z., Zhou, M.: Scheduling real-time parallel applications in cloud to minimize energy consumption. IEEE Trans. Cloud Comput. 10(1), 662–674 (2022)

    Article  Google Scholar 

  17. Kalyan Chakravarthi, K., Shyamala, L., Vaidehi, V.: Budget aware scheduling algorithm for workflow applications in IaaS clouds. Clust. Comput. 23(4), 3405–3419 (2020)

    Article  Google Scholar 

  18. Li, W., **a, Y., Zhou, M., Sun, X., Zhu, Q.: Fluctuation-aware and predictive workflow scheduling in cost-effective infrastructure-as-a-service clouds. IEEE Access 6, 61488–61502 (2018)

    Article  Google Scholar 

  19. Rocha, A.M.A., Costa, M.F.P., Fernandes, E.M.: A shifted hyperbolic augmented lagrangian-based artificial fish two-swarm algorithm with guaranteed convergence for constrained global optimization. Eng. Optim. 48(12), 2114–2140 (2016)

    Article  MathSciNet  Google Scholar 

  20. Tang, J., Liu, G., Pan, Q.: A review on representative swarm intelligence algorithms for solving optimization problems: Applications and trends. IEEE/CAA J. Autom. Sinica 8(10), 1627–1643 (2021)

    Article  MathSciNet  Google Scholar 

  21. Li, H., Wang, D., Zhou, M., Fan, Y., **a, Y.: Multi-swarm co-evolution based hybrid intelligent optimization for bi-objective multi-workflow scheduling in the cloud. IEEE Trans. Parallel Distrib. Syst. 33(9), 2183–2197 (2022)

    Article  Google Scholar 

  22. Rashedi, E., Nezamabadi-pour, H., Saryazdi, S.: Gsa: A gravitational search algorithm. Inf. Sci. 179(13), 2232–2248 (2009)

    Article  Google Scholar 

  23. Chen, Z.G., Zhan, Z.H., Lin, Y., Gong, Y.J., Gu, T.L., Zhao, F., Yuan, H.Q., Chen, X., Li, Q., Zhang, J.: Multiobjective cloud workflow scheduling: A multiple populations ant colony system approach. IEEE Trans. Cybern. 49(8), 2912–2926 (2018)

    Article  PubMed  Google Scholar 

  24. Wang, Y., Gao, S., Zhou, M., Yu, Y.: A multi-layered gravitational search algorithm for function optimization and real-world problems. IEEE/CAA J. Autom. Sinica 8(1), 94–109 (2021)

    Article  CAS  Google Scholar 

  25. Bi, J., Yuan, H., Duanmu, S., Zhou, M., Abusorrah, A.: Energy-optimized partial computation offloading in mobile-edge computing with genetic simulated-annealing-based particle swarm optimization. IEEE Internet Things J. 8(5), 3774–3785 (2021)

    Article  Google Scholar 

  26. Bi, J., Yuan, H., Zhai, J., Zhou, M., Poor, H.: Self-adaptive bat algorithm with genetic operations. IEEE/CAA J. Autom. Sinica 9(7), 1284–1294 (2022)

    Article  Google Scholar 

  27. Ebadifard, F., Doostali, S., Babamir, S.M.: A firefly-based task scheduling algorithm for the cloud computing environment: formal verification and simulation analyses. 2018 9th International Symposium on Telecommunications (IST) pp. 664–669 (2018)

  28. Thennarasu, S.R., Selvam, M., Srihari, K.: A new whale optimizer for workflow scheduling in cloud computing environment. J. Amb. Intell. Humaniz. Comput. 12(3), 3807–3814 (2021)

    Article  Google Scholar 

  29. Wang, Y., Zuo, X.: An effective cloud workflow scheduling approach combining PSO and idle time slot-aware rules. IEEE/CAA J. Autom. Sinica 8(5), 1079–1094 (2021)

    Article  Google Scholar 

  30. Li, H., Wang, D., Xu, G., Yuan, Y., **a, Y.: Improved swarm search algorithm for scheduling budget-constrained workflows in the cloud. Soft Comput. 26(8), 3809–3824 (2022)

    Article  Google Scholar 

  31. Aziza, H., Krichen, S.: A hybrid genetic algorithm for scientific workflow scheduling in cloud environment. Neural Comput. Appl. 32(18), 15263–15278 (2020)

    Article  Google Scholar 

  32. Yang, C., Zhang, X.: Workflow tasks scheduling optimization based on genetic algorithm in clouds. In: 2018 IEEE 3rd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), pp. 6–10. IEEE (2018)

  33. Wu, Q., Zhou, M., Zhu, Q., **a, Y., Wen, J.: Moels: Multiobjective evolutionary list scheduling for cloud workflows. IEEE Trans. Autom. Sci. Eng. 17(1), 166–176 (2020)

    Article  Google Scholar 

  34. Li, H., Wang, B., Yuan, Y., Zhou, M., Fan, Y., **a, Y.: Scoring and dynamic hierarchy-based NSGA-II for multiobjective workflow scheduling in the cloud. IEEE Trans. Autom. Sci. Eng. 19(2), 982–993 (2022)

    Article  Google Scholar 

  35. Li, H.H., Chen, Z.G., Zhan, Z.H., Du, K.J., Zhang, J.: Renumber coevolutionary multiswarm particle swarm optimization for multi-objective workflow scheduling on cloud computing environment. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1419–1420. Association for Computing Machinery (2015)

  36. Qiu, J., Chen, Y., Tian, Z., Guizani, N., Du, X.: The security of internet of vehicles network: Adversarial examples for trajectory mode detection. IEEE Network 35(5), 279–283 (2021)

    Article  Google Scholar 

  37. Qiu, J., Du, L., Chen, Y., Tian, Z., Du, X., Guizani, M.: Nei-tte: Intelligent traffic time estimation based on fine-grained time derivation of road segments for smart city. IEEE Trans. Ind. Inf. 16(4), 2659–2666 (2020)

    Article  Google Scholar 

  38. Qiu, J., Du, L., Zhang, D., Su, S., Tian, Z.: Artificial intelligence security in 5g networks: Adversarial examples for estimating a travel time task. IEEE Veh. Technol. Mag. 15(3), 95–100 (2020)

    Article  Google Scholar 

  39. Kaur, A., Singh, P., Batth, R.S., Lim, C.P.: Deep-q learning-based heterogeneous earliest finish time scheduling algorithm for scientific workflows in cloud. Softw.: Prac. Exp. 52, 689–709 (2020)

    Google Scholar 

  40. Tong, Z., Chen, H., Deng, X., Li, K., Li, K.: A scheduling scheme in the cloud computing environment using deep Q-learning. Inf. Sci. 512, 1170–1191 (2020)

    Article  Google Scholar 

  41. Li, H., Huang, J., Wang, B., Fan, Y.: Weighted double deep Q-network based reinforcement learning for bi-objective multi-workflow scheduling in the cloud. Clust. Comput. 25(2), 751–768 (2022)

    Article  Google Scholar 

  42. Arabnejad, H., Barbosa, J.G., Prodan, R.: Low-time complexity budget-deadline constrained workflow scheduling on heterogeneous resources. Future Gener. Comput. Syst. 55, 29–40 (2016)

    Article  Google Scholar 

  43. Rizvi, N., Dharavath, R., Edla, D.R.: Cost and makespan aware workflow scheduling in iaas clouds using hybrid spider monkey optimization. Simul. Model. Prac. Theory 110, 102328 (2021)

    Article  Google Scholar 

  44. Elsayed, S.M., Sarker, R.A., Essam, D.L.: Multi-operator based evolutionary algorithms for solving constrained optimization problems. Comput. Oper. Res. 38(12), 1877–1896 (2011)

    Article  MathSciNet  Google Scholar 

  45. Chen, W., Deelman, E.: Workflowsim: A toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)

  46. Sahni, J., Vidyarthi, D.P.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2015)

    Article  Google Scholar 

  47. Nzanywayingoma, F., Yang, Y.: Analysis of particle swarm optimization and genetic algorithm based on task scheduling in cloud computing environment. Int. J. Adv. Comput. Sci. Appl. 8(1), 19–25 (2017)

    Google Scholar 

  48. Zhou, Y., Huang, X.: Scheduling workflow in cloud computing based on ant colony optimization algorithm. In: 2013 Sixth International Conference on Business Intelligence and Financial Engineering, pp. 57–61. IEEE (2013)

Download references

Funding

This work is supported in part by the National Natural Science Foundation of China under Grant No.61836001; and in part by the National Key Research and Development Program of China under Grant No.2018YFB1003700.

Author information

Authors and Affiliations

  1. State Key Laboratory of Intelligent Control and Decision of Complex Systems, Bei**g Institute of Technology, Bei**g, 100081, China

    Huifang Li, Bing Chen, **gwei Huang, Senchun Chai & Yuanqing **a

  2. Development Department, Integral Automation Company (CEDAI), Villa Clara, Cuba

    Julio Ruben Cañizares Abreu

Authors
  1. Huifang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. **gwei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julio Ruben Cañizares Abreu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Senchun Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuanqing **a
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to this work from different aspects. Material preparation, data collection, validation and results analysis were performed by HL, BC, JH, JRCA, SC and YX. The original draft of this manuscript was written by HL and BC, and all authors commented on previous versions of this manuscript, and then read and approved the final version. HL: Conceptualization, Methodology, Formal Modeling, Supervision, Writing-Reviewing and Editing. BC and JRCA: Methodology, Software, Data curation, Validation, Visualization, Writing original draft. JH: Formal analysis. SC and YX: Resources.

Corresponding author

Correspondence to Huifang Li.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Chen, B., Huang, J. et al. Mutation-driven and population grou** PRO algorithm for scheduling budget-constrained workflows in the cloud. Cluster Comput 27, 1137–1158 (2024). https://doi.org/10.1007/s10586-023-04006-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-023-04006-w

Keywords

Search

Navigation