SpringerLink
Log in

Extending parallel programming patterns with adaptability features

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Today, all computers have some degree of usable parallelism. Modern computers are explicitly equipped with hardware support for parallelism, such as multiple nodes, multicores, multiple CPUs, and accelerators. At the same time, the Cloud Continuum has become a viable platform for running parallel applications. Building software for these parallel and distributed platforms can be challenging due to the numerous considerations programmers must make during the development process. With this in mind, the high-performance computing literature proposed the concept of parallel patterns to hide some complexities. However, there are no patterns that address the design and creation of adaptive applications. Taking the compute continuum era in mind, we present how adaptability features can be explored within each parallel programming pattern, providing technical details on managing dynamic resources and handling changes in application behavior. In addition to this contribution, we also address practical implications by presenting some frameworks that can be used to implement adaptive applications and examples of using them with the proposed patterns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  1. Risco, S., Moltó, G., Naranjo, D.M., Blanquer, I.: Serverless workflows for containerised applications in the cloud continuum. J. Grid Comput. (2021). https://doi.org/10.1007/s10723-021-09570-2

    Article  Google Scholar 

  2. Gill, S.S., Wu, H., Patros, P., Ottaviani, C., Arora, P., Pujol, V.C., Haunschild, D., Parlikad, A.K., Cetinkaya, O., Lutfiyya, H., Stankovski, V., Li, R., Ding, Y., Qadir, J., Abraham, A., Ghosh, S.K., Song, H.H., Sakellariou, R., Rana, O., Rodrigues, J.J., Kanhere, S.S., Dustdar, S., Uhlig, S., Ramamohanarao, K., Buyya, R.: Modern computing: vision and challenges. Telemat. Inform. Rep. 13, 100116 (2024)

    Article  Google Scholar 

  3. Martín-Álvarez, I., Aliaga, J.I., Castillo, M., Iserte, S., Mayo, R.: Dynamic spawning of mpi processes applied to malleability. Int. J. High Perform. Comput. Appl. (2023). https://doi.org/10.1177/10943420231176527

    Article  Google Scholar 

  4. Sena, A. C., Ribeiro, F. S., Rebello, V. E., Nascimento, A. P., Boeres, C.: Autonomic malleability in iterative mpi applications. In: 2013 25th International Symposium on Computer Architecture and High Performance Computing. pp. 192–199 (2013)

  5. Kehrer, S., Blochinger, W.: Migrating parallel applications to the cloud: assessing cloud readiness based on parallel design decisions. SICS Softw.-Intensive Cyber-Phys. Syst. 34(2–3), 73–84 (2019). https://doi.org/10.1007/s00450-019-00396-8

    Article  Google Scholar 

  6. Wan, L., Cui, X., Li, Y., Zheng, W., Yuan, X.: Heteropp: a directive-based heterogeneous cooperative parallel programming framework. Concurr. Comput.: Pract. Exp. (2024). https://doi.org/10.1002/cpe.8014

    Article  Google Scholar 

  7. Galante, G., da Rosa Righi, R.: Adaptive parallel applications: from shared memory architectures to fog computing (2002–2022). Clust. Comput. 25(6), 4439–4461 (2022). https://doi.org/10.1007/s10586-022-03692-2

    Article  Google Scholar 

  8. Galante, G., da Rosa Righi, R.: Exploring cloud elasticity in scientific applications. In: Antonopoulos, N., Gillam, L. (eds.) Cloud computing - principles, systems and applications, second edition, computer communications and networks, pp. 101–125. Cham, Springer (2017)

    Google Scholar 

  9. Czappa, F., Calotoiu, A., Höhl, T., Mantel, H., Nguyen, T., Wolf, F.: Design-time performance modeling of compositional parallel programs. Parallel Comput. 108, 102839 (2021). https://doi.org/10.1016/j.parco.2021.102839

    Article  MathSciNet  Google Scholar 

  10. Mattson, T., Sanders, B., Massingill, B.: Patterns for parallel programming, 1st edn. Addison-Wesley Professional, Boston (2004)

    Google Scholar 

  11. McCool, M.D., Robison, A.D., Reinders, J.: Structured parallel programming patterns for efficient computation. Elsevier, Waltham (2012)

    Google Scholar 

  12. Dooley, J.F.: Parallel Design Patterns, Apress, Berkeley, CA. Ch. vol. 6, pp. 191–209 (2017).

  13. Danelutto, M., Mencagli, G., Torquati, M., González-Vélez, H., Kilpatrick, P.: Algorithmic skeletons and parallel design patterns in mainstream parallel programming. Int. J. Parallel Program. 49(2), 177–198 (2021). https://doi.org/10.1007/s10766-020-00684-w

    Article  Google Scholar 

  14. Keutzer, K., Mattson, T.: Our pattern language,  (2023)https://patterns.eecs.berkeley.edu/ Accessed June 2023

  15. Gamma, E., Helm, R., Johnson, R., Vlissides, J.M.: Design patterns: elements of reusable object-oriented software. Addison-Wesley Professional, Boston (1994)

    Google Scholar 

  16. Rinaldi, L., Torquati, M., Mencagli, G., Danelutto, M., Menga, T.: Accelerating actor-based applications with parallel patterns. In: 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2019, IEEE, pp. 140–147 (2019) https://doi.org/10.1109/EMPDP.2019.8671602

  17. Wilder, B.: Cloud Architecture Patterns. O’Reilly Media Inc, Sebastopol (2012)

    Google Scholar 

  18. Erl, T., Cope, R., Naserpour, A.: Cloud computing design patterns, 1st edn. Prentice Hall Press, Hoboken (2015)

    Google Scholar 

  19. Abuseta, Y., Swesi, K.: Design patterns for self adaptive systems engineering. Int. J. Softw. Eng. Appl. 6(4), 11–28 (2015). https://doi.org/10.5121/ijsea.2015.6402

    Article  Google Scholar 

  20. Kehrer, S., Blochinger, W.: Elastic parallel systems for high performance cloud computing: State-of-the-art and future directions. Parallel Processing Letters 29(02), 1950006 (2019). https://doi.org/10.1142/s0129626419500063

    Article  MathSciNet  Google Scholar 

  21. Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., Merle, P.: Elasticity in cloud computing: state of the art and research challenges. IEEE Trans. Serv. Comput. 11(2), 430–447 (2018). https://doi.org/10.1109/tsc.2017.2711009

    Article  Google Scholar 

  22. Straesser, M., Grohmann, J., von Kistowski, J., Eismann, S., Bauer, A., Kounev, S.: Why is it not solved yet?: Challenges for production-ready autoscaling. In: Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering, ICPE ’22, ACM, (2022) https://doi.org/10.1145/3489525.3511680

  23. Asanovic, K., Bodik, R., Demmel, J., Keaveny, T., Keutzer, K., Kubiatowicz, J., Morgan, N., Patterson, D., Sen, K., Wawrzynek, J., Wessel, D., Yelick, K.: A view of the parallel computing landscape. Commun. ACM 52(10), 56–67 (2009). https://doi.org/10.1145/1562764.1562783

    Article  Google Scholar 

  24. Martín, G., Singh, D.E., Marinescu, M.-C., Carretero, J.: Enhancing the performance of malleable mpi applications by using performance-aware dynamic reconfiguration. Parallel Comput. 46, 60–77 (2015). https://doi.org/10.1016/j.parco.2015.04.003

    Article  Google Scholar 

  25. Aldinucci, M., Campa, S., Danelutto, M., Kilpatrick, P., Torquati, M.: Design patterns percolating to parallel programming framework implementation. Int. J. Parallel Program. 42(6), 1012–1031 (2014). https://doi.org/10.1007/s10766-013-0273-6

    Article  Google Scholar 

  26. Sterling, T., Anderson, M., Brodowicz, M.: High Performance Computing: Modern Systems and Practices, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2017)

    Google Scholar 

  27. Senger, H., da Silva, F.A.B.: Bounds on the scalability of bag-of-tasks applications running on master-slave platforms. Parallel Processing Letters 22(02), 1250004 (2012). https://doi.org/10.1142/S0129626412500041

    Article  MathSciNet  Google Scholar 

  28. Danelutto, M., De Matteis, T., Mencagli, G., Torquati, M.: A divide-and-conquer parallel pattern implementation for multicores, in: Proceedings of the 3rd International Workshop on Software Engineering for Parallel Systems, SEPS 2016, Association for Computing Machinery, New York, NY, USA, 2016, p. 10-19. https://doi.org/10.1145/3002125.3002128

  29. Ghit, B., Yigitbasi, N., Iosup, A., Epema, D.: Balanced resource allocations across multiple dynamic mapreduce clusters. SIGMETRICS Perform. Eval. Rev. 42(1), 329–341 (2014). https://doi.org/10.1145/2637364.2591998

    Article  Google Scholar 

  30. Czarnul, P.: Parallel programming for modern high performance computing systems. CRC Press, Boca Raton (2018)

    Book  Google Scholar 

  31. Rinaldi, L., Torquati, M., Sensi, D.D., Mencagli, G., Danelutto, M.: Improving the performance of actors on multi-cores with parallel patterns. Int. J. Parallel Program. 48(4), 692–712 (2020). https://doi.org/10.1007/s10766-020-00663-1

    Article  Google Scholar 

  32. Tasharofi, S., Dinges, P., Johnson, R. E.: Why do scala developers mix the actor model with other concurrency models?, in: G. Castagna (Ed.), ECOOP 2013 - Object-Oriented Programming - 27th European Conference, Montpellier, France, July 1-5, 2013. Proceedings, Vol. 7920 of Lecture Notes in Computer Science, Springer, 2013, pp. 302–326. https://doi.org/10.1007/978-3-642-39038-8_13

  33. Aubanel, E.: Elements of parallel computing, 1st edn. Chapman & Hall/CRC, Boca Raton (2016)

    Book  Google Scholar 

  34. Prasad, S.K., Gupta, A., Rosenberg, A.L., Sussman, A., Weems, C.C.: Topics in parallel and distributed computing: introducing concurrency in undergraduate courses, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2015)

    Google Scholar 

  35. Aliaga, J.I., Castillo, M., Iserte, S., Martín-Álvarez, I., Mayo, R.: A survey on malleability solutions for high-performance distributed computing. Appl. Sci. (2022). https://doi.org/10.3390/app12105231

    Article  Google Scholar 

  36. Baldo, L., Brenner, L., Fernandes, L. G., Fernandes, P., Sales, A.: Performance models for master/slave parallel programs, Electronic Notes in Theoretical Computer Science 128 (4), 101–121 (2005) proceedings of the First International Workshop on Practical Applications of Stochastic Modelling (PASM 2004). https://doi.org/10.1016/j.entcs.2005.01.015

  37. Wang, X. W., Zhang, J.. Liao, H. M., Zha, L. Dynamic split model of resource utilization in mapreduce, in: Proceedings of the Second International Workshop on Data Intensive Computing in the Clouds, DataCloud-SC ’11, Association for Computing Machinery, New York, NY, USA, 2011, p. 21-30. https://doi.org/10.1145/2087522.2087528

  38. Liu, Z., Zhang, Q., Zhani, M.F., Boutaba, R., Liu, Y., Gong, Z.: Dreams: Dynamic resource allocation for mapreduce with data skew, in. IFIP/IEEE International Symposium on Integrated Network Management (IM) 2015, 18–26 (2015). https://doi.org/10.1109/INM.2015.7140272

    Article  Google Scholar 

  39. Moreno, A., Sikora, A., César, E., Sorribes, J., Margalef, T.: Hedpm: Load balancing of linear pipeline applications on heterogeneous systems. J. Supercomput. 73(9), 3738–3760 (2017). https://doi.org/10.1007/s11227-017-1971-4

    Article  Google Scholar 

  40. Francesquini, E., Goldman, A., Méhaut, J.-F.: Improving the performance of actor model runtime environments on multicore and manycore platforms. In: Proceedings of the 2013 Workshop on Programming Based on Actors, Agents, and Decentralized Control, AGERE! 2013, Association for Computing Machinery, pp. 109-114. New York, NY, USA (2013)  https://doi.org/10.1145/2541329.2541342

  41. Mo-Hellenbrand, A.: Resource-aware and elastic parallel software development for distributed-memory hpc systems, Ph.D. thesis, Technischen Universität München (2019). https://mediatum.ub.tum.de/doc/1471007/1471007.pdf

  42. Rajan, D., Thrasher, A., Abdul-Wahid, B., Izaguirre, J.A., Emrich, S., Thain, D., Case studies in designing elastic applications. In: 13th IEEE/ACM International Symposium on Cluster. Cloud, and Grid Computing vol. 2013, pp. 466–473 (2013). https://doi.org/10.1109/CCGrid.2013.46

  43. da Rosa Righi, R., Rodrigues, V.F., da Costa, C.A., Galante, G., Bona, L.C.E.D., Ferreto, T.C.: Autoelastic: automatic resource elasticity for high performance applications in the cloud. IEEE Trans. Cloud Comput. 4(1), 6–19 (2016)

    Article  Google Scholar 

  44. Houzeaux, G., Badia, R., Borrell, R., Dosimont, D., Ejarque, J., Garcia-Gasulla, M., López, V.: Dynamic resource allocation for efficient parallel cfd simulations. Comput. Fluids 245, 105577 (2022). https://doi.org/10.1016/j.compfluid.2022.105577

    Article  MathSciNet  Google Scholar 

  45. Munhoz, V., Castro, M., Mendizabal, O.: Strategies for fault-tolerant tightly-coupled hpc workloads running on low-budget spot cloud infrastructures. In: 2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD). pp. 263–272 (2022). https://doi.org/10.1109/SBAC-PAD55451.2022.00037

  46. Galante, G., Bona, L. C. E.: Supporting elasticity in openmp applications. In: Proceedings of the 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP ’14, IEEE Computer Society.  pp. 188-195. USA (2014).  https://doi.org/10.1109/PDP.2014.36

  47. Zhao, J., Gao, X., Li, Y.: Research on elastic extension of multi type resources for openmp program. In: 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). pp. 971–978. (2022).

  48. Wrzesinska, G., van Nieuwpoort, R., Maassen, J., Bal, H.: Fault-tolerance, malleability and migration for divide-and-conquer applications on the grid. In: 19th IEEE International Parallel and Distributed Processing Symposium, pp. 10-pp. (2005). https://doi.org/10.1109/IPDPS.2005.224

  49. Aubin, M. R., da Rosa Righi, R., Valiati, V. H., da Costa, C. A., Antunes, R. S., Galante, G.: Helastic: On combining threshold-based and serverless elasticity approaches for optimizing the execution of bioinformatics applications. J.  Comput. Sci. vol. 53,  p. 101407. (2021). https://doi.org/10.1016/j.jocs.2021.101407

  50. Fadika, Z., Govindaraju, M., Delma: Dynamically elastic mapreduce framework for cpu-intensive applications. In: 11th IEEE/ACM International Symposium on Cluster. Cloud and Grid Computing 2011. pp. 454–463 (2011). https://doi.org/10.1109/CCGrid.2011.71

  51. Gordon, A. W., Lu, P.: Elastic phoenix: Malleable mapreduce for shared-memory systems In: E. R. Altman, W. Shi (eds.), Network and Parallel Computing - 8th IFIP International Conference, NPC 2011, Vol. 6985 of Lecture Notes in Computer Science, pp. 1–16. Springer (2011). https://doi.org/10.1007/978-3-642-24403-2_1

  52. Chalvantzis, N., Konstantinou, I., Kozyris, N., Bbq: Elastic mapreduce over cloud platforms. In: 17th IEEE/ACM International Symposium on Cluster. Cloud and Grid Computing (CCGRID) 2017, pp. 766–771 (2017). https://doi.org/10.1109/CCGRID.2017.140

  53. Meyer, V., Rodrigues, V.F., Righi, R.D.R., Costa, C.A.D., Galante, G., Both, C.B.: Pipel: exploiting resource reorganisation to optimise performance of pipeline-structured applications in the cloud. Int. J. Comput. Syst. Eng. (2019). https://doi.org/10.1504/ijcsyse.2019.10015444

    Article  Google Scholar 

  54. Vogel, A., Griebler, D., De Sensi, D., Danelutto, M., Fernandes, L. G.: Autonomic and latency-aware degree of parallelism management in spar, in: G. Mencagli, D. B. Heras, V. Cardellini, E. Casalicchio, E. Jeannot, F. Wolf, A. Salis, C. Schifanella, R. R. Manumachu, L. Ricci, M. Beccuti, L. Antonelli, J. D. Garcia Sanchez, S. L. Scott (eds.), Euro-Par 2018: Parallel Processing Workshops, pp. 28–39 Springer International Publishing, Cham (2019)

  55. Sang, B., Roman, P.-L., Eugster, P., Lu, H., Ravi, S., Petri, G.: Plasma: Programmable elasticity for stateful cloud computing applications. In: Proceedings of the Fifteenth European Conference on Computer Systems, EuroSys ’20, Association for Computing Machinery, New York, NY, USA,  pp. 1–15. (2020). https://doi.org/10.1145/3342195.3387553

  56. Galante, G., Bona, L.C.E.D.: A programming-level approach for elasticizing parallel scientific applications. J. Syst. Softw. 110, 239–252 (2015). https://doi.org/10.1016/j.jss.2015.08.051

    Article  Google Scholar 

  57. De Sensi, D., De Matteis, T., Danelutto, M.: Simplifying self-adaptive and power-aware computing with nornir. Future Gener. Comput. Syst. 87, 136–151 (2018)

    Article  Google Scholar 

  58. Mishra, N., Imes, C., Lafferty, J. D., Hoffmann, H.: Caloree: Learning control for predictable latency and low energy. In: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’18, Association for Computing Machinery, New York, NY, USA, pp. 184-198. (2018). https://doi.org/10.1145/3173162.3173184

  59. Pervaiz, A., Yang, Y. H., Duracz, A., Bartha, F., Sai, R., Imes, C., Cartwright, R., Palem, K., Lu, S., Hoffmann, H.: Goal: Supporting general and dynamic adaptation in computing systems. In: Proceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2022, Association for Computing Machinery, New York, NY, USA, pp. 16-32. (2022). https://doi.org/10.1145/3563835.3567655

  60. Rubio, J., Bilbao, C., Saez, J. C., Prieto-Matias, M.: Exploiting elasticity via os-runtime cooperation to improve cpu utilization in multicore systems. In: 2024 32nd Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP),  pp. 35–43. (2024). https://doi.org/10.1109/PDP62718.2024.00014

  61. Eberius, D., Rahman, M. W.-U., Ozog, D.: Evaluating the potential of elastic jobs in hpc systems, in: Proceedings of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, SC-W ’23, Association for Computing Machinery, New York, NY, USA, p. 1324-1333. (2023). https://doi.org/10.1145/3624062.3624199

  62. Galante, G., Bona, L. C. E. D., Schepke, C.: Improving OLAM with cloud elasticity. In: Computational Science and Its Applications – ICCSA 2014, Springer, pp. 46–60. (2014). https://doi.org/10.1007/978-3-319-09153-2_4

  63. Moretti, C., Thrasher, A., Yu, L., Olson, M., Emrich, S., Thain, D.: A framework for scalable genome assembly on clusters, clouds, and grids. IEEE Trans. Parallel Distrib. Syst. 23(12), 2189–2197 (2012). https://doi.org/10.1109/tpds.2012.80

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank to the following Brazilian agencies: FAPERGS (process 23/2551-0002202-8) and CNPq (process 305263/2021-8).

Funding

The authors have not disclosed any funding.

Author information

Author notes

  1. Guilherme Galante, Rodrigo da Rosa Righi and Cristiane Andrade have contributed equally to this work.

Authors and Affiliations

  1. PPGComp, Unioeste, Cascavel, Paraná, Brazil

    Guilherme Galante & Cristiane de Andrade

  2. PPGCA, Unisinos, São Leopoldo, Rio Grande do Sul, Brazil

    Rodrigo da Rosa Righi

Authors
  1. Guilherme Galante
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodrigo da Rosa Righi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cristiane de Andrade
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guilherme Galante.

Ethics declarations

Competing interests

The authors have not disclosed any competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Galante, G., da Rosa Righi, R. & de Andrade, C. Extending parallel programming patterns with adaptability features. Cluster Comput (2024). https://doi.org/10.1007/s10586-024-04622-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-024-04622-0

Keywords

Search

Navigation