Abstract
Multicore systems present on-board memory hierarchies and communication networks that influence performance when executing shared-memory parallel codes. Characterising this influence is complex, and understanding the effect of particular hardware configurations on different codes is of paramount importance. In this context, precise monitoring information can be extracted from hardware counters (HC) at runtime to characterise the behaviour of each thread of a parallel code. This technology provides high accuracy with a low overhead. In particular, we introduce a new tool to get this information from hardware counters in terms of number of floating point operations per second, operational intensity, latency of memory access, and energy consumption. Note the first two parameters define the well-known Roofline Model, an intuitive visual performance model used to provide performance estimates of applications running on multi-core architectures. The third parameter quantifies data locality and the fourth one is related to the load of each node of the system. All this information is accessed through the perf\(\_\)events interface provided by Linux, with the aid of the libpfm library. This tool can be used to utilise its monitoring information to optimise execution efficiency in NUMA systems by balancing or scheduling the workloads, guiding thread and page migration strategies in order to increase locality and affinity. The designated migrations are based on optimisation strategies, supported by runtime information provided by hardware counters. Overall, the profiling application is launched from a terminal as a background process, it does not require superuser permissions to run properly, and can lead to performance optimization in multithreaded applications and power saving in NUMA systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Hennessy, J.L., Patterson, D.A.: Computer Architecture: A Quantitative Approach, 4th edn. Morgan Kaufmann Publishers Inc., San Francisco (2006)
Mosberger, D., Eranian, S.: IA-64 Linux Kernel: Design and Implementation. Prentice Hall PTR, Upper Saddle River (2001)
Precise Event-Based Sampling (PEBS). http://perfmon2.sourceforge.net/pfmon_intel_core.html#pebs
Ashby, S., et al.: The opportunities and challenges of exascale computing. Summary Report of the Advanced Scientific Computing Advisory Committee (ASCAC) (2010)
Wlotzka, M., Heuveline, V., Dolz, M.F., Reza Heidari, M., Ludwig, T., Cristiano, A., Malossi, I., Quintana-Ortí, E.S.: Energy-aware high performance computing. IntechOpen (2017)
Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52(4), 65–76 (2009)
Lorenzo, O.G., Pena, José Carlos Cabaleiro, T.F., Pichel, J.C., Rivera, F.F.: 3DyRM: a dynamic roofline model including memory latency information. J. Supercomput. 70(2), 696–708 (2014)
Shende, S.S., Malony, A.D.: The Tau parallel performance system. Int. J. High Perfor. Comput. Appl. 20(2), 287–311 (2006)
Dyninst Project, Dyninst. http://dyninst.org/
HP, HP Caliper. https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c02919899 (2013)
Intel®VTune\(\text{TM}\) Amplifi | Intel®Software. http://software.intel.com/en-us/vtune
40 Years of Microprocessor Trend Data | Karl Rupp. http://www.karlrupp.net/2015/06/40-years-of-microprocessor-trend-data/
Martínez, D.R., Blanco, V., Cabaleiro, J.C., Pena, T.F., Rivera, F.F.: Modeling the performance of parallel applications using model selection techniques. Practice and Experience, Concurrency and Computation (2013)
Wu, X.: Performance, Evaluation. Prediction and Visualization of Parallel Systems. Kluwer Academic Publishers, Amsterdam (1999)
Valerie, T., Wu, X., Rick, S.: Prophesy: an infrastructure for performance analysis and modeling of parallel and grid applications. ACM SIGMETRICS Perform. Eval. Rev. 30(4), 13–18 (2003)
Choi, J.W., Bedard, D., Fowler, R., Vuduc, R.: A roofline model of energy. In: 2013 IEEE 27th International Symposium on Parallel and Distributed Processing, pp. 661–672 (2013)
Ilic, A., Pratas, F., Sousa, L.: Cache-aware roofline model: upgrading the loft. IEEE Comput. Archit. Lett. 13(1), 21–24 (2014)
Lorenzo, Ó.G.: Hardware counter based performance analysis, modelling and improvement through thread migration in NUMA systems. Ph.D. thesis, Universidad de Santiago de Compostela (2016)
The Unofficial Linux Perf Events Web-Page. http://web.eece.maine.edu/~vweaver/projects/perf_events
David, H., Gorbatov, E., Hanebutte, U.R., Khanna, R., Le, C.: Rapl: Memory power estimation and cap**. In: 2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED), pp. 189–194 (2010)
Desrochers, S., Paradis, C., Weaver, V.M.: A validation of DRAM RAPL power measurements. MEMSYS 2016 (2016)
Weaver, V.: Reading RAPL energy measurements from Linux. University of Maine, Orono
**, H., Frumkin, M., Yan, J.: The OpenMP implementation of NAS parallel benchmarks and its performance. Technical Report NAS-99-011, NASA Ames Research Center (1999)
Becoña, M.L., Lorenzo, O.G., Pena, T.F., Cabaleiro, J.C., Rivera, F.F., Lorenzo, J.A.: Caracterización de aplicaciones mediante información de contadores hardware en sistemas NUMA. In: XXVIII Jornadas de Paralelismo, Málaga (2017)
Acknowledgements
This work was supported in part by the Ministry of Science and Innovation of Spain under Grant PID2019-104834GB-I00 and AEI/FEDER, EU, in part by the Consellería de Cultura, Educación e Ordenación Universitaria of the Xunta de Galicia (accreditation 2019-2022 ED431G-2019/04 and reference competitive group 2019-2021, ED431C 2018/19), and in part by the European Regional Development Fund (ERDF), which acknowledges the CiTIUS-Centro de Investigación en Tecnolox Intelixentes of the University of Santiago de Compostela as a Research Center of the Galician University System.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Becoña, M.L., Lorenzo, O.G., Pena, T.F., Cabaleiro, J.C., Rivera, F.F., Lorenzo, J.A. (2021). A Tool for Runtime Analysis of Performance and Energy Usage in NUMA Systems. In: Mix, H., Niethammer, C., Zhou, H., Nagel, W.E., Resch, M.M. (eds) Tools for High Performance Computing 2018 / 2019. Springer, Cham. https://doi.org/10.1007/978-3-030-66057-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-66057-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66056-7
Online ISBN: 978-3-030-66057-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)