Abstract
This paper focuses on optimizing the Himeno benchmark for the vector computing system SX-Aurora TSUBASA and analyzes its performance in detail. The Vector Engine (VE) of SX-Aurora TSUBASA achieves a high memory bandwidth by High Bandwidth Memory (HBM2). The Himeno benchmark solves Poisson’s equation using the Jacobi iteration method. The kernel performs 19-point stencil calculations in the 3D domain, which is known as a memory-intensive kernel. This paper introduces four optimizations in a single VE or multiple VEs for the Himeno benchmark. First, for a single VE, to exploit the high bandwidth of the last-level cache (LLC) in the VE, the highly reusable array elements are stored in the LLC with the highest priority. Second, the computational domain is decomposed by considering the architecture of the VE so that this optimization can achieve a high LLC hit ratio and a long vector length. Third, to alleviate the loop overhead that tends to be large for vector computation, loop unrolling is applied to the kernel. Fourth, for multiple VEs, the optimization to improve the sustained MPI communication bandwidth is applied. The process map** is optimized by considering different types of communication mechanisms of SX-Aurora TSUBASA. The evaluation results show that the optimizations contribute to the long vector length, the high LLC hit ratio, and the short MPI communication time of the Himeno benchmark. As a result, the performance and the power efficiency are improved due to efficient vector processing through the optimizations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Himeno benchmark. http://i.riken.jp/en/supercom/documents/himenobmt/. Accessed 10 Aug 2020
Intel Xeon Gold 6126 Processor. https://www.intel.com/content/www/us/en/products/processors/xeon/scalable/gold-processors/gold-6126.html. Accessed 10 August 2020
MVAPICH: MPI over InfiniBand, Omni-Path, Ethernet/iWARP, and RoCE. http://mvapich.cse.ohio-state.edu/benchmarks/. Accessed 12 Aug 2020
NEC SX-Aurora TSUBASA. https://www.nec.com/en/global/solutions/hpc/sx/index.html. Accessed 12 Aug 2020
NVIDIA V100 TENSOR CORE GPU. https://www.nvidia.com/en-us/data-center/v100/. Accessed 10 Aug 2020
STREAM: Sustainable Memory Bandwidth in High Performance Computers. https://www.cs.virginia.edu/stream/. Accessed 10 Aug 2020
SX-Aurora TSUBASA C/C++ Compiler User’s Guide. https://www.hpc.nec/documents/sdk/pdfs/g2af01e-C++UsersGuide-018.pdf. Accessed 15 Aug 2020
SX-Aurora TSUBASA PROGINF/FTRACE User’s Guide. https://www.hpc.nec/documents/sdk/pdfs/g2at03e-PROGINF_FTRACE_User_Guide_en.pdf. Accessed 15 Aug 2020
Afanasyev, I.V., Voevodin, V.V., Voevodin, V.V., Komatsu, K., Kobayashi, H.: Analysis of relationship between SIMD-processing features used in NVIDIA GPUs and NEC SX-Aurora TSUBASA vector processors. In: Malyshkin, V. (ed.) PaCT 2019. LNCS, vol. 11657, pp. 125–139. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25636-4_10
Davidson, J.W., **turkar, S.: Aggressive loop unrolling in a retargetable, optimizing compiler. In: Gyimóthy, T. (ed.) CC 1996. LNCS, vol. 1060, pp. 59–73. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61053-7_53
Hart, A., Ansaloni, R., Gray, A.: Porting and scaling OpenACC applications on massively-parallel, GPU-accelerated supercomputers. Eur. Phys. J. Spec. Top. 210, 5–16 (2012). https://doi.org/10.1140/epjst/e2012-01634-y
Komatsu, K., Kishitani, T., Sato, M., Kobayashi, H.: An appropriate computing system and its system parameters selection based on bottleneck prediction of applications. In: 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 768–777 (2019)
Komatsu, K., et al.: Performance evaluation of a vector supercomputer SX-aurora tsubasa. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2018. IEEE Press (2018)
Matsuoka, S., Aoki, T., Endo, T., Nukada, A., Kato, T., Hasegawa, A.: GPU accelerated computing-from hype to mainstream, the rebirth of vector computing. J. Phys.: Conf. Ser. 180, 012043 (2009). https://doi.org/10.1088/1742-6596/180/1/012043
Phillips, E.H., Fatica, M.: Implementing the himeno benchmark with CUDA on GPU clusters. In: 2010 IEEE International Symposium on Parallel Distributed Processing (IPDPS), pp. 1–10 (2010). https://doi.org/10.1109/IPDPS.2010.5470394
Pietrzyk, J., Habich, D., Damme, P., Lehner, W.: First investigations of the vector supercomputer SX-aurora TSUBASA as a co-processor for database systems. In: Meyer, H., Ritter, N., Thor, A., Nicklas, D., Heuer, A., Klettke, M. (eds.) BTW 2019 - Workshopband, pp. 33–50. Gesellschaft für Informatik, Bonn (2019). https://doi.org/10.18420/btw2019-ws-03
Sarkar, V.: Optimized unrolling of nested loops. In: Proceedings of the 14th International Conference on Supercomputing, ICS 2000, pp. 153–166. Association for Computing Machinery, New York (2000). https://doi.org/10.1145/335231.335246
Van Werkhoven, B., Maassen, J., Seinstra, F.J., Bal, H.E.: Performance models for CPU-GPU data transfers. In: 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 11–20 (2014)
Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Commun. ACM 52(4), 65–76 (2009). https://doi.org/10.1145/1498765.1498785
Yamada, Y., Momose, S.: Vector engine processor of NEC’s brand-new supercomputer SX-aurora TSUBASA. In: Proceedings of A Symposium on High Performance Chips, Hot Chips, vol. 30, pp. 19–21 (2018)
Acknowledgments
This research was partially supported by MEXT Next Generation High-Performance Computing Infrastructures and Applications R&D Program, entitled “R&D of A Quantum-Annealing-Assisted Next Generation HPC Infrastructure and its Applications”. The authors also would like to acknowledge HPC Solutions for providing Infiniband products.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Onodera, A., Komatsu, K., Fujimoto, S., Isobe, Y., Sato, M., Kobayashi, H. (2021). Optimization of the Himeno Benchmark for SX-Aurora TSUBASA. In: Wolf, F., Gao, W. (eds) Benchmarking, Measuring, and Optimizing. Bench 2020. Lecture Notes in Computer Science(), vol 12614. Springer, Cham. https://doi.org/10.1007/978-3-030-71058-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-71058-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71057-6
Online ISBN: 978-3-030-71058-3
eBook Packages: Computer ScienceComputer Science (R0)