Abstract
Turbulent flows play important roles in many flow-related phenomena that appear in various fields. However, despite numerous studies on turbulence, the nature of turbulence has not yet been fully clarified. Direct numerical simulation (DNS) of incompressible homogeneous turbulence in a periodic box is currently a powerful method for studying turbulent flows. However, even modern world-class supercomputers do not have sufficient computational resources to carry out DNS at very high Reynolds number (Re). Memory capacity constraints are particularly severe. Therefore, we have developed an out-of-core DNS (ooc-DNS) code that uses storage to overcome memory limitations. The ooc-DNS code can reduce memory usage by up to a quarter and allows DNS at a higher Re, which would be impossible under normal usage due to memory limitations. When implementing the ooc-DNS code, however, it is crucial to accelerate file input/output (I/O) because the I/O time for storage accounts for a large percentage of the execution time. In this paper, we evaluate the I/O performance of the ooc-DNS code when using a file system called the Lightweight Layered I/O Accelerator of the supercomputer Fugaku. We also evaluate the impact of the I/O cache and its size on I/O performance and show that the I/O processing can be accelerated by using the cache and optimizing its size. Finally, by taking on I/O cache size when executing the ooc-DNS code with \(8,192^3\) grid points, the I/O speed and overall execution speed are increased by 2.4 times and 1.9 times compared to that without the I/O cache.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Github: IOR. https://github.com/hpc/ior
System Overview (ES). https://www.jamstec.go.jp/es/jp/es1/system/system.html. Accessed 12 Dec 2023
Performance development (2023). https://www.top500.org/statistics/perfdevel/. Accessed 12 Dec 2023
Akimoto, H., Okamoto, T., Kagami, T., Seki, K., Sakai, K., Imade, H., et al.: File system and power management enhanced for supercomputer Fugaku. Fujitsu Tech. Rev. 3, 2020–2023 (2020)
Canuto, C., Hussaini, M.Y., Quarteroni, A., Zang, T.A.: Spectral Methods in Fluid Dynamics. Springer, Heidelberg (1988)
Fu, H., Liao, J., Yang, J., Wang, L., Song, Z., Huang, X., et al.: The Sunway TaihuLight supercomputer: system and applications. Sci. China Inf. Sci. 59(7), June 2016. https://doi.org/10.1007/s11432-016-5588-7
Ishihara, T., Kaneda, Y., Morishita, K., Yokokawa, M., Uno, A.: Second-order velocity structure functions in direct numerical simulations of turbulence with \({R}_{\lambda }\) up to 2250. Phys. Rev. Fluids 5, 104608 (2020). https://doi.org/10.1103/PhysRevFluids.5.104608
Kaneda, Y., Ishihara, T., Yokokawa, M., Itakura, K., Uno, A.: Energy dissipation rate and energy spectrum in high resolution direct numerical simulations of turbulence in a periodic box. Phys. Fluids 15(2), L21–L24 (2003). https://doi.org/10.1063/1.1539855
Komatsu, K., Momose, S., Isobe, Y., Watanabe, O., Musa, A., Yokokawa, M., et al.: Performance evaluation of a vector supercomputer sx-aurora tsubasa. In: SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 685–696 (2018). https://doi.org/10.1109/SC.2018.00057
Liao, X., **ao, L., Yang, C., Lu, Y.: MilkyWay-2 supercomputer: system and application. Front. Comp. Sci. 8(3), 345–356 (2014). https://doi.org/10.1007/s11704-014-3501-3
Mitsuhisa, S., Yutaka, I., Hirofumi, T., Yuetsu, K., Tetsuya, O., Miwako, T., et al.: Co-design for A64FX manycore processor and “Fugaku". In: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis p. 1, November 2020. https://doi.org/10.1109/sc41405.2020.00051, https://cir.nii.ac.jp/crid/1360013173149649280
Orszag, S.A.: Numerical methods for the simulation of turbulence. Phys. Fluids 12(12), II–250–II–257 (1969).https://doi.org/10.1063/1.1692445
Vazhkudai, S.S., De Supinski, B.R., Bland, A.S., Geist, A., Sexton, J., Kahle, J., et al.: The design, deployment, and evaluation of the CORAL pre-exascale systems. In: SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 661–672. IEEE (2018)
Yang, X.J., Liao, X.K., Lu, K., Hu, Q.F., Song, J.Q., Su, J.S.: The TianHe-1A supercomputer: its hardware and software. J. Comput. Sci. Technol. 26(3), 344–351 (2011). https://doi.org/10.1007/s02011-011-1137-8
Yokokawa, M., Yamane, Y., Yamaguchi, K., Soga, T., Matsumoto, T., et al.: I/O Performance evaluation of a memory-saving DNS code on SX-aurora TSUBASA. In: 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 692–696 (2023). https://doi.org/10.1109/IPDPSW59300.2023.00117
Acknowledgment
This study used the computational resources of the Supercomputer Fugaku provided by the RIKEN Center for Computational Science through the HPCI System Research project (Project ID No. hp230143). This work was partially conducted and funded at Joint-Research Division of High-Performance Computing (NEC) of Cyberscience Center at Tohoku University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hatanaka, Y. et al. (2024). File I/O Cache Performance of Supercomputer Fugaku Using an Out-of-Core Direct Numerical Simulation Code of Turbulence. In: Franco, L., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2024. ICCS 2024. Lecture Notes in Computer Science, vol 14837. Springer, Cham. https://doi.org/10.1007/978-3-031-63778-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-63778-0_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-63777-3
Online ISBN: 978-3-031-63778-0
eBook Packages: Computer ScienceComputer Science (R0)