Abstract
In this study, we present a new heuristic sparse approximate inverse (SPAI) preconditioning algorithm on graphics processing unit (GPU), called HeuriSPAI. For the proposed HeuriSPAI, there are the following novelties: (1) a heuristic method is proposed, which gives the potential candidate indices of the nonzero entries of the preconditioner in advance to guide the selection of the new indices, so as to improve the quality of the obtained preconditioner; and (2) a parallel framework of constructing the heuristic SPAI preconditioner on GPU is presented on the basis of the new proposed heuristic SPAI preconditioning algorithm; and (3) each component of the preconditioner is computed in parallel inside a group of threads. HeuriSPAI fuses the advantages of static and dynamic SPAI preconditioning algorithms, and alleviates the drawback of the existing dynamic SPAI preconditioning algorithms on GPU that are not suitable for large matrices. Experimental results show that HeuriSPAI is effective for large matrices, and outperforms the popular preconditioning algorithms in three public libraries, as well as a recent parallel static SPAI preconditioning algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anzt, H., Gates, M., Dongarra, J., et al.: Preconditioned Krylov solvers on GPUs. Parallel Comput. 68, 32–44 (2017)
Anzt, H., Huckle, T.K., Brackle, J., Dongarra, J.: Incomplete sparse approximate inverses for parallel preconditioning. Parallel Comput. 71, 1–22 (2018)
Benzi, M., Meyer, C.D., Tuma, M.: A sparse approximate inverse preconditioner for the conjugate gradient method. SIAM J. Sci. Comput. 17(5), 1135–1149 (1996)
Benzi, M., Cullum, J., Tuma, M.: Robust approximate inverse preconditioning for the conjugate gradient method. SIAM J. Sci. Comput. 22(4), 1318–1332 (2000)
Bernaschi, M., Bisson, M., Fantozzi, C., Janna, C.: A factored sparse approximate inverse preconditioned conjugate gradient solver on graphics processing units. SIAM. J. Sci. Comput. 38(1), C53–C72 (2016)
Brandes, T., Arnold, A., Soddemann, T., Reith, D.: CPU vs. GPU - performance comparison for the Gram-Schmidt algorithm. Eur Phys J Spec Top 210(1), 73–88 (2012)
Chow, E.: A priori sparsity patterns for parallel sparse approximate inverse preconditioners. SIAM J. Sci. Comput. 21(5), 1804–1822 (2000)
Cosgrove, J.D.F., Diaz, J.C., Griewank, A.: Approximate inverse preconditioning for sparse linear systems. Int. J. Comput. Math. 44(1–2), 91–110 (1992)
Davis, T.A., Hu, Y.: The university of florida sparse matrix collection. ACM T. Math. Software 38(1), 1–25 (2011)
Dehnavi, M.M., Fern\(\acute{\text{ a }}\)ndez, D. M., Gaudiot, J. L., Giannacopoulos, D. D.: Parallel sparse approximate inverse preconditioning on graphic processing units. IEEE T. Parall. Distr 24(9), 852–1861 (2013)
Duin, A.C.N.V.: Scalable parallel preconditioning with the sparse approximate inverse of triangular systems. SIAM J. Matrix Anal. Appl. 20(4), 987–1006 (1999)
Ferronato, M., Janna, C., Pini, G.: A generalized block FSAI preconditioner for nonsymmetric linear systems. J. Comput. Appl. Math. 256, 230–241 (2014)
Gao, J., Liang, R., Wang, J.: Research on the conjugate gradient algorithm with a modified incomplete Cholesky preconditioner on GPU. J. Parallel Distr. Com. 74(2), 2088–2098 (2014)
Gao, J., Wu, K., Wang, Y., Qi, P., He, G.: GPU-accelerated preconditioned GMRES method for two-dimensional Maxwell’s equations. Int. J. Comput. Math. 94(10), 2122–2144 (2017)
Gao, J., Chen, Q., He, G.: A thread-adaptive sparse approximate inverse preconditioning algorithm on multi-GPUs. Parallel Comput. 101, 102724 (2021)
Grote, M., Huckle, T.: Parallel preconditioning with sparse approximate inverses. SIAM J. Sci. Comput. 18(3), 838–853 (1997)
He, G., Yin, R., Gao, J.: An efficient sparse approximate inverse preconditioning algorithm on GPU. Concurr. Comput.-Pract. Exp. 32(7), e5598 (2020)
Jia, Z., Zhu, B.: A power sparse approximate inverse preconditioning procedure for large sparse linear systems. Numer. Linear Algebr. 16(4), 259–299 (2009)
Kolotilina, L.Y., Yeremin, A.Y.: Factorized sparse approximate inverse preconditioning I. Theory. SIAM J. Matrix Anal. Appl. 14(1), 45–58 (1993)
NVIDIA, CUBLAS Library, v11.1 (2021)
NVIDIA, CUDA C Programming Guide, v11.1 (2021)
NVIDIA, CUSPARSE Library, v11.1 (2021)
Rupp, K., Tillet, R., Rudolf, F.: ViennaCL–linear algebra library for multi- and many-core architectures. SIAM J. Sci. Comput. 38(5), S412–S439 (2016)
Saad, Y.: Iterative Methods for Sparse Linear Systems, second version. SIAM, Philadelphia, PA (2003)
Saad, Y., Schultz, M.H.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)
van der Vorst, H.A.: A vectorizable variant of some ICCG methods. SIAM J. Sci. Stat. Comput. 3(3), 350–356 (1982)
van der Vorst, H.A.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetirc linear systems. SIAM J. Sci. Stat. Comput. 12(3), 631–644 (1992)
Acknowledgements
This work was funded by the Natural Science Foundation of China under grant number 61872422.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gao, J., Chu, X. & Wang, Y. HeuriSPAI: a heuristic sparse approximate inverse preconditioning algorithm on GPU. CCF Trans. HPC 5, 160–170 (2023). https://doi.org/10.1007/s42514-023-00142-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42514-023-00142-2