SpringerLink
Log in

Average block column action methods for solving least squares problems

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

The column action methods of algebraic iterative techniques play a pivotal role in the image reconstruction process. These methods converge to a least squares solution of inconsistent linear systems, providing a means to conserve computational resources by limiting small updates. In this paper, we introduce an improved version of the block column iteration called the average block column iteration, designed to enhance image reconstruction from projections in computerized tomography. Unlike its predecessor, the block column iteration (Linear Algebra Appl 484:322–343, 2015), the average block column iteration obviates the necessity of computing pseudoinverses of submatrices or solving subsystems. To substantiate the efficacy of this average variant, we present convergence analyses based on both random and greedy strategies. Additionally, we investigate techniques such as constraining to mitigate computation time. Furthermore, we furnish convergence results for methodologies employing these techniques. To validate our approach, we conduct numerical experiments and comparisons utilizing synthetic data and real image reconstruction scenarios. These experiments demonstrate the effectiveness and superiority of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

No data was used for the research described in the article.

References

  1. Fessler, J.A., Sutton, B.P.: Nonuniform fast Fourier transforms using min-max interpolation. IEEE Trans. Signal Process. 51(2), 560–574 (2003)

    MathSciNet  Google Scholar 

  2. Kak, A.C., Slaney, M.: Principles of Computerized Tomographic Imaging. SIAM, Philadelphia (2001)

    Google Scholar 

  3. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Google Scholar 

  4. Natterer, F.: The Mathematics of Computerized Tomography. SIAM, Philadelphia (2001)

    Google Scholar 

  5. Andersen, A.H., Kak, A.C.: Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm. Ultrason. Imaging 6(1), 81–94 (1984)

    Google Scholar 

  6. Censor, Y., Gordon, D., Gordon, R.: BICAV: a block-iterative parallel algorithm for sparse systems with pixel-related weighting. IEEE Trans. Med. Imaging 20(10), 1050–1060 (2001)

    Google Scholar 

  7. Censor, Y.: Block-iterative algorithms with diagonally scaled oblique projections for the linear feasibility problem. SIAM J. Matrix Anal. Appl. 24(1) (2002)

  8. Jiang, M., Wang, G.: Convergence of the simultaneous algebraic reconstruction technique (SART). IEEE Trans. Image Process. 12(8), 957–61 (2003)

    MathSciNet  Google Scholar 

  9. Wang, J., Zheng, Y.: On the convergence of generalized simultaneous iterative reconstruction algorithms. IEEE Trans. Image Process. 16(1), 1–6 (2007)

    MathSciNet  Google Scholar 

  10. Gregor, J., Benson, T.: Computational analysis and improvement of SIRT. IEEE Trans. Med. Imaging 27(7), 918–924 (2008)

    Google Scholar 

  11. Elfving, T., Nikazad, T.: Properties of a class of block-iterative methods. Inverse Probl. 25(11), 115011 (2009)

    MathSciNet  Google Scholar 

  12. Qu, G., Wang, C., Jiang, M.: Necessary and sufficient convergence conditions for algebraic image reconstruction algorithms. IEEE Trans. Image Process. 18(2), 435–440 (2009)

    MathSciNet  Google Scholar 

  13. Grecu, L., Popa, C.: Constrained SART algorithm for inverse problems in image reconstruction. Inverse Probl. Imag. 7(1), 199–216 (2013)

    MathSciNet  Google Scholar 

  14. Guo, X.-P.: Convergence studies on block iterative algorithms for image reconstruction. Appl. Math. Comput. 273, 525–534 (2016)

    MathSciNet  Google Scholar 

  15. Herman, G.T., Lent, A.: Iterative reconstruction algorithms. Comput. Biol. Med. 64, 273–294 (1976)

    Google Scholar 

  16. Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23(4), 444–466 (1981)

    MathSciNet  Google Scholar 

  17. Kaczmarz, S.: Angenäherte auflösung von systemen linearer gleichungen. Bull. Int. Acad. Polon. Sci. Lett. A 35, 355–357 (1937)

  18. Needell, D.: Randomized Kaczmarz solver for noisy linear systems. BIT 50(2), 395–403 (2010)

    MathSciNet  Google Scholar 

  19. Zouzias, A., Freris, N.: Randomized extended Kaczmarz for solving least squares. SIAM J. Matrix Anal. Appl. 34, 773–793 (2013)

    MathSciNet  Google Scholar 

  20. Needell, D., Zhao, R., Zouzias, A.: Randomized block Kaczmarz method with projection for solving least squares. Linear Algebra Appl. 484, 322–343 (2015)

    MathSciNet  Google Scholar 

  21. Petra, S., Popa, C.: Single projection Kaczmarz extended algorithms. Numer. Algorithms 73(3), 791–806 (2016)

    MathSciNet  Google Scholar 

  22. Lorenz, D.A., Rose, S., Schopfer, F.: The randomized Kaczmarz method with mismatched adjoint. BIT 58(4), 1079–1098 (2018)

    MathSciNet  Google Scholar 

  23. Du, K., Si, W.-T., Sun, X.-H.: Randomized extended average block Kaczmarz for solving least squares. SIAM J. Sci. Comput. 42(6), 3541–3559 (2020)

    MathSciNet  Google Scholar 

  24. Chen, J.-Q., Huang, Z.-D.: On the error estimate of the randomized double block Kaczmarz method. Appl. Math. Comput. 370, 124907 (2020)

    MathSciNet  Google Scholar 

  25. Zhang, Y., Li, H.: Splitting-based randomized iterative methods for solving indefinite least squares problem. Appl. Math. Comput. 446, 127892 (2023)

    MathSciNet  Google Scholar 

  26. Björck, Å., Elfving, T.: Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations. BIT 19(2), 145–163 (1979)

    MathSciNet  Google Scholar 

  27. Watt, D.W.: Column-relaxed algebraic reconstruction algorithm for tomography with noisy data. Appl. Opt. 33(20), 4420–7 (1994)

    Google Scholar 

  28. Elfving, T., Hansen, P.C., Nikazad, T.: Convergence analysis for column-action methods in image reconstruction. Numer. Algor. 74(3), 905–924 (2017)

    MathSciNet  Google Scholar 

  29. Karimpour, M., Nikazad, T.: On the convergence of nonstationary column-oriented version of algebraic iterative methods. Math. Meth. Appl. Sci. 43(10), 6131–6139 (2020)

    MathSciNet  Google Scholar 

  30. Nikazad, T., Karimpour, M.: Column-oriented algebraic iterative methods for nonnegative constrained least squares problems. Numer. Algor. 86, 1265–1284 (2021)

    MathSciNet  Google Scholar 

  31. Wright, S.J.: Coordinate descent algorithms. Math. Program. 151(1), 3–34 (2015)

    MathSciNet  Google Scholar 

  32. Elfving, T.: Block-iterative methods for consistent and inconsistent linear equations. Numer. Math. 35, 1–12 (1980)

    MathSciNet  Google Scholar 

  33. Altman, M.: On the approximate solution of linear algebraic equations. Bulletin de l’Academie Polonaise des Sciences Cl. III 35(4), 365–370 (1957)

  34. Galántai, A.: Projectors and Projection Methods. Springer, New York (2004)

    Google Scholar 

  35. Leventhal, D., Lewis, A.S.: Randomized methods for linear constraints: convergence rates and conditioning. Math. Oper. Res. 35(3), 641–654 (2010)

    MathSciNet  Google Scholar 

  36. Gower, R.M., Richtárik, P.: Randomized iterative methods for linear systems. SIAM J. Matrix Anal. Appl. 36(4), 1660–1690 (2015)

    MathSciNet  Google Scholar 

  37. Hefny, A., Needell, D., Ramdas, A.: Rows versus columns: randomized Kaczmarz or Gauss-Seidel for ridge regression. SIAM J. Sci. Comput. 39(5), 528–542 (2017)

    MathSciNet  Google Scholar 

  38. Edalatpour, V., Hezari, D., Salkuyeh, D.K.: A generalization of the Gauss-Seidel iteration method for solving absolute value equations. Appl. Math. Comput. 293, 156–167 (2017)

    MathSciNet  Google Scholar 

  39. Ma, A., Needell, D., Ramdas, A.: Convergence properties of the randomized extended Gauss-Seidel and Kaczmarz methods. SIAM J. Matrix Anal. Appl. 36(4), 1590–1604 (2015)

    MathSciNet  Google Scholar 

  40. Du, K., Sun, X.-H.: A doubly stochastic block Gauss-Seidel algorithm for solving linear equations. Appl. Math. Comput. 408, 126373 (2021)

    MathSciNet  Google Scholar 

  41. Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41(3), 239–274 (2006)

    MathSciNet  Google Scholar 

  42. Necoara, I., Richtárik, P., Patrascu, A.: Randomized projection methods for convex feasibility: conditioning and convergence rates. SIAM J. Optim. 29(4), 2814–2852 (2019)

    MathSciNet  Google Scholar 

  43. Necoara, I.: Faster randomized block Kaczmarz algorithms. SIAM J. Matrix Anal. Appl. 40(4), 1425–1452 (2019)

    MathSciNet  Google Scholar 

  44. Moorman, J.D., Tu, T.K., Molitor, D., Needell, D.: Randomized Kaczmarz with averaging. BIT 61(1), 337–359 (2021)

    MathSciNet  Google Scholar 

  45. Tropp, J.A.: Column subset selection, matrix factorization, and eigenvalue optimization. In: Proceedings of the 2009 Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 978–986. SIAM (2009)

  46. Tropp, J.A.: Improved analysis of the subsampled randomized hadamard transform. Adv. Adapt. Data Anal. 3(1–2), 115–126 (2011)

    MathSciNet  Google Scholar 

  47. Needell, D., Tropp, J.A.: Paved with good intentions: analysisof a randomized block Kaczmarz method. Linear Algebra Appl. 441, 199–221 (2014)

    MathSciNet  Google Scholar 

  48. Demmel, J.W.: The probability that a numerical analysis problem is difficult. Math. Comput. 50(182), 449–480 (1988)

    MathSciNet  Google Scholar 

  49. Koltracht, I., Lancaster, P.: Contraining strategies for linear iterative processes. IMA J. Numer. Anal. 10, 555–567 (1990)

    MathSciNet  Google Scholar 

  50. Popa, C.: Constrained Kaczmarz extended algorithm for image reconstruction. Linear Algebra Appl. 429(8–9), 2247–2267 (2008)

    MathSciNet  Google Scholar 

  51. Davis, T.A., Hu, Y.: The university of florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 1–25 (2011)

    MathSciNet  Google Scholar 

  52. Hansen, P.C.: Discrete Inverse Problems: Insight and Algorithms. SIAM, Philadelphia (2010)

    Google Scholar 

  53. Hansen, P.C., Jørgensen, J.S.: AIR Tools II: algebraic iterative reconstruction methods, improved implementation. Numer. Algor. 79(1), 107–137 (2018)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 12071196).

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Lanzhou University, Lanzhou, 730000, China

    Yu-Qi Niu & Bing Zheng

Authors
  1. Yu-Qi Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Zheng.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niu, YQ., Zheng, B. Average block column action methods for solving least squares problems. J. Appl. Math. Comput. 70, 2361–2386 (2024). https://doi.org/10.1007/s12190-024-02060-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12190-024-02060-0

Keywords

Mathematics Subject Classification

Search

Navigation