SpringerLink
Log in

An Accelerated Tensorial Double Proximal Gradient Method for Total Variation Regularization Problem

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We consider the constrained tensorial total variation minimization problem for regularizing ill-posed multidimensional problems arising in many fields, such as image and video processing and multidimensional data completion. The nonlinearity and the non-differentiability of the total variation minimization problem make the resolution directly more complex. The aim of the present paper is to bring together the resolution of this problem using an iterative tensorial double proximal gradient algorithm and the acceleration of the convergence rate by updating some efficient extrapolation techniques in the tensor form. The general structure of the proposed method expands its fields of application. We will restrict our numerical application to the multidimensional data completion which illustrates the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H.: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, vol. 49. Springer, New York (2011)

    MATH  Google Scholar 

  2. Bauschke, H.H., Combettes, P.L., et al.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, vol. 408. Springer, New York (2011)

    MATH  Google Scholar 

  3. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    MathSciNet  MATH  Google Scholar 

  4. Benchettou, O., Bentbib, A.H., Bouhamidi, A.: Tensorial total variation-based image and video restoration with optimized projection methods. Opt. Softw. 1–32 (2022)

  5. Bouhamidi, A., Bellalij, M., Enkhbat, R., Jbilou, K., Raydan, M.: Conditional gradient method for double-convex fractional programming matrix problems. J. Optim. Theory Appl. 176(1), 163–177 (2018)

    MathSciNet  MATH  Google Scholar 

  6. Bouhamidi, A., Jbilou, K., Reichel, L., Sadok, H.: An extrapolated TSVD method for linear discrete ill-posed problems with Kronecker structure. Linear Algebra Appl. 434(7), 1677–1688 (2011)

    MathSciNet  MATH  Google Scholar 

  7. Brezinski, C., Zaglia, M.R.: Extrapolation Methods: Theory and Practice. Elsevier, Amsterdam (2013)

    MATH  Google Scholar 

  8. Candès, E.J., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2010)

    MathSciNet  MATH  Google Scholar 

  9. Chehab, J.-P., Raydan, M.: Geometrical inverse matrix approximation for least-squares problems and acceleration strategies. Numer. Algorithms 1–19 (2020)

  10. Chen, C., Wu, Z.-B., Chen, Z.-T., Zheng, Z.-B., Zhang, X.-J.: Auto-weighted robust low-rank tensor completion via tensor-train. Inf. Sci. 567, 100–115 (2021)

    MathSciNet  Google Scholar 

  11. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.-I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, New Jersey (2009)

    Google Scholar 

  12. Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. Fixed-Point Algorithms Inverse Probl. Sci. Eng. 185–212 (2011)

  13. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)

    MathSciNet  MATH  Google Scholar 

  14. Duran, J., Coll, B., Sbert, C.: Chambolle’s projection algorithm for total variation denoising. Image processing on Line 3, 311–331 (2013)

    Google Scholar 

  15. Ekeland, I., Temam, R.: Convex Analysis and Variational Problems (Studies in Mathematics and its Applications), vol. 1. Elsevier, New York, North-Holland (1976)

    MATH  Google Scholar 

  16. El Ichi, A., Jbilou, K., Sadaka, R.: Tensor global extrapolation methods using the n-mode and the Einstein products. Mathematics 8(8), 1298 (2020)

    Google Scholar 

  17. Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Prob. 27(2), 025010 (2011)

    MathSciNet  MATH  Google Scholar 

  18. Getreuer, P.: Rudin-osher-fatemi total variation denoising using split bregman. Image Process. Line 2, 74–95 (2012)

    Google Scholar 

  19. Getreuer, P.: Total variation inpainting using split Bregman. Image Process. Line 2, 147–157 (2012)

    Google Scholar 

  20. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Tata Institute of Fundamental Research (1980)

  21. Haddad, A.: Stability in a class of variational methods. Appl. Comput. Harmon. Anal. 23(1), 57–73 (2007)

    MathSciNet  MATH  Google Scholar 

  22. Jbilou, K., Messaoudi, A.: Block extrapolation methods with applications. Appl. Numer. Math. 106, 154–164 (2016)

    MathSciNet  MATH  Google Scholar 

  23. Jbilou, K., Reichel, L., Sadok, H.: Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numer. Algorithms 51(2), 195–208 (2009)

    MathSciNet  MATH  Google Scholar 

  24. Jbilou, K., Sadok, H.: Vector extrapolation methods. Applications and numerical comparison. J. Comput. Appl. Math. 122(1–2), 149–165 (2000)

    MathSciNet  MATH  Google Scholar 

  25. Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435(3), 641–658 (2011)

    MathSciNet  MATH  Google Scholar 

  26. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    MathSciNet  MATH  Google Scholar 

  27. Lee, C.W., Jung, K., Kim, H.J.: Automatic text detection and removal in video sequences. Pattern Recogn. Lett. 24(15), 2607–2623 (2003)

    Google Scholar 

  28. Lee, N., Cichocki, A.: Fundamental tensor operations for large-scale data analysis using tensor network formats. Multidimens. Syst. Signal Process. 29(3), 921–960 (2018)

    MathSciNet  MATH  Google Scholar 

  29. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28. Curran Associates Inc, New York (2015)

    Google Scholar 

  30. Li, M., Li, B.: A double fittings based total variation model for image denoising. In: 2021 IEEE 6th International Conference on Signal and Image Processing (ICSIP), pp. 467–471 (2021)

  31. Li, X., Ye, Y., Xu, X.: Low-rank tensor completion with total variation for visual data inpainting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 of AAAI’17 (2017)

  32. Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2012)

    Google Scholar 

  33. López, W., Raydan, M.: An acceleration scheme for Dykstra’s algorithm. Comput. Optim. Appl. 63(1), 29–44 (2016)

    MathSciNet  MATH  Google Scholar 

  34. Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., Yan, S.: Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 925–938 (2019)

    Google Scholar 

  35. Lysaker, M., Lundervold, A., Tai, X.-C.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12(12), 1579–1590 (2003)

    MATH  Google Scholar 

  36. Modha, U., Dave, P.: Image inpainting-automatic detection and removal of text from images. Int. J. Eng. Res. Appl. (IJERA) ISSN 2248–9622 (2014)

  37. Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. Fr. 93, 273–299 (1965)

    MATH  Google Scholar 

  38. Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)

    MathSciNet  MATH  Google Scholar 

  39. Pang, Z.-F., Zhou, Y.-M., Wu, T., Li, D.-J.: Image denoising via a new anisotropic total-variation-based model. Signal Process. Image Commun. 74, 140–152 (2019)

    Google Scholar 

  40. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Opt. 1(3), 127–239 (2014)

    Google Scholar 

  41. Rockafellar, R.T.: Convex Analysis. Princeton University Press, New Jersey (2015)

    Google Scholar 

  42. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)

    MathSciNet  MATH  Google Scholar 

  43. Sidi, A.: Efficient implementation of minimal polynomial and reduced rank extrapolation methods. J. Comput. Appl. Math. 36(3), 305–337 (1991)

    MathSciNet  MATH  Google Scholar 

  44. Sridevi, G., Kumar, S.S.: Image inpainting based on fractional-order nonlinear diffusion for image reconstruction. Circuits Syst. Signal Process. 38(8), 3802–3817 (2019)

    Google Scholar 

  45. Van Leeuwen, J., Leeuwen, J.: Handbook of Theoretical Computer Science: Algorithms and Complexity. Elsevier, Amsterdam (1990)

    MATH  Google Scholar 

  46. Yokota, T., Zhao, Q., Cichocki, A.: Smooth parafac decomposition for tensor completion. IEEE Trans. Signal Process. 64(20), 5423–5436 (2016)

    MathSciNet  MATH  Google Scholar 

  47. Zhang, Z., Aeron, S.: Exact tensor completion using t-svd. IEEE Trans. Signal Process. 65(6), 1511–1526 (2016)

    MathSciNet  MATH  Google Scholar 

  48. Zhang, Z., Ely, Z., Aeron, S., Hao, N., Kilmer, M.: Novel methods for multilinear data completion and de-noising based on tensor-svd. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3842–3849 (2014)

  49. Zhao, Q., Zhang, L., Cichocki, A.: Bayesian cp factorization of incomplete tensors with automatic rank determination. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1751–1763 (2015)

    Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous referees for their recommendations and helpful remarks, which improved the quality of this paper. The second author would like to thank the Moroccan Ministry of Higher Education, Scientific Research and Innovation and the OCP Foundation, which support him through the APRD research program.

Author information

Authors and Affiliations

  1. Laboratory LAMAI, University Cadi Ayyad, Marrakech, Morocco

    Oumaima Benchettou & Abdeslem Hafid Bentbib

  2. Laboratory L.M.P.A, University Littoral, Côte d’Opale, Calais, France

    Oumaima Benchettou & Abderrahman Bouhamidi

Authors
  1. Oumaima Benchettou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdeslem Hafid Bentbib
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abderrahman Bouhamidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oumaima Benchettou.

Additional information

Communicated by Olivier Fercoq.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Benchettou, O., Bentbib, A.H. & Bouhamidi, A. An Accelerated Tensorial Double Proximal Gradient Method for Total Variation Regularization Problem. J Optim Theory Appl 198, 111–134 (2023). https://doi.org/10.1007/s10957-023-02234-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-023-02234-z

Keywords

Search

Navigation