SpringerLink
Log in

A singular value shrinkage thresholding algorithm for folded concave penalized low-rank matrix optimization problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we study the low-rank matrix optimization problem, where the loss function is smooth but not necessarily convex, and the penalty term is a nonconvex (folded concave) continuous relaxation of the rank function. Firstly, we give the closed-form singular value shrinkage thresholding operators for several matrix-valued folded concave penalty functions. Secondly, we adopt a singular value shrinkage thresholding (SVST) algorithm for the nonconvex low-rank matrix optimization problem, and prove that the proposed SVST algorithm converges to a stationary point of the problem. Furthermore, we show that the limit point satisfies a global necessary optimality condition which can exclude too many stationary points even local minimizers in order to refine the solutions. We conduct a large number of numeric experiments to test the performance of SVST algorithm on the randomly generated low-rank matrix completion problem, the real 2D and 3D image recovery problem and the multivariate linear regression problem. Numerical results show that SVST algorithm is very competitive for low-rank matrix optimization problems in comparison with some state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availibility

Data will be made available on reasonable request.

References

  1. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. Ser. A 137, 91–129 (2013)

    MathSciNet  Google Scholar 

  2. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    MathSciNet  Google Scholar 

  3. Beck, A., Hallak, N.: Optimization problems involving group sparsity terms. Math. Program. 178, 39–67 (2019)

    MathSciNet  Google Scholar 

  4. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. Ser. A 146, 459–494 (2014)

    MathSciNet  Google Scholar 

  5. Cai, J., Candès, E., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    MathSciNet  Google Scholar 

  6. Candès, E., Plan, Y.: Tight Oracle bounds for low-rank matrix recovery from a minimal number of noisy random measurements. IEEE Trans. Inf. Theory 57(4), 2342–2359 (2011)

    Google Scholar 

  7. Candès, E., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2010)

    MathSciNet  Google Scholar 

  8. Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM Publisher, New York (1990)

    Google Scholar 

  9. Cui, A., Peng, J., Li, H., Zhang, C., Yu, Y.: Corrigendum to “Affine matrix rank minimization problem via non-convex fraction function penalty” [J. Comput. Appl. Math. 336 (2018) 353-374]. J. Comput. Appl. Math. 352(15): 478–485

  10. Drineas, P., Kannan, R., Mahoney, M.: Fast Monte Carlo algorithms for matrices II: computing low-rank approximations to a matrix. SIAM J. Comput. 36(1), 158–183 (2006)

    MathSciNet  Google Scholar 

  11. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)

    MathSciNet  Google Scholar 

  12. Fazel, M.: Matrix rank minimization with applications. PhD thesis, Stanford University (2002)

  13. Foucart, S., Lai, M.-J.: Sparsest solutions of underdetermined linear systems via \(\ell _q\) minimization for \(0 < q \le 1\). Appl. Comput. Harmon. A. 26, 395–407 (2009)

    Google Scholar 

  14. Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J.: A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of the 30th International Conference on International Conference on Machine Learning (ICML’13), 28(2), pp. 37-45 (2013)

  15. Gu, S., **e, Q., Meng, D., et al.: Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vis. 121(2), 183–208 (2017)

    Google Scholar 

  16. He, L., Wang, Y., **ang, Z.: Support driven wavelet frame-based image deblurring. Inf. Sci. 479, 250–269 (2019)

    Google Scholar 

  17. Huang, J., Jiao, Y., **, B., Liu, J., Lu, X., Yang, C.: A unified primal dual active set algorithm for nonconvex sparse recovery. Stat. Sci. 36, 215–238 (2021)

    MathSciNet  Google Scholar 

  18. **, Z.F., Wan, Z., Jiao, Y., Lu, X.: An alternating direction method with continuation for nonconvex low rank minimization. J. Sci. Comput. 66(2), 849–869 (2016)

    MathSciNet  Google Scholar 

  19. Lai, M.-J., Xu, Y., Yin, W.: Improved iteratively rewighted least squares for unconstrained smoothed \(\ell _p\) minimization. SIAM J. Numer. Anal. 51(2), 927–957 (2013)

    MathSciNet  Google Scholar 

  20. Lee, C., Lam, E.: Computationally efficient truncated nuclear norm minimization for high dynamic range imaging. IEEE Trans. Image Process. 25(9), 4145–4157 (2016)

    MathSciNet  PubMed  ADS  Google Scholar 

  21. Li, Y., Shang, K., Huang, Z.: A singular value p-shrinkage thresholding algorithm for low rank matrix recovery. Comput. Optim. Appl. 73, 453–476 (2019)

    MathSciNet  Google Scholar 

  22. Liu, Y., Sun, D., Toh, K.-C.: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. 133(1–2), 399–436 (2012)

    MathSciNet  Google Scholar 

  23. Lu, C., Zhu, C., Xu, C., Yan, S., Lin, Z.: Generalized singular value thresholding. In: Proceedings of the AAAI Conference on Artificial Intelligence 29(1), 1805–1811 (2015)

  24. Lu, Z., Zhang, Y., Liu, X.: Penalty decomposition methods for rank minimization. Optim. Methods Softw. 30, 531–558 (2015)

    MathSciNet  Google Scholar 

  25. Lu, Z., Zhang, Y., Lu, J.: \(\ell _p\) regularized low-rank approximation via iterative reweighted singular value minimization. Comput. Optim. Appl. 68, 619–642 (2017)

    MathSciNet  Google Scholar 

  26. Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. 128(1–2), 321–353 (2011)

    MathSciNet  Google Scholar 

  27. Ma, T., Lou, Y., Huang, T.: Truncated \(\ell _{1-2}\) models for sparse recovery and rank minimization. SIAM J. Imaging Sci. 10(3), 1346–1380 (2017)

    MathSciNet  Google Scholar 

  28. Marjanovic, G., Solo, V.: On \(\ell _q\) optimization and matrix completion. IEEE Trans. Signal Process. 60(11), 5714–5724 (2012)

    MathSciNet  ADS  Google Scholar 

  29. Oymak, S., Hassibi, B.: New null space results and recovery thresholds for matrix rank minimization. Eprint Arxiv 58(4), 766–773 (2010)

    Google Scholar 

  30. Pan, L., Chen, X.: Group sparse optimization for images recovery using capped folded concave functions. SIAM J. Imaging Sci. 14(1), 1–25 (2021)

    MathSciNet  Google Scholar 

  31. Peng, D., **u, N., Yu, J.: \(S_{1/2}\) regularization methods and fixed point algorithms for affine rank minimization problems. Comput. Optim. Appl. 67, 543–569 (2017)

    MathSciNet  Google Scholar 

  32. Peng, D., **u, N., Yu, J.: Global optimality conditions and fixed point continuation algorithm for non-Lipschitz \(\ell _{p}\) regularized matrix minimization. Sci China Math 61, 1139–1152 (2018)

    MathSciNet  Google Scholar 

  33. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)

    MathSciNet  Google Scholar 

  34. Recht, B., Xu, W., Hassibi, B.: Null space conditions and thresholds for rank minimization. Math. Program. 127(1), 175–202 (2011)

    MathSciNet  Google Scholar 

  35. Rockafellar, R., Wets, R.-J.: Variational Analysis. Springer, Berlin (1998)

    Google Scholar 

  36. Su, X., Wang, Y., Kang, X., Tao, R.: Nonconvex truncated nuclear norm minimization based on adaptive bisection method. IEEE Trans. Circuits Syst. Video Technol. 29(11), 3159–3172 (2019)

    Google Scholar 

  37. Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. Ser. B 125, 263–295 (2010)

    MathSciNet  Google Scholar 

  38. Tütüncü, R., Toh, K.-C., Todd, M.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95(2), 189–217 (2003)

    MathSciNet  Google Scholar 

  39. Toh, K.-C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6(3), 615–640 (2010)

    MathSciNet  Google Scholar 

  40. Wang, S., Liu, D., Zhang, Z.: Nonconvex relaxation approaches to robust matrix recovery. In: The 23rd international conference on artificial intelligence in 2013 (IJCAI-2013), pp. 1764-1770 (2013)

  41. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    PubMed  ADS  Google Scholar 

  42. Wang, Z., Wang, W., Wang, J., Chen, S.: Fast and efficient algorithm for matrix completion via closed-form \(2/3\)-thresholding operator. Neurocomputing 330, 212–222 (2019)

    Google Scholar 

  43. Wang, W., Zhang, F., Wang, J.: Low-rank matrix recovery via regularized nuclear norm minimization. Appl. Comput. Harmon. A. 54, 1–19 (2021)

    MathSciNet  Google Scholar 

  44. **ao, Y., **, Z.: An alternating direction method for linear-constrained matrix nuclear norm minimization. Numer. Linear Algebra Appl. 19(3), 541–554 (2012)

    MathSciNet  Google Scholar 

  45. Yang, J., Yuan, X.: Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math. Comput. 82(281), 301–329 (2013)

    MathSciNet  Google Scholar 

  46. Yu, Q., Zhang, X.: A smoothing proximal gradient algorithm for matrix rank minimization problem. Comput. Optim. Appl. 81, 519–538 (2022)

    MathSciNet  Google Scholar 

  47. Zhang, C.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)

    MathSciNet  Google Scholar 

  48. Zhang, T.: Analysis of multi-stage convex relaxation for sparse regularization. J. Mach. Learn. Res. 11, 1081–1107 (2010)

    MathSciNet  ADS  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (12261020, 11861020), the Guizhou Provincial Science and Technology Program (ZK[2021]009), the Growth Project of Education Department of Guizhou Province for Young Talents in Science and Technology ([2018]121), and the Research Foundation for Postgraduates of Guizhou Province (YJSCXJH[2020]085).

The authors are deeply grateful to the three anonymous reviewers for their valuable suggestions and comments that help us to revise the paper into the present form.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Guizhou University, Guiyang, 550025, China

    **an Zhang, Dingtao Peng & Yanyan Su

Authors
  1. **an Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dingtao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanyan Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dingtao Peng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Peng, D. & Su, Y. A singular value shrinkage thresholding algorithm for folded concave penalized low-rank matrix optimization problems. J Glob Optim 88, 485–508 (2024). https://doi.org/10.1007/s10898-023-01322-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-023-01322-8

Keywords

Mathematics Subject Classification

Search

Navigation