SpringerLink
Log in

A Strengthened SDP Relaxation for Quadratic Optimization Over the Stiefel Manifold

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We study semidefinite programming (SDP) relaxations for the NP-hard problem of globally optimizing a quadratic function over the Stiefel manifold. We introduce a strengthened relaxation based on two recent ideas in the literature: (i) a tailored SDP for objectives with a block-diagonal Hessian; and (ii) the use of the Kronecker matrix product to construct SDP relaxations. Using synthetic instances on five problem classes, we show that, in general, our relaxation significantly strengthens existing relaxations, although at the expense of longer solution times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Note that, although the relaxation in [19] was applied to \(g \ne 0\), the authors only experimented with \(g = 0\) due to their specific application of interest. We will also test with \(g = 0\) in Sect. 3.

  2. This result and proof was shared with us by Dr. Gang Luo of Huawai Technologies Compnay, whom we gratefully acknowledge.

  3. When \(\text {rank}(Y) > 1\), one could also try to reduce the rank of Y by moving to an extreme point of the optimal solution set using a procedure such as the rank-reduction algorithm described in [29]. If the subsequent rank is 1, then one recovers a primal value as just discussed. However, we have not implemented such a rank-reduction procedure in our computational results.

References

  1. Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008)

    Book  MATH  Google Scholar 

  2. Anjos, M. F., Wolkowicz, H.: Strengthened semidefinite relaxations via a second lifting for the Max-Cut problem. volume 119, pages 79–106. Foundations of heuristics in combinatorial optimization ( 2002)

  3. Anstreicher, K., Chen, X., Wolkowicz, H., Yuan, Y.-X.: Strong duality for a trust-region type relaxation of the quadratic assignment problem. Linear Algebra Appl. 301(1–3), 121–136 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  4. Anstreicher, K., Wolkowicz, H.: On Lagrangian relaxation of quadratic matrix constraints. SIAM J. Matrix Anal. Appl. 22(1), 41–55 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  5. Anstreicher, K.M.: Kronecker product constraints with an application to the two-trust-region subproblem. SIAM J. Opt. 27(1), 368–378 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  6. Birtea, P., Caşu, I., Comănescu, D.: Second order optimality on orthogonal Stiefel manifolds. Bulletin des Sci. Mathématiques 161, 102868 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  7. Boumal, N.: An introduction to optimization on smooth manifolds. To appear with Cambridge University Press, (Apr 2022)

  8. Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15(42), 1455–1459 (2014)

    MATH  Google Scholar 

  9. Breloy, A., Kumar, S., Sun, Y., Palomar, D.P.: Majorization-minimization on the Stiefel manifold with application to robust sparse PCA. IEEE Transact. Signal Process. 69, 1507–1520 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chrétien, S., Guedj, B.: Revisiting clustering as matrix factorisation on the Stiefel manifold. In G. Nicosia, V. Ojha, E. La Malfa, G. Jansen, V. Sciacca, P. Pardalos, G. Giuffrida, and R. Umeton, editors, International Conference on Machine Learning, Optimization, and Data Science, pp. 1–12. Springer, (2020)

  11. Chu, M.T., Trendafilov, N.T.: The orthogonally constrained regression revisited. J. Comput. Graph. Stat. 10(4), 746–771 (2001)

    Article  MathSciNet  Google Scholar 

  12. de Klerk, E., Sotirov, R.: Exploiting group symmetry in semidefinite programming relaxations of the quadratic assignment problem. Math. Program 122(2), 225–246 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. de Meijer, F., Sotirov, R., Wiegele, A., Zhao, S.: Partitioning through projections: strong sdp bounds for large graph partition problems, (2022)

  14. Dodig, M., Stošić, M., Xavier, J.: On minimizing a quadratic function on Stiefel manifold. Linear Algebra Appl. 475, 251–264 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  16. Eldén, L.: Algorithms for the regularization of ill-conditioned least squares problems. BIT Num. Math. 17(2), 134–145 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  17. Eldén, L., Park, H.: A Procrustes problem on the Stiefel manifold. Num. Math. 82(4), 599–619 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  18. Fu, M., Luo, Z.-Q., Ye, Y.: Approximation algorithms for quadratic programming. J. Combinat. Opt. 2, 29–50 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gilman, K., Burer, S., Balzano, L.: A semidefinite relaxation for sums of heterogeneous quadratics on the Stiefel manifold, (2022)

  20. Gower, J. C., Dijksterhuis, G. B.: Procrustes Problems. Oxford University Press, (2004)

  21. Hu, J., Liu, X., Wen, Z.-W., Yuan, Y.-X.: A brief introduction to manifold optimization. J. Operat. Res. Soc. China 8(2), 199–248 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  22. Jiang, R., Li, D.: Second order cone constrained convex relaxations for nonconvex quadratically constrained quadratic programming. J. Global Opt. 75(2), 461–494 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  23. Kim, J., Kang, M., Kim, D., Ha, S.-Y., Yang, I.: A stochastic consensus method for nonconvex optimization on the Stiefel manifold. In 2020 59th IEEE Conference on Decision and Control (CDC), pp. 1050–1057. IEEE, (2020)

  24. Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11(3), 796–817 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  25. Li, X., Pong, T.K., Sun, H., Wolkowicz, H.: A strictly contractive Peaceman-Rachford splitting method for the doubly nonnegative relaxation of the minimum cut problem. Comput. Opt. Appl. 78(3), 853–891 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  26. Nemirovski, A.: Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Math Program. 109(2), 283–317 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  27. Nesterov, Y., Wolkowicz, H., Ye, Y.: Semidefinite programming relaxations of nonconvex quadratic optimization. In R. Saigal, L. Vandenberghe, and H. Wolkowicz, editors, Handbook of semidefinite programming, vol 27 of Internat. Ser. Oper. Res. Management Sci., pages 361–419. Kluwer Acad. Publ., Boston, MA, (2000)

  28. Overton, M.L., Womersley, R.S.: On the sum of the largest eigenvalues of a symmetric matrix. SIAM J. Matrix Anal. Appl. 13(1), 41–45 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Pataki, G.: The geometry of semidefinite programming. In H. Wolkowicz, R. Saigal, and L. Vandenberghe, editors, Handbook of Semidefinite Programming: Theory, Algorithms, and Applications. Kluwer Academic Publishers, (2000)

  30. Sanyal, R., Sottile, F., Sturmfels, B.: Orbitopes. Mathematika 57(2), 275–314 (2011)

  31. Saunderson, J., Parrilo, P.A., Willsky, A.S.: Semidefinite descriptions of the convex hull of rotation matrices. SIAM J. Optim. 25(3), 1314–1343 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  32. Sherali, H. D., Adams, W. P.: A Reformulation-Linearization Technique for Solving Discrete and Continuous Nonconvex Problems. Kluwer, (1997)

  33. Smith, S. T.: Geometric Optimization Methods for Adaptive Filtering. PhD thesis, Harvard University, Cambridge, MA, (May 1993)

  34. Stern, R.J., Wolkowicz, H.: Indefinite trust region subproblems and nonsymmetric eigenvalue perturbations. SIAM J. Optim. 5(2), 286–313 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1), 397–434 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wolkowicz, H., Zhao, Q.: Semidefinite programming relaxations for the graph partitioning problem. vol 96/97, pages 461–479. 1999. The satisfiability problem (Certosa di Pontignano, 1996); Boolean functions

  37. **a, Y., Han, Y.-W.: Partial Lagrangian relaxation for the unbalanced orthogonal Procrustes problem. Math. Methods Oper. Res. 79(2), 225–237 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  38. Ye, Y.: A new complexity result on minimization of a quadratic function with a sphere constraint. In: Floudas, C., Pardalos, P. (eds.) Recent Advances in Global Optimization. Princeton University Press, Princeton, NJ (1992)

    Google Scholar 

  39. Zhang, X., Zhu, J., Wen, Z., Zhou, A.: Gradient type optimization methods for electronic structure calculations. SIAM J. Sci. Comput. 36(3), C265–C289 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  40. Zhao, Q., Karisch, S. E., Rendl, F., Wolkowicz, H.: Semidefinite programming relaxations for the quadratic assignment problem. volume 2, pages 71–109. 1998. Semidefinite programming and interior-point approaches for combinatorial optimization problems (Toronto, ON, 1996)

  41. Zohrizadeh, F., Kheirandishfard, M., Kamangar, F., Madani, R.: Non-smooth optimization over stiefel manifolds with applications to dimensionality reduction and graph clustering. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp 1319–1326. International Joint Conferences on Artificial Intelligence Organization, 7 (2019)

Download references

Acknowledgements

We thank Kurt Anstreicher and Laura Balzano for contributing extremely helpful comments on a draft version of this research. In addition, we appreciate the constructive comments and suggestions of the two anonymous referees and handling editor. Finally, we express our thanks to Yong **a for providing a critical reference and Gang Luo for sharing Proposition 2.1 with us.

Author information

Authors and Affiliations

  1. Department of Business Analytics, University of Iowa, Iowa City, IA, 52242-1994, USA

    Samuel Burer & Kyungchan Park

Authors
  1. Samuel Burer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyungchan Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samuel Burer.

Additional information

Communicated by Etienne de Klerk.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Burer, S., Park, K. A Strengthened SDP Relaxation for Quadratic Optimization Over the Stiefel Manifold. J Optim Theory Appl (2023). https://doi.org/10.1007/s10957-023-02168-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10957-023-02168-6

Keywords

Search

Navigation