SpringerLink
Log in

Solving orthogonal group synchronization via convex and low-rank optimization: tightness and landscape analysis

  • Full Length Paper
  • Series A
  • Published:
Mathematical Programming Submit manuscript

Abstract

Group synchronization aims to recover the group elements from their noisy pairwise measurements. It has found many applications in community detection, clock synchronization, and joint alignment problem. This paper focuses on the orthogonal group synchronization which is often used in cryo-EM and computer vision. However, it is generally NP-hard to retrieve the group elements by finding the least squares estimator. In this work, we first study the semidefinite programming (SDP) relaxation of the orthogonal group synchronization and its tightness, i.e., the SDP estimator is exactly equal to the least squares estimator. Moreover, we investigate the performance of the Burer-Monteiro factorization in solving the SDP relaxation by analyzing its corresponding optimization landscape. We provide deterministic sufficient conditions which guarantee: (i) the tightness of SDP relaxation; (ii) optimization landscape arising from the Burer-Monteiro approach is benign, i.e., the global optimum is exactly the least squares estimator and no other spurious local optima exist. Our result provides a solid theoretical justification of why the Burer-Monteiro approach is remarkably efficient and effective in solving the large-scale SDPs arising from orthogonal group synchronization. We perform numerical experiments to complement our theoretical analysis, which gives insights into future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Here we are actually referring to the nonconvexity of \(-f({\varvec{S}}).\)

References

  1. Abbe, E.: Community detection and stochastic block models: recent developments. J. Mach. Learn. Res. 18(1), 6446–6531 (2017)

    MathSciNet  Google Scholar 

  2. Abbe, E., Bandeira, A.S., Bracher, A., Singer, A.: Decoding binary node labels from censored edge measurements: Phase transition and efficient recovery. IEEE Trans. Netw. Sci. Eng. 1(1), 10–22 (2014)

    MathSciNet  Google Scholar 

  3. Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, New Jersey (2009)

    MATH  Google Scholar 

  4. Arie-Nachimson, M., Kovalsky, S.Z., Kemelmacher-Shlizerman, I., Singer, A., Basri, R.: Global motion estimation from point matches. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, pp. 81–88. IEEE (2012)

  5. Baik, J., Arous, G.B., Péché, S., et al.: Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33(5), 1643–1697 (2005)

    MathSciNet  MATH  Google Scholar 

  6. Bandeira, A.S.: Random Laplacian matrices and convex relaxations. Found. Comput. Math. 18(2), 345–379 (2018)

    MathSciNet  MATH  Google Scholar 

  7. Bandeira, A.S., Boumal, N., Singer, A.: Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. Math. Program. 163(1–2), 145–167 (2017)

    MathSciNet  MATH  Google Scholar 

  8. Bandeira, A.S., Boumal, N., Voroninski, V.: On the low-rank approach for semidefinite programs arising in synchronization and community detection. In: Conference on Learning Theory, pp. 361–382 (2016)

  9. Arous, G.B., Mei, S., Montanari, A., Nica, M.: The landscape of the spiked tensor model. Commun. Pure Appl. Math. 72(11), 2282–2330 (2019)

    MathSciNet  MATH  Google Scholar 

  10. Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)

    MATH  Google Scholar 

  11. Benaych-Georges, F., Nadakuditi, R.R.: The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math. 227(1), 494–521 (2011)

    MathSciNet  MATH  Google Scholar 

  12. Boumal, N.: A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints. ar**v preprint ar**v:1506.00575 (2015)

  13. Boumal, N.: Nonconvex phase synchronization. SIAM J. Optim. 26(4), 2355–2377 (2016)

    MathSciNet  MATH  Google Scholar 

  14. Boumal, N., Voroninski, V., Bandeira, A.: The non-convex Burer-Monteiro approach works on smooth semidefinite programs. In: Advances in Neural Information Processing Systems, pp. 2757–2765 (2016)

  15. Boumal, N., Voroninski, V., Bandeira, A.S.: Deterministic guarantees for Burer-Monteiro factorizations of smooth semidefinite programs. Commun. Pure Appl. Math. 73(3), 581–608 (2020)

    MathSciNet  MATH  Google Scholar 

  16. Burer, S., Monteiro, R.D.: A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95(2), 329–357 (2003)

    MathSciNet  MATH  Google Scholar 

  17. Burer, S., Monteiro, R.D.: Local minima and convergence in low-rank semidefinite programming. Math. Program. 103(3), 427–444 (2005)

    MathSciNet  MATH  Google Scholar 

  18. Candes, E.J., Li, X., Soltanolkotabi, M.: Phase retrieval via wirtinger flow: theory and algorithms. IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015)

    MathSciNet  MATH  Google Scholar 

  19. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    MathSciNet  MATH  Google Scholar 

  20. Capitaine, M., Donati-Martin, C., Féral, D.: The largest eigenvalues of finite rank deformation of large wigner matrices: convergence and nonuniversality of the fluctuations. Ann. Probab. 37(1), 1–47 (2009)

    MathSciNet  MATH  Google Scholar 

  21. Chaudhury, K.N., Khoo, Y., Singer, A.: Global registration of multiple point clouds using semidefinite programming. SIAM J. Optim. 25(1), 468–501 (2015)

    MathSciNet  MATH  Google Scholar 

  22. Chen, Y., Candès, E.J.: The projected power method: an efficient algorithm for joint alignment from pairwise differences. Commun. Pure Appl. Math. 71(8), 1648–1714 (2018)

    MathSciNet  MATH  Google Scholar 

  23. Dellaert, F., Rosen, D.M., Wu, J., Mahony, R., Carlone, L.: Shonan rotation averaging: global optimality by surfing SO\((p)^{n}\). In: European Conference on Computer Vision, pp. 292–308. Springer, New York (2020)

    Google Scholar 

  24. Gao, B., Liu, X., Chen, X., Yuan, Y.-X.: A new first-order algorithmic framework for optimization problems with orthogonality constraints. SIAM J. Optim. 28(1), 302–332 (2018)

    MathSciNet  MATH  Google Scholar 

  25. Ge, R., **, C., Zheng, Y.: No spurious local minima in nonconvex low rank problems: A unified geometric analysis. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1233–1242 (2017)

  26. Giridhar, A., Kumar, P.R.: Distributed clock synchronization over wireless networks: algorithms and analysis. In: Proceedings of the 45th IEEE Conference on Decision and Control, pp. 4915–4920. IEEE (2006)

  27. Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM (JACM) 42(6), 1115–1145 (1995)

    MathSciNet  MATH  Google Scholar 

  28. Huang, Q.-X., Guibas, L.: Consistent shape maps via semidefinite programming. Comput. Graph. Forum 32(5), 177–186 (2013)

    Google Scholar 

  29. Iwen, M.A., Preskitt, B., Saab, R., Viswanathan, A.: Phase retrieval from local measurements: improved robustness via eigenvector-based angular synchronization. Appl. Comput. Harmon. Anal. 48(1), 415–444 (2020)

    MathSciNet  MATH  Google Scholar 

  30. Jung, J.H., Chung, H.W., Lee, J.O.: Weak detection in the spiked wigner model with general rank. ar**v preprint ar**v:2001.05676 (2020)

  31. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56(6), 2980–2998 (2010)

    MathSciNet  MATH  Google Scholar 

  32. Lerman, G., Shi, Y.: Robust group synchronization via cycle-edge message passing. Found. Comput. Math. (2021)

  33. Ling, S.: Generalized power method for generalized orthogonal Procrustes problem: global convergence and optimization landscape analysis. ar**v preprint ar**v:2106.15493 (2021)

  34. Ling, S.: Improved performance guarantees for orthogonal group synchronization via generalized power method. SIAM J. Optim 32(2), 1018–1048 (2022). https://doi.org/10.1137/20M1389571

    Article  MathSciNet  MATH  Google Scholar 

  35. Ling, S.: Near-optimal performance bounds for orthogonal and permutation group synchronization via spectral methods. Appl. Comput. Harmon. Anal. 60, 20–52 (2022)

    MathSciNet  MATH  Google Scholar 

  36. Ling, S., Xu, R., Bandeira, A.S.: On the landscape of synchronization networks: a perspective from nonconvex optimization. SIAM J. Optim. 29(3), 1879–1907 (2019)

    MathSciNet  MATH  Google Scholar 

  37. Liu, H., Yue, M.-C., So, A.M.-C.: On the estimation performance and convergence rate of the generalized power method for phase synchronization. SIAM J. Optim. 27(4), 2426–2446 (2017)

    MathSciNet  MATH  Google Scholar 

  38. Liu, H., Yue, M.-C., So, A.M.-C.: A unified approach to synchronization problems over subgroups of the orthogonal group. ar**v preprint ar**v:2009.07514 (2020)

  39. Markdahl, J., Thunberg, J., Goncalves, J.: Almost global consensus on the \( n \)-sphere. IEEE Trans. Autom. Control 63(6), 1664–1675 (2017)

    MathSciNet  MATH  Google Scholar 

  40. Markdahl, J., Thunberg, J., Goncalves, J.: High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globally. Automatica 113, 108736 (2020)

    MathSciNet  MATH  Google Scholar 

  41. Mei, S., Misiakiewicz, T., Montanari, A., Oliveira, R.I.: Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality. In: Conference on Learning Theory, pp. 1476–1515 (2017)

  42. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87. Springer, New York (2013)

    MATH  Google Scholar 

  43. Pachauri, D., Kondor, R., Singh, V.: Solving the multi-way matching problem by permutation synchronization. In: Advances in Neural Information Processing Systems, pp. 1860–1868 (2013)

  44. Pataki, G.: On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues. Math. Oper. Res. 23(2), 339–358 (1998)

    MathSciNet  MATH  Google Scholar 

  45. Perry, A., Wein, A.S., Bandeira, A.S., Moitra, A.: Optimality and sub-optimality of PCA I: spiked random matrix models. Ann. Stat. 46(5), 2416–2451 (2018)

    MathSciNet  MATH  Google Scholar 

  46. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)

    MathSciNet  MATH  Google Scholar 

  47. Rosen, D.M., Carlone, L., Bandeira, A.S., Leonard, J.J.: Se-sync: a certifiably correct algorithm for synchronization over the special Euclidean group. Int. J. Robot. Res. 38(2–3), 95–125 (2019)

    Google Scholar 

  48. Singer, A.: Angular synchronization by eigenvectors and semidefinite programming. Appl. Comput. Harmon. Anal. 30(1), 20–36 (2011)

    MathSciNet  MATH  Google Scholar 

  49. Singer, A., et al.: Mathematics for cryo-electron microscopy. Proc. Int. Congr. Math. (ICM) 3, 3981–4000 (2018)

    MATH  Google Scholar 

  50. Singer, A., Shkolnisky, Y.: Three-dimensional structure determination from common lines in cryo-em by eigenvectors and semidefinite programming. SIAM J. Imaging Sci. 4(2), 543–572 (2011)

    MathSciNet  MATH  Google Scholar 

  51. Sun, D., Toh, K.-C., Yuan, Y., Zhao, X.-Y.: SDPNAL+: a Matlab software for semidefinite programming with bound constraints (version 1.0). Optim. Methods Softw. 35(1), 87–115 (2020)

    MathSciNet  MATH  Google Scholar 

  52. Sun, J., Qu, Q., Wright, J.: Complete dictionary recovery over the sphere i: overview and the geometric picture. IEEE Trans. Inf. Theory 63(2), 853–884 (2016)

    MathSciNet  MATH  Google Scholar 

  53. Sun, J., Qu, Q., Wright, J.: A geometric analysis of phase retrieval. Found. Comput. Math. 18(5), 1131–1198 (2018)

    MathSciNet  MATH  Google Scholar 

  54. Tütüncü, R.H., Toh, K.-C., Todd, M.J.: Solving semidefinite-quadratic-linear programs using SDPT3. Math. Program. 95(2), 189–217 (2003)

    MathSciNet  MATH  Google Scholar 

  55. Vershynin, R.: High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47. Cambridge University Press, Cambridge (2018)

    MATH  Google Scholar 

  56. Waldspurger, I., Waters, A.: Rank optimality for the Burer-Monteiro factorization. SIAM J. Optim. 30(3), 2577–2602 (2020)

    MathSciNet  MATH  Google Scholar 

  57. Wang, L., Singer, A.: Exact and stable recovery of rotations for robust synchronization. Inf. Inference J. IMA 2(2), 145–193 (2013)

    MathSciNet  MATH  Google Scholar 

  58. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1–2), 397–434 (2013)

    MathSciNet  MATH  Google Scholar 

  59. Xu, R.: On the Landscape of phase synchronization. Master’s Thesis. New York University (2019)

  60. Yang, L., Sun, D., Toh, K.-C.: SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints. Math. Program. Comput. 7(3), 331–366 (2015)

    MathSciNet  MATH  Google Scholar 

  61. Yurtsever, A., Tropp, J.A., Fercoq, O., Udell, M., Cevher, V.: Scalable semidefinite programming. SIAM J. Math. Data Sci. 3(1), 171–200 (2021)

    MathSciNet  MATH  Google Scholar 

  62. Zhang, T.: Tightness of the semidefinite relaxation for orthogonal trace-sum maximization. ar**v preprint ar**v:1911.08700 (2019)

  63. Zhong, Y., Boumal, N.: Near-optimal bounds for phase synchronization. SIAM J. Optim. 28(2), 989–1016 (2018)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. New York University Shanghai, 1555 Century Avenue, Shanghai, 200122, China

    Shuyang Ling

Authors
  1. Shuyang Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuyang Ling.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is (partially) financially supported by the National Key R &D Program of China, Project Number 2021YFA1002800, National Natural Science Foundation of China (NSFC) No.12001372, Shanghai Municipal Education Commission (SMEC) via Grant 0920000112, and NYU Shanghai Boost Fund.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ling, S. Solving orthogonal group synchronization via convex and low-rank optimization: tightness and landscape analysis. Math. Program. 200, 589–628 (2023). https://doi.org/10.1007/s10107-022-01896-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-022-01896-3

Keywords

Mathematics Subject Classification

Search

Navigation