SpringerLink
Log in

Approximate Real Symmetric Tensor Rank

  • Research Exposition
  • Published:
Arnold Mathematical Journal Aims and scope Submit manuscript

Abstract

We investigate the effect of an \(\varepsilon \)-room of perturbation tolerance on symmetric tensor decomposition. To be more precise, suppose a real symmetric d-tensor f, a norm \(\left\Vert \cdot \right\Vert \) on the space of symmetric d-tensors, and \(\varepsilon >0\) are given. What is the smallest symmetric tensor rank in the \(\varepsilon \)-neighborhood of f? In other words, what is the symmetric tensor rank of f after a clever \(\varepsilon \)-perturbation? We prove two theorems and develop three corresponding algorithms that give constructive upper bounds for this question. With expository goals in mind, we present probabilistic and convex geometric ideas behind our results, reproduce some known results, and point out open problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3

Similar content being viewed by others

References

  1. Acar, E., Yener, B.: Unsupervised multiway data analysis: a literature survey. IEEE Trans. Knowl. Data Eng. 21(1), 6–20 (2008)

    Article  Google Scholar 

  2. Albiac, F., Kalton, N.J.: Topics in Banach Space Theory, Graduate Texts in Mathematics, vol. 233. Springer, New York (2006)

    MATH  Google Scholar 

  3. Anandkumar, A., Ge, R., Hsu, D., Kakade, S.M., Telgarsky, M.: Tensor decompositions for learning latent variable models. J. Mach. Learn. Res. 15, 2773–2832 (2014)

    MathSciNet  MATH  Google Scholar 

  4. Aubrun, G., Szarek, S.J.: Alice and Bob meet Banach, Mathematical Surveys and Monographs, vol. 223. American Mathematical Society, Providence (2017). The interface of asymptotic geometric analysis and quantum information theory

  5. Barman, S.: Approximating Nash equilibria and dense subgraphs via an approximate version of Carathéodory’s theorem. SIAM J. Comput. 47(3), 960–981 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  6. Barvinok, A.: Estimating \({L}_{\infty }\) norms by \({L}_{2k}\) norms for functions on orbits. Found. Comput. Math. 2(4), 393–412 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  7. Blekherman, G., Teitler, Z.: On maximum, typical and generic ranks. Math. Ann. 362(3), 1021–1031 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Brachat, J., Comon, P., Mourrain, B., Tsigaridas, E.: Symmetric tensor decomposition. Linear Algebra Appl. 433(11–12), 1851–1872 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Brambilla, M.C., Ottaviani, G.: On the Alexander–Hirschowitz theorem. J. Pure Appl. Algebra 212, 1229–1251 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  10. Combettes, C.W., Pokutta, S.: Revisiting the approximate Caratheodory problem via the Frank–Wolfe algorithm, ar**v preprint. ar**v:1911.04415 (2019)

  11. Comon, P.: Independent component analysis, a new concept? Signal Process. 36(3), 287–314 (1994)

    Article  MATH  Google Scholar 

  12. Comon, P., Golub, G., Lim, L.-H., Mourrain, B.: Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl. 30(3), 1254–1279 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Conner, A., Gesmundo, F., Landsberg, J.M., Ventura, E.: Rank and border rank of Kronecker powers of tensors and Strassen’s laser method. Comput. Complex. 31(1), 1–40 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  14. Cristancho, S., Velasco, M.: Harmonic hierarchies for polynomial optimization, ar**v preprint. ar**v:2202.12865 (2022)

  15. Cucker, F., Ergür, A.A., Tonelli-Cueto, J.: Functional norms, condition numbers and numerical algorithms in algebraic geometry, ar**v preprint. ar**v:2102.11727 (2021)

  16. de la Vega, W.F., Karpinski, M., Kannan, R., Vempala, S.: Tensor decomposition and approximation schemes for constraint satisfaction problems. In: Proceedings of the Thirty-Seventh Annual ACM Symposium on Theory of Computing, pp. 747–754 (2005)

  17. De las Cuevas, G., Klingler, A., Netzer, T.: Approximate tensor decompositions: disappearance of many separations. J. Math. Phys. 62(9), 093502 (2021)

  18. Derksen, H.: On the nuclear norm and the singular value decomposition of tensors. Found. Comput. Math. 16(3), 779–811 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  19. Ergür, A.A.: Approximating nonnegative polynomials via spectral sparsification. SIAM J. Optim. 29(1), 852–873 (2019)

  20. Fang, K., Fawzi, H.: The sum-of-squares hierarchy on the sphere and applications in quantum information theory. Math. Program. 190(1), 331–360 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  21. Fornasier, M., Klock, T., Rauchensteiner, M.: Robust and resource-efficient identification of two hidden layer neural networks. Constr. Approx. 55(1), 475–536 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  22. Friedland, S., Lim, L.-H.: Nuclear norm of higher-order tensors. Math. Comput. 87(311), 1255–1281 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Ge, R.: Tensor methods in machine learning. http://www.offconvex.org/2015/12/17/tensor-decompositions/ (2015)

  24. Gowers, W.T.: Decompositions, approximate structure, transference, and the Hahn–Banach theorem. Bull. Lond. Math. Soc. 42(4), 573–606 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  25. Hale, N., Townsend, A.: Fast and accurate computation of Gauss–Legendre and Gauss–Jacobi quadrature nodes and weights. SIAM J. Sci. Comput. 35(2), A652–A674 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hillar, C.J, Lim, L.-H.: Most tensor problems are NP-Hard. J. ACM (JACM) 60(6), 1–39 (2013)

  27. Ivanov, G.: Approximate Carathéodory’s theorem in uniformly smooth Banach spaces. Discrete Comput. Geom. 66(1), 273–280 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  28. Kileel, J., Klock, T., Pereira, J.M.: Landscape analysis of an improved power method for tensor decomposition. Adv. Neural Inf. Process. Syst. 34, 6253–6265 (2021)

    Google Scholar 

  29. Kolda, T.G., Mayo, J.R.: An adaptive shifted power method for computing generalized tensor eigenpairs. SIAM J. Matrix Anal. Appl. 35(4), 1563–1581 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  30. Landsberg, J.M., Teitler, Z.: On the ranks and border ranks of symmetric tensors. Found. Comput. Math. 10(3), 339–366 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ledoux, M., Talagrand, M.: Probability in Banach spaces, Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)], vol. 23. Springer, Berlin (1991). Isoperimetry and processes

  32. Lim, L.-H.: Tensors in computations. Acta Numer. 30, 555–764 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  33. Lovász, L., Szegedy, B.: Szemerédi’s lemma for the analyst. GAFA Geom. Funct. Anal. 17(1), 252–270 (2007)

    Article  MATH  Google Scholar 

  34. Moitra, A.: Algorithmic Aspects of Machine Learning. Cambridge University Press, Cambridge (2018)

    Book  MATH  Google Scholar 

  35. Nie, J.: Low rank symmetric tensor approximations. SIAM J. Matrix Anal. Appl. 38(4), 1517–1540 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  36. Nie, J.: Symmetric tensor nuclear norms. SIAM J. Appl. Algebra Geom. 1(1), 599–625 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  37. Oymak, S., Soltanolkotabi, M.: Learning a deep convolutional neural network via tensor decomposition. Inf. Inference J. IMA 10(3), 1031–1071 (2021)

    MathSciNet  MATH  Google Scholar 

  38. Panagakis, Y., Kossaifi, J., Chrysos, G.G., Oldfield, J., Nicolaou, M.A., Anandkumar, A.: Tensor methods in computer vision and deep learning. Proc. IEEE 109(5), 863–890 (2021)

    Article  Google Scholar 

  39. Pisier, G.: Remarques sur un résultat non publié de B. Maurey, Séminaire d’Analyse fonctionnelle (dit “Maurey-Schwartz”) (1980–1981), talk:5

  40. Pratt, K.: Waring rank, parameterized and exact algorithms. In: 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS). IEEE, pp. 806–823 (2019)

  41. Sanyal, R., Sottile, F., Sturmfels, B.: Orbitopes. Mathematika 57(2), 275–314 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  42. Schechtman, G., Zinn, J.: On the volume of the intersection of two \(L^n_p\) balls. Proc. Am. Math. Soc. 110(1), 217–224 (1990)

    MATH  Google Scholar 

  43. Sidiropoulos, N.D., De Lathauwer, L., Fu, X., Huang, K., Papalexakis, E.E., Faloutsos, C.: Tensor decomposition for signal processing and machine learning. IEEE Trans. Signal Process. 65(13), 3551–3582 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  44. Spielman, D.A.: The smoothed analysis of algorithms. In: International Symposium on Fundamentals of Computation Theory. Springer, pp. 17–18 (2005)

  45. Tao, T.: Structure and Randomness. American Mathematical Society, Providence (2008). Pages from year one of a mathematical blog

  46. Tomczak-Jaegermann, N.: Banach–Mazur distances and finite-dimensional operator ideals. Pitman Monographs and Surveys in Pure and Applied Mathematics, vol. 38, Pure and Applied Mathematics, vol. 38, 395 (1989)

  47. Vershynin, R.: High-dimensional probability, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 47. Cambridge University Press, Cambridge (2018). An introduction with applications in data science, With a foreword by Sara van de Geer

Download references

Acknowledgements

We thank Jiawang Nie for answering our questions on optimization of low-rank symmetric tensors using sum of squares. We thank Sergio Cristancho and Mauricio Velasco for explaining the mathematical underpinning of their quadrature rule in [14], and allowing us to implement it in Python. We thank Carlos Castro Rey for his valuable help and feedback on the Python implementation. A.E. was partially supported by NSF CCF 2110075. P.V. is supported by Simons Foundation grant 638224.

Author information

Authors and Affiliations

  1. Mathematics and Computer Science, The University of Texas at San Antonio, San Antonio, Texas, USA

    Alperen A. Ergür

  2. Mathematics & Electrical Enginering and Computer Science, University of Missouri, Columbia, Missouri, USA

    Petros Valettas

  3. Mathematics, The University of Texas at San Antonio, San Antonio, Texas, USA

    Jesus Rebollo Bueno

Authors
  1. Alperen A. Ergür
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesus Rebollo Bueno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petros Valettas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alperen A. Ergür.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ergür, A.A., Rebollo Bueno, J. & Valettas, P. Approximate Real Symmetric Tensor Rank. Arnold Math J. 9, 455–480 (2023). https://doi.org/10.1007/s40598-023-00235-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40598-023-00235-4

Keywords

Search

Navigation