Abstract
We consider the problem of predicting the covariance of a zero mean Gaussian vector, based on another feature vector. We describe a covariance predictor that has the form of a generalized linear model, i.e., an affine function of the features followed by an inverse link function that maps vectors to symmetric positive definite matrices. The log-likelihood is a concave function of the predictor parameters, so fitting the predictor involves convex optimization. Such predictors can be combined with others, or recursively applied to improve performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Robert Tibshirani, personal communication.
References
Anderson T (1973) Asymptotically efficient estimation of covariance matrices with linear structure. Ann Stat 1(1):135–141
Anscombe J (1961) Examination of residuals. In: Proceedings of the Berkeley symposium on mathematical statistics and probability
Asuncion A, Newman D (2007) UCI machine learning repository
Bilmes J (1998) A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Int Comput Sci Inst 4(510):126
Bollerslev T (1986) Generalized autoregressive conditional heteroskedasticity. J Econom 31(3):307–327
Bollerslev T (1990) Modelling the coherence in short-run nominal exchange rates: a multivariate generalized ARCH model. Rev Econom Stat, pp 498–505
Bollerslev T, Engle R, Wooldridge J (1988) A capital asset pricing model with time-varying covariances. J Polit Econ 96(1):116–131
Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge
Chiu T, Leonard T, Tsui K-W (1996) The matrix-logarithmic covariance model. J Am Stat Assoc 91(433):198–210
Cleveland W, Devlin S (1988) Locally-weighted regression: an approach to regression analysis by local fitting. J Am Stat Assoc 83(403):596–610
Cook D, Weisberg S (1983) Diagnostics for heteroscedasticity in regression. Biometrika 70(1):1–10
Davidian M, Carroll R (1987) Variance function estimation. J Am Stat Assoc 82(400):1079–1091
Dempster A (1972) Covariance selection. Biometrics, pp 157–175
Dorta G, Vicente S, Agapito L, Campbell N, Simpson I (2018) Structured uncertainty prediction networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5477–5485
Engle R (1982) Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econom J Econom Soc, pp 987–1007
Engle R, Kroner K (1995) Multivariate simultaneous generalized ARCH. Econom Theory, pp 122–150
Fama E, French K (1992) The cross-section of expected stock returns. J Finance 47(2):427–465
Francq C, Zakoian J-M (2019) GARCH models: structure, statistical inference and financial applications. John Wiley and Sons, New Jersey
Freund Y, Schapire R (1996) Experiments with a new boosting algorithm. In: icml, vol 96, pp 148–156. Citeseer
Friedman J, Hastie T, Tibshirani R (2008) Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3):432–441
Grinold R, Kahn R (2000) Active portfolio management. McGraw Hill, New York
Harper D (2009) Exploring the exponentially weighted moving average. Investopedia
Hawkins D, Maboudou-Tchao E (2008) Multivariate exponentially weighted moving covariance matrix. Technometrics 50(2):155–166
Heiden M (2015) Pitfalls of the Cholesky decomposition for forecasting multivariate volatility. Available at SSRN 2686482
Huang J, Liu N, Pourahmadi M, Liu L (2006) Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93(1):85–98
Liu D, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math Program 45(1–3):503–528
Longerstaey J, Spencer M (1996) Riskmetrics – Technical Document. JP Morgan and Reuters
Meier L, Van De Geer S, Bühlmann P (2008) The group lasso for logistic regression. J R Stat Soc Ser B (Stat Methodol) 70(1):53–71
Menchero J, Orr DJ, Wang J (2011) The Barra US equity model (USE4), methodology notes. MSCI Barra
Jianxin P (2021) jmcm: Joint mean-covariance models using ‘Armadillo’ and S4. R package version 0.2.4
Posthuma P (2019) lmvar: Linear Regression with Non-Constant Variances. R package version 1.5.2
Pourahmadi M (1999) Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika 86(3):677–690
Pourahmadi M (2000) Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika 87(2):425–435
Pourahmadi M (2011) Covariance estimation: The GLM and regularization perspectives. Stat Sci, pp 369–387
Recht B, Fazel M, Parrilo P (2010) Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev 52(3):471–501
Redmond M, Baveja A (2002) A data-driven software tool for enabling cooperative information sharing among police departments. Eur J Oper Res 141:660–678
Rothman A, Levina E, Zhu J (2010) A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika 97(3):539–550
Rubin D, Thayer D (1982) EM algorithms for ML factor analysis. Psychometrika 47(1):69–76
Liangjun S, Wang X (2017) On time-varying factor models: estimation and testing. J Econom 198(1):84–101
Tuck J, Barratt S, Boyd S (2021) A distributed method for fitting Laplacian regularized stratified models. J Mach Learn Res 22:60–1
Tuck J, Barratt S, Boyd S (2021) Portfolio construction using stratified models. ar**v preprint ar**v:2101.04113
Tuck J, Boyd S (2020) Fitting Laplacian regularized stratified gaussian models. ar**v preprint ar**v:2005.01752
Bureau of the Census US Department of Commerce. Census of Population and Housing 1990 United States: Summary tape file 1a and 3a (computer files)
Bureau of Justice Statistics US Department of Justice (1992) Law enforcement management and administrative statistics (computer file)
Federal Bureau of Investigation US Department of Justice (1995) Crime in the United States (computer file)
Vandenberghe L, Boyd S (1996) Semidefinite programming. SIAM Rev 38(1):49–95
Williams P (1996) Using neural networks to model conditional multivariate densities. Neural Comput 8(4):843–854
Williams P (1999) Matrix logarithm parametrizations for neural network covariance models. Neural Netw 12(2):299–308
Wei W, Pourahmadi M (2003) Nonparametric estimation of large covariance matrices of longitudinal data. Biometrika 90(4):831–844
Acknowledgements
The authors gratefully acknowledge conversations and discussions about some of the material in this paper with Misha van Beek, Linxi Chen, David Greenberg, Ron Kahn, Trevor Hastie, Rob Tibshirani, Emmanuel Candes, Mykel Kochenderfer, and Jonathan Tuck.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Barratt, S., Boyd, S. Covariance prediction via convex optimization. Optim Eng 24, 2045–2078 (2023). https://doi.org/10.1007/s11081-022-09765-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11081-022-09765-w