Abstract
Many machine learning tasks can be formulated as an optimization problem given in the form of
where f, x, and X denote the objective function, decision variables, and feasible set, respectively. Unfortunately, solving an optimization problem is challenging. In general, we cannot guarantee whether one can find an optimal solution, and if so, how much computational effort one needs. However, it turns out that we can provide such guarantees for a special but broad class optimization problems, namely convex optimization, where X is a convex set and f is a convex function. In fact, many machine learning models we formulated so far, such as least square linear regression, logistic regression, and support vector machine, are convex optimization problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D. Bertsekas, Nonlinear Programming, 2nd edn. (Athena Scientific, New York, 1999)
S. Boyd, L. Vandenberghe, Convex Optimization. (Cambridge University Press, Cambridge, 2004)
J.-B. Hiriart-Urruty, C. Lemaréchal, Convex Analysis and Minimization Algorithms I. Comprehensive Study in Mathematics, vol. 305 (Springer, New York, 1993)
D.G. Luenberger, Y. Ye, Linear and Nonlinear Programming (Springer, New York, 2008)
A.S. Nemirovski, Optimization III (Georgia Tech, 2013). https://www2.isye.gatech.edu/~nemirovs/OPTIII_LectureNotes2018.pdf
Y.E. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course (Kluwer Academic Publishers, Boston, 2004)
J. Nocedal, S.J. Wright, Numerical Optimization (Springer, New York, 1999)
B. Polyak, Introduction to Optimization. (Optimization Software, New York, 1987)
R.T. Rockafellar, Convex Analysis (Princeton University Press, Princeton, 1970)
A. Ruszczyński, Nonlinear Optimization, 1st edn. (Princeton University Press, Princeton, 2006)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lan, G. (2020). Convex Optimization Theory. In: First-order and Stochastic Optimization Methods for Machine Learning. Springer Series in the Data Sciences. Springer, Cham. https://doi.org/10.1007/978-3-030-39568-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-39568-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39567-4
Online ISBN: 978-3-030-39568-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)