Abstract
Convex relaxations of nonconvex functions provide useful bounding information in applications such as deterministic global optimization and reachability analysis. In some situations, the original nonconvex functions may not be known explicitly, but are instead described implicitly by nonlinear equation systems. In these cases, established convex relaxation methods for closed-form functions are not directly applicable. This article presents a new general strategy to construct convex relaxations for such implicit functions. These relaxations are described as convex parametric programs whose constraints are convex relaxations of the original residual function. This relaxation strategy is straightforward to implement, produces tight relaxations in practice, is particularly efficient to carry out when monotonicity properties can be exploited, and does not assume the existence or uniqueness of an implicit function on the entire intended domain. Unlike all previous approaches to the authors’ knowledge, this new approach permits any relaxations of the residual function; it does not require the residual relaxations to be factorable or to be obtained from a McCormick-like traversal of a computational graph. This new convex relaxation strategy is extended to inverse functions, compositions involving implicit functions, feasible-set map**s in constraint satisfaction problems, and solutions of parametric ODEs. Based on a proof-of-concept implementation in Julia, numerical examples are presented to illustrate the convex relaxations produced for various implicit functions and optimal-value functions.
Similar content being viewed by others
Data availibility statement
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Code Availability
Our Julia code for our numerical examples is available at https://github.com/kamilkhanlab/implicit-func-relaxations
References
Mitsos, A., Chachuat, B., Barton, P.I.: McCormick-based relaxations of algorithms. SIAM J. Optim. 20(2), 573–601 (2009)
Adjiman, C.S., Dallwig, S., Floudas, C.A., Neumaier, A.: A global optimization method, \(\alpha \)BB, for general twice-differentiable constrained NLPs - I. Theor. Adv. Comput. Chem. Eng. 22(9), 1137–1158 (1998)
McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: part I-convex underestimating problems. Math. Program. 10(1), 147–175 (1976)
Khan, K.A., Watson, H.A.J., Barton, P.I.: Differentiable McCormick relaxations. J. Global Optim. 67(4), 687–729 (2017)
Chachuat, B.: MC++: a toolkit for bounding factorable functions (2014)
Wilhelm, M.E., Stuber, M.D.: EAGO.jl: Easy Advanced Global Optimization in Julia. Optim. Methods Softw. 1–26 (2020)
Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Map**s: A View from Variational Analysis, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer, New York (2014)
Wechsung, A., Scott, J.K., Watson, H.A.J., Barton, P.I.: Reverse propagation of McCormick relaxations. J. Global Optim. 63(1), 1–36 (2015)
Scott, J.K., Stuber, M.D., Barton, P.I.: Generalized McCormick relaxations. J. Global Optim. 51(4), 569–606 (2011)
Stuber, M.D., Scott, J.K., Barton, P.I.: Convex and concave relaxations of implicit functions. Optim. Methods Softw. 30(3), 424–460 (2015)
Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, New York (2000)
Khan, K.A., Wilhelm, M., Stuber, M.D., Cao, H., Watson, H.A.J., Barton, P.I.: Corrections to: differentiable McCormick relaxations. J. Global Optim. 70(3), 705–706 (2018)
Wilhelm, M.E., Le, A.V., Stuber, M.D.: Global optimization of stiff dynamical systems. AIChE J. 65(12), e16836 (2019)
Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA (2008)
Tsoukalas, A., Mitsos, A.: Multivariate McCormick relaxations. J. Global Optim. 59(2–3), 633–662 (2014)
Smith, E.M.B., Pantelides, C.C.: Global optimisation of nonconvex MINLPs. Comput. Chem. Eng. 21, 791–796 (1997)
Scott, J.K., Barton, P.I.: Improved relaxations for the parametric solutions of ODEs using differential inequalities. J. Global Optim. 57(1), 143–176 (2013)
Song, Y., Cao, H., Mehta, C., Khan, K.A.: Bounding convex relaxations of process models from below by tractable black-box sampling. Comput. Chem. Eng. 153, 107413 (2021)
Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I: Fundamentals. Springer, Berlin (1993)
Cao, H., Khan, K.A.: Enclosing reachable sets for nonlinear control systems using complementarity-based intervals. IFAC-PapersOnLine 54(3), 590–595 (2021)
Fiacco, A.V., Kyparisis, J.: Convexity and concavity properties of the optimal value function in parametric nonlinear programming. J. Optim. Theory Appl. 48(1), 95–126 (1986)
Scholtes, S.: Introduction to Piecewise Differentiable Equations. SpringerBriefs in Optimization. Springer, New York (2012)
Hansen, E., Greenberg, R.: An interval Newton method. Appl. Math. Comput. 12(2–3), 89–98 (1983)
Neumaier, A.: Interval Methods for Systems of Equations. Encyclopedia of Mathematics and Its Applications. Cambridge University Press, Cambridge (1991)
Clarke, F.: Optimization and Nonsmooth Analysis. Classics in Applied Mathematics. SIAM, Philadelphia (1990)
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006)
Drud, A.S.: CONOPT - a large-scale GRG code. ORSA J. Comput. 6(2), 207–216 (1994)
Nesterov, Y.: Lectures on Convex Optimization. Springer Optimization and Its Applications. Springer, Cham (2018)
Beckers, M., Mosenkis, V., Naumann, U.: Adjoint mode computation of subgradients for McCormick relaxations. In: Forth, S., Hovland, P., Phipps, E., Utke, J., Walther, A. (eds.) Recent Advances in Algorithmic Differentiation, pp. 103–113. Springer, Berlin (2012)
Khan, K.A.: Branch-locking AD techniques for nonsmooth composite functions and nonsmooth implicit functions. Optim. Methods Softw. 33(4–6), 1127–1155 (2018)
Hansen, E., Sengupta, S.: Bounding solutions of systems of equations using interval analysis. BIT 21(2), 203–211 (1981)
Bompadre, A., Mitsos, A.: Convergence rate of McCormick relaxations. J. Global Optim. 52(1), 1–28 (2012)
Nerantzis, D., Adjiman, C.S.: Tighter \(\alpha \)BB relaxations through a refinement scheme for the scaled Gerschgorin theorem. J. Global Optim. 73(3), 467–483 (2019)
Cao, H., Khan, K.A.: Bounding nonconvex optimal control problems using Pontryagin’s minimum principle (Submitted) (2021)
Song, Y., Khan, K.A.: Optimization-based convex relaxations for nonconvex parametric systems of ordinary differential equations. Math. Program. 196, 521–565 (2022)
Ralph, D., Dempe, S.: Directional derivatives of the solution of a parametric nonlinear program. Math. Program. 70(1–3), 159–172 (1995)
Khan, K.A., Yuan, Y.: Constructing a subgradient from directional derivatives for functions of two variables. J. Nonsmooth Anal. Optim. 6061 (2020)
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NJ (1996)
Gleixner, A.M., Berthold, T., Müller, B., Weltge, S.: Three enhancements for optimization-based bound tightening. J. Global Optim. 67(4), 731–757 (2017)
Zamora, J.M., Grossmann, I.E.: A branch and contract algorithm for problems with concave univariate, bilinear and linear fractional terms. J. Global Optim. 14(3), 217–249 (1999)
Misener, R., Floudas, C.A.: ANTIGONE: Algorithms for coNTinuous/Integer Global Optimization of Nonlinear Equations. J. Global Optim. 59(2–3), 503–526 (2014)
Bezanson, J., Edelman, A., Karpinski, S., Shah, V.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017)
Dunning, I., Huchette, J., Lubin, M.: JuMP: a modeling language for mathematical optimization. SIAM Rev. 59(2), 295–320 (2017)
Atkins, P., de Paula, J.: Physical Chemistry, 8th edn. W. H. Freeman, New York (2006)
Ascher, U.M., Petzold, L.R.: Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, 1st edn. Society for Industrial and Applied Mathematics, USA (1998)
Sahlodin, A.M., Chachuat, B.: Convex/concave relaxations of parametric ODEs using Taylor models. Comput. Chem. Eng. 35(5), 844–857 (2011)
Hartman, P.: Ordinary Differential Equations. SIAM, Philadelphia, PA (2002)
Sahlodin, A.M., Chachuat, B.: Discretize-then-relax approach for convex/concave relaxations of the solutions of parametric ODEs. Appl. Numer. Math. 61(7), 803–820 (2011)
Funding
This work was supported by the McMaster Advanced Control Consortium (MACC), and by the Natural Sciences and Engineering Research Council of Canada (NSERC) under Grant RGPIN-2017-05944.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design, and to the formulation and proofs of mathematical results. Numerical experiments were performed by Huiyi Cao. All authors wrote, edited, read, and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cao, H., Khan, K.A. General convex relaxations of implicit functions and inverse functions. J Glob Optim 86, 545–572 (2023). https://doi.org/10.1007/s10898-023-01281-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-023-01281-0