Abstract
Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn et al. (Math Program 130(1):177–209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. A computational study on two example problems demonstrates that using a two-stage LDR can yield significantly better primal policies and modestly better dual policies than using policies based on a static LDR.
Similar content being viewed by others
References
Ahmed, S.: Multistage stochastic optimization (2016). https://www.ima.umn.edu/materials/2015-2016/ND8.1-12.16/25386/mssp.pdf. New Directions Short Course on Mathematical Optimization
Bakir, I., Boland, N., Dandurand, B., Erera, A.: Scenario set partition dual bounds for multistage stochastic programming: a hierarchy of bounds and a partition sampling approach (2016). http://www.optimization-online.org/DB_FILE/2016/01/5311.pdf
Bampou, D., Kuhn, D.: Scenario-free stochastic programming with polynomial decision rules. In: 2011 50th IEEE Conference on Decision and Control and European Control Conference, pp. 7806–7812. IEEE (2011)
Ben-Tal, A., Goryashko, A., Guslitzer, E., Nemirovski, A.: Adjustable robust solutions of uncertain linear programs. Math. Program. 99(2), 351–376 (2004)
Ben-Tal, A., Nemirovski, A.: Robust optimization—methodology and applications. Math. Program. 92, 453–480 (2002)
Bertsimas, D., Caramanis, C.: Adaptability via sampling. In: 2007 46th IEEE Conference on Decision and Control, pp. 4717–4722. IEEE (2007)
Bertsimas, D., Georghiou, A.: Design of near optimal decision rules in multistage adaptive mixed-integer optimization. Oper. Res. 63(3), 610–627 (2015)
Birge, J.: Decomposition and partitioning methods for multistage stochastic linear programs. Oper. Res. 33(5), 989–1007 (1985)
Birge, J.R.: Aggregation bounds in stochastic linear programming. Math. Program. 31(1), 25–41 (1985)
Birge, J.R., Donohue, C.J., Holmes, D.F., Svintsitski, O.G.: A parallel implementation of the nested decomposition algorithm for multistage stochastic linear programs. Math. Program. 75(2), 327–352 (1996)
Calafiore, G., Campi, M.: The scenario approach to robust control design. IEEE Trans. Automat. Contr. 51, 742–753 (2006)
Casey, M.S., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Math. Oper. Res. 30(3), 615–631 (2005)
Chen, X., Sim, M., Sun, P., Zhang, J.: A linear decision-based approximation approach to stochastic programming. Oper. Res. 56(2), 344–357 (2008)
Chen, Z.L., Powell, W.: Convergent cutting-plane and partial-sampling algorithm for multistage stochastic linear programs with recourse. J. Optim. Theory Appl. 102(3), 497–524 (1999)
de Maere d’Aertrycke, G., Shapiro, A., Smeers, Y.: Risk exposure and Lagrange multipliers of nonanticipativity constraints in multistage stochastic problems. Math. Methods Oper. Res. 77(3), 393–405 (2013)
Dowson, O., Kapelevich, L.: SDDP.jl: a Julia package for stochastic dual dynamic programming. Optimization Online (2017). http://www.optimization-online.org/DB_HTML/2017/12/6388.html
Dyer, M., Stougie, L.: Computational complexity of stochastic programming problems. Math. Program. 106(3), 423–432 (2006)
Eisner, M., Olsen, P.: Duality for stochastic programming interpreted as L.P. in \(L_p\)-space. SIAM J. Appl. Math 28(4), 779–792 (1975)
Fábián, C.I., Szőke, Z.: Solving two-stage stochastic programming problems with level decomposition. Comput. Manag. Sci. 4, 313–353 (2007)
Fu, M.C.: Feature article: optimization for simulation: theory vs. practice. INFORMS J. Comput. 14(3), 192–215 (2002)
Garstka, S., Wets, R.: On decision rules in stochastic programming. Math. Program. 7(1), 117–143 (1974)
Gassmann, H.I.: Mslip: a computer code for the multistage stochastic linear programming problem. Math. Program. 47(1), 407–423 (1990)
Georghiou, A., Wiesemann, W., Kuhn, D.: Generalized decision rule approximations for stochastic programming via liftings. Math. Program. 152(1–2), 301–338 (2015)
Girardeau, P., Leclere, V., Philpott, A.: On the convergence of decomposition methods for multistage stochastic convex programs. Math. Oper. Res. 40, 130–145 (2015)
Guigues, V.: SDDP for some interstage dependent risk-averse problems and application to hydro-thermal planning. Comput. Optim. Appl. 57, 167–203 (2014)
Guigues, V.: Convergence analysis of sampling-based decomposition methods for risk-averse multistage stochastic convex programs. SIAM J. Optim. 26, 2468–2494 (2016)
Hanasusanto, G.A., Kuhn, D., Wiesemann, W.: A comment on “computational complexity of stochastic programming problems”. Math. Program. 159(1), 557–569 (2016)
Heitsch, H., Römisch, W.: Scenario tree modeling for multistage stochastic programs. Math. Program. 118(2), 371–406 (2009)
Higle, J., Sen, S.: Multistage stochastic convex programs: duality and its implications. Ann. Oper. Res. 142(1), 129–146 (2006)
Homem de Mello, T.: On rates of convergence for stochastic optimization problems under non-independent and identically distributed sampling. SIAM J. Optim. 19, 524–551 (2008)
Hong, L.J., Nelson, B.L.: A brief introduction to optimization via simulation. In: Winter Simulation Conference, WSC ’09, Winter Simulation Conference, pp. 75–85 (2009)
Høyland, K., Wallace, S.: Generating scenario trees for multistage decision problems. Manag. Sci. 47(2), 295–307 (2001)
Infanger, G., Morton, D.: Cut sharing for multistage stochastic linear programs with interstage dependency. Math. Program. 75(2), 241–256 (1996)
Koivu, M.: Variance reduction in sample approximations of stochastic programs. Math. Program. 103, 463–485 (2005)
Kuhn, D., Wiesemann, W., Georghiou, A.: Primal and dual linear decision rules in stochastic and robust optimization. Math. Program. 130(1), 177–209 (2011)
Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)
Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69, 111–147 (1995)
Linderoth, J., Shapiro, A., Wright, S.: The empirical behavior of sampling methods for stochastic programming. Ann. Oper. Res. 142, 215–241 (2006)
Mak, W.K., Morton, D., Wood, R.: Monte Carlo bounding techniques for determining solution quality in stochastic programs. Oper. Res. Lett. 24, 47–56 (1999)
Nemirovski, A., Juditsky, A., Lan, G., Shapiro, A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4), 1574–1609 (2009)
Pennanen, T.: Epi-convergent discretizations of multistage stochastic programs via integration quadratures. Math. Program. 116, 461–479 (2009)
Pereira, M., Pinto, L.: Multi-stage stochastic optimization applied to energy planning. Math. Program. 52(1–3), 359–375 (1991)
Philpott, A., De Matos, V.: Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion. Eur. J. Oper. Res. 218(2), 470–483 (2012)
Philpott, A., Guan, Z.: On the convergence of stochastic dual dynamic programming and related methods. Oper. Res. Lett. 36(4), 450–455 (2008)
Philpott, A., Pritchard, G.: Emi-doasa. Technical report, Electric Power Optimization Centre (2013). http://www.emi.ea.govt.nz/Content/Tools/Doasa/DOASA%20paper%20by%20SOL.pdf
Polyak, B., Juditsky, A.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30, 838–855 (1992)
Powell, W.: Approximate Dynamic Programming: Solving the Curses of Dimensionality, vol. 703. Wiley, New York (2007)
Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)
Rockafellar, R., Wets, R.B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16(1), 119–147 (1991)
Ruszczyński, A.: A regularized decomposition method for minimizing a sum of polyhedral functions. Math. Program. 35(3), 309–333 (1986)
Sen, S., Zhou, Z.: Multistage stochastic decomposition: a bridge between stochastic programming and approximate dynamic programming. SIAM J. Optim. 24(1), 127–153 (2014)
Shapiro, A.: Stochastic programming approach to optimization under uncertainty. Math. Program. 112, 183–220 (2008)
Shapiro, A.: Analysis of stochastic dual dynamic programming method. Eur. J. Oper. Res. 209(1), 63–72 (2011)
Shapiro, A.: Topics in stochastic programming. Universite Catholique de Louvain, CORE Lecture Series (2011)
Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on stochastic programming: modeling and theory, vol. 16. SIAM (2014)
Shapiro, A., Homem-de Mello, T.: On the rate of convergence of optimal solutions of monte carlo approximations of stochastic programs. SIAM J. Optim. 11(1), 70–86 (2000)
Shapiro, A., Nemirovski, A.: On complexity of stochastic programming problems. In: Jeyakumar, V., Rubinov, A. (eds.) Continuous Optimization, pp. 111–146. Springer, Heidelberg (2005)
Shapiro, A., Tekaya, W., da Paulo Costa, J., Pereira Soares, M.: Risk neutral and risk averse stochastic dual dynamic programming method. Eur. J. Oper. Res. 224(2), 375–391 (2013)
Van Slyke, R.M., Wets, R.: L-shaped linear programs with applications to optimal control and stochastic programming. SIAM J. Appl. Math. 17(4), 638–663 (1969)
Vayanos, P., Kuhn, D., Rustem, B.: A constraint sampling approach for multi-stage robust optimization. Automatica 48(3), 459–471 (2012)
Wallace, S.W., Ziemba, W.T.: Applications of Stochastic Programming, vol. 5. SIAM, Philadelphia (2005)
Acknowledgements
The research of James Luedtke has been supported in part by the National Science Foundation under Grant CMMI-1634597, and by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under Contract Number DE-AC02-06CH11347. The research of Merve Bodur has been supported in part by the Natural Sciences and Engineering Research Council of Canada under the Discovery Grant RGPIN-2018-04984. We would like to thank the reviewers for their careful and constructive comments that have significantly improved the paper. Also, we would like to thank Angelos Georghiou for providing data and clarifications on the inventory planning example, and Oscar Dowson for his assistance using the sddp.jl code.
Author information
Authors and Affiliations
Corresponding author
Appendices
Data for the capacity expansion problem
Additional models for the capacity expansion example
1.1 Primal model and Benders decomposition
P-LDR-2S of the capacity expansion model is obtained by substituting
in (17). Drop** \(\xi ^t\) dependences for variables to simplify the notation, we obtain
where, for \(t \in [2,T]\), \(Q_t({\beta },\xi ^{t})\) is defined as the optimal objective value of the following problem:
We note that P-LDR-2S of the capacity expansion model does not have relatively complete recourse since the recourse constraints (20i) and (20j) might be violated under some scenarios.
Let \(\xi ^{T}_n, n \in [N]\), be an independent and identically distributed (i.i.d.) random sample of the random vector \(\xi ^{T}\). We solve the obtained primal SAA problem with Benders decomposition, using a single aggregate cut per time-stage. That is, we have a master problem of the following form:
The variable \(\eta _t\) represents the expected second-stage cost at period \(t \in [2,T]\), i.e., \(\frac{1}{N} \sum _{n \in [N]} [ Q_t({\beta },\xi ^{t}_n) ]\). Note that since all the original decision variables are defined to be nonnegative, and all the cost parameters are assumed to be nonnegative, \(Q_t({\beta },\xi ^{t}) \ge 0\) for any given \(\beta \), thus (21f) are valid. Constraints (21c) and (21d) correspond to the set of Benders optimality and feasibility cuts, respectively. As \({\beta }\) variables belong to the master problem, we add constraints (20i) and (20j) for each scenario in the sample to the master problem as (21e) which can be seen as an additional set of feasibility cuts.
The subproblem decomposes not only by scenario but also by stage. For \(t \in [2,T]\) and \(n \in [N]\), we have the corresponding subproblem (20a)–(20h), denoted by SP(t, n).
At every iteration of the Benders decomposition algorithm, we solve the master problem, get a candidate \(\beta \) solution which is fixed in the subproblems, and solve all the subproblems. For \(t \in [2,T]\), if there is at least one index \(n \in [N]\) for which SP(t, n) is infeasible, then we generate a Benders feasibility cut and add it to the master problem. Otherwise, we generate a Benders optimality cut, but add it to the master problem only if it is violated at the current master problem solution. We repeat this procedure until all the subproblems are feasible and no violated optimality cuts are found.
1.2 Dual model and level method
Let \(\lambda , \gamma , \theta ^+, \theta ^-, {\Gamma }^{u^+},{\Gamma }^{u^-}, {\Gamma }^{s}\) be the dual variables associated with the constraints (17b)–(17h) in (17), respectively. Then, the dual of (17) is:
Observing that \(\theta ^-\) variables are redundant, we remove them to simplify the model. D-LDR-2S of the capacity expansion model is obtained by substituting
in (22). Drop** \(\xi ^t\) dependences for variables to simplify the notation, we obtain
where for \(t \in [2,T]\), \(G_t({\Lambda },\xi ^{t})\) is defined as the optimal objective value of the following problem:
Let \(\xi ^{T}_n, n \in [N]\), be an independent and identically distributed (i.i.d.) random sample of the random vector \(\xi ^{T}\). We solve the obtained dual SAA problem with the bundle-level method, because the Benders decomposition method converged slowly for this problem. We use cuts aggregated over scenarios, thus introduce \(\zeta _{t}\) variable to represent the expected second-stage cost value, i.e., \(\frac{1}{N} \sum _{n \in [N]} G_t({\Lambda },\xi ^{t}_n)\), for \(t \in [2,T]\).
We observe that the subproblem (24) can be further decomposed into two: one problem including only the \(u^+\) and \(u^-\) variables, and the other problem including the remaining set of variables.
We exploit this decomposition to disaggregate the optimality cuts in the master problem. Thus, we introduce additional variables \(\zeta ^{upart}_{t}\) and \(\zeta ^{rest}_{t}\) for \(t \in [2,T]\) and obtain the following master problem:
where \(\mathcal {U}^{t}\) and \(\mathcal {R}^{t}\) represent the optimality cuts derived from problems (\(\text {DSP}^{upart}\)) and (\(\text {DSP}^{rest}\)), respectively. Moreover, we introduce the upper bounds on the new auxiliary variables, which are derived from the subproblems \((\text {DSP}^{upart})\) and \((\text {DSP}^{rest})\).
The level method also uses a quadratic program for regularization which projects the previous iterate on the level set of the current approximation of the objective function. We use the following problem for this projection:
where \(\hat{\Lambda }\) and L denote the current \({\Lambda }\) solution (i.e., the previous iterate) and the level target, respectively. The optimal solution values of \({\Lambda }\) variables determine the next iterate.
The details of the level method are provided in Algorithm 1 where LB and UB denote lower bound and upper bound, respectively.
Rights and permissions
About this article
Cite this article
Bodur, M., Luedtke, J.R. Two-stage linear decision rules for multi-stage stochastic programming. Math. Program. 191, 347–380 (2022). https://doi.org/10.1007/s10107-018-1339-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-018-1339-4