Correction to: Complexity of stochastic dual dynamic programming

1 Correction to: Mathematical Programming (2022) 191:717–754 https://doi.org/10.1007/s10107-020-01567-1

In this paper, we point out some corrections needed in “Complexity of Stochastic Dual Dynamic Programming”, a paper accepted to Mathematical Programming, 2020, online-first issue, DOI: https://doi.org/10.1007/s10107-020-01567-1.

The recent work in [2] studies the complexity of stochastic dual dynamic programming (SDDP) type algorithms for solving multi-stage stochastic optimization problems. It consists of a few new results on the dual dynamic programming (DDP), the original SDDP, and a new variant called explorative dual dynamic programming (EDDP). It shows that both DDP and EDDP can have linear dependence on the time horizon, while the original SDDP will have an additional dependence on the number of scenarios.

While the results on DDP and EDDP are new and solid, there exists an error in the analysis of the original version of SDDP (i.e., Lemma 9), whose correction would lead to an even worse dependence of SDDP on time horizon than reported before. More specifically, in the proof of Lemma 9 of [2], the inequality

$$\begin{aligned}&\mathop {\mathrm{Prob}}\big \{ V_{t}^k\big (x_{t-1}^k\big ) - \underline{V}_{t}\big (x_{t-1}^k\big ) \le \epsilon _{t-1}, g(x_{t-1}^k)> \delta _{t-1} \\&\quad |\tilde{g}_{t-1}^k > \delta _{t-1}; \tilde{g}_{i}^k \le \delta _{i}, i=t, \ldots , T-1\big \} \ge \tfrac{1}{N_{t-1}} \end{aligned}$$

does not necessarily hold due to the correlation between the events $\tilde{g}_{t-1}^k > \delta _{t-1}$ and $\tilde{g}_{i}^k \le \delta _{i}, i=t, \ldots , T-1$. As a result, in the statement of Lemma 9, the definition of $\bar{N}$ should be modified to

$$\begin{aligned} \bar{N} := \textstyle \prod _{i=2}^{T-1} N_i. \end{aligned}$$

Below we provide a more detailed proof for Lemma 9 with this correction.

Proof of Lemma 9

Let A denote that the event that that $\tilde{g}_{t}^k > \delta _{t}$ for some $t = 1, \ldots , T-1$. Clearly we have $\mathop {\mathrm{Prob}}\{A\} = 1- \mathop {\mathrm{Prob}}\{\tilde{g}_{i}^k \le \delta _{i}, i=1, \ldots , T-1\}$. Assume that the event A happens. Let S denote the set of sample paths, i.e., selection of T i.i.d. uniformly sample indices, where there exists at least one index with $\tilde{g}_{t}^k > \delta _{t}$. Clearly we have $|S| \le \textstyle \prod _{t=2}^{T-1} N_t$, and each sample path occurs with equal probability. We will show that there exists at least one sample path in S that generates and selects an $\epsilon _t$-saturated and $\delta _t$-distinguishable search point. Let us consider the following cases.

• (a) There exists a sample path in S such that $\tilde{g}_{T-1}^k > \delta _{T-1}$. In this case, there exists at least one search point $x_{T-1,i}^k$ such that $g_{T-1}^k(x_{T-1,i}^k) > \delta _{T-1}$, since every search point in stage $T-1$ is $\epsilon _{T-1}$-saturated, we are done.

• (b) Amongst all sample paths, no path will have $\tilde{g}_{T-1}^k > \delta _{T-1}$. Consider the set of sample paths with a stage t such that $\tilde{g}_t^k > \delta _t$. There exists at least one search point $x_{ti}^k$ such that $g_t^k(x_{ti}^k) > \delta _t$. At least $1/N_t$ fraction of these sample paths will select $x_{ti}^k$ as the search point. Now, one of the following two cases must occur upon selecting $x_t^k \leftarrow x_{ti}^k$:

  • (b1) The sample path will have $\tilde{g}_{t+1}^k \le \delta _{t+1}$. Then, by Lemma 8, $x_t^k$ will be $\epsilon _t$-saturated. Since we have already shown $x_t^k$ is also $\delta _t$-distinguishable, we are done.

  • (b2) The sample path will have $\tilde{g}_{t+1}^k > \delta _{t+1}$. Repeat the same argument with $t=t+1$. By the assumption, this incremental argument must terminate since we cannot have a sample path with $\tilde{g}_{T-1}^k > \delta _{T-1}$.

In both cases (a) and (b), we have shown the existence of a sample path that generates and selects an $\epsilon _t$-saturated and $\delta _t$-distinguishable search point. Therefore, we have

$$\begin{aligned} \mathop {\mathrm{Prob}}\{ {\varvec{q}}^k = 1| A\} \ge \tfrac{1}{|S|} \ge \textstyle \prod _{t=2}^{N-1} \tfrac{1}{N_t} = \tfrac{1}{\bar{N}}, \end{aligned}$$

from which the result immediately follows. $\square$

With this correction, we should replace the statement in Section 1 about “the iteration complexity of SDDP is worse than that of DDP and EDDP by a factor of $\bar{N} := \max \{N_2,\ldots ,N_T\}$, but still mildly increases w.r.t. T" by “the iteration complexity of SDDP is worse than that of DDP and EDDP by a factor of $\bar{N} := \textstyle \prod _{i=2}^{T-1} N_i$, which increases exponentially w.r.t. the horizon T". Also in the abstract, we should replace “the complexity of these methods mildly increases with the number of stages T" by “the complexity of some deterministic variants of SDDP mildly increases with the number of stages T".

It should be noted that although the complexity of SDDP is worse than those for DDP and EDDP, its performance in earlier phase of the algorithm should be similar to that of DDP. Intuitively, for earlier iterations, the tolerance parameter $\delta _t$ are large. As long as $\delta _t$ are large enough so that the solutions $\tilde{x}^k_{t i}$ are contained within a ball with diameter roughly in the order of $\delta _t$, one can choose any point randomly from $\tilde{x}^k_{t i}$ as $x^k_t$. In this case, SDDP will perform similarly to DDP and EDDP. This may explain why SDDP exhibits good practical performance for low accuracy region. For high accuracy region, the new EDDP algorithm seems to be a much better choice in terms of its theoretical complexity. In practice, it might make sense to run SDDP in earlier phases (due to its simplicity), and then switch to EDDP to achieve higher accuracy.

The above corrections and discussions are incorporated in the ar**v version of the paper at https://arxiv.org/abs/1912.07702.

Remark 1

Note that in a recent and related work, Zhang and Sun [3] among many developments analyzed a different deterministic variant of SDDP by Bauche et. al. [1] and obtained complexity results similar to that of EDDP in [2].¹ They also gave a similar exponential complexity result of SDDP with a slightly different approach.