Equilibrium modeling and solution approaches inspired by nonconvex bilevel programming

Abstract

Methods for finding pure Nash equilibria have been dominated by variational inequalities and complementarity problems. Since these approaches fundamentally rely on the sufficiency of first-order optimality conditions for the players’ decision problems, they only apply as heuristic methods when the players are modeled by nonconvex optimization problems. In contrast, this work approaches Nash equilibrium using theory and methods for the global optimization of nonconvex bilevel programs. Through this perspective, we draw precise connections between Nash equilibria, feasibility for bilevel programming, the Nikaido–Isoda function, and classic arguments involving Lagrangian duality and spatial price equilibrium. Significantly, this is all in a general setting without the assumption of convexity. Along the way, we introduce the idea of minimum disequilibrium as a solution concept that reduces to traditional equilibrium when an equilibrium exists. The connections with bilevel programming and related semi-infinite programming permit us to adapt global optimization methods for those classes of problems, such as constraint generation or cutting plane methods, to the problem of finding a minimum disequilibrium solution. We propose a specific algorithm and show that this method can find a pure Nash equilibrium even when the players are modeled by mixed-integer programs. Our computational examples include practical applications like unit commitment in electricity markets.

Appendices

A Alternate derivation and connections with the Nikaido–Isoda function

Optimization formulations of equilibrium problems have been presented in the literature before. Many rely on the Nikaido–Isoda (NI) function, first proposed by [41]; see also [14, 59] for recent treatments and generalizations. We give an alternate proof of Proposition 1 using the NI function approach, which some readers might find more intuitive.

Recall the game $\widehat{{\mathcal {G}}}(\left( h_i,H_i \right) _{i\in \left\{ 0,1,\dots ,m \right\} })$ from Sect. 2.2.2; player i is defined by

$$\begin{aligned} \begin{aligned} T_i(y_{-i}) = \arg \min _{y_i}\;&h_i(y_{-i}, y_i) \\ \mathrm {s.t.}\;&(y_{-i}, y_i) \in H_i, \end{aligned} \end{aligned}$$

and $(y_0^*, \dots , y_m^*)$ is a GNE of $\widehat{{\mathcal {G}}}(\left( h_i,H_i \right) _{i\in \left\{ 0,1,\dots ,m \right\} })$ if $y_i^* \in T_i(y_{-i}^*),$ for all $i \in \left\{ 0,1,\dots ,m \right\}$. For $y = (y_0, y_1, \dots , y_m)$, define

$$\begin{aligned} \begin{aligned} \phi (y) \equiv \sup _{z_0, z_1, \dots , z_m}\;&\sum _{i=0}^{m} \left( h_i(y_{-i},y_i) - h_i(y_{-i},z_i) \right) \\ \mathrm {s.t.}\;&(y_{-i},z_i) \in H_i, \forall i \in \left\{ 0,1,\dots ,m \right\} . \end{aligned} \end{aligned}$$

The objective function in this optimization problem defining $\phi$ is the NI function as defined by [14, 59]. Theorem 3.2 of [14] states that $y^* = (y_0^*, y_1^*, \dots , y_m^*)$ is a GNE of $\widehat{{\mathcal {G}}}(\left( h_i,H_i \right) _{i\in \left\{ 0,1,\dots ,m \right\} })$ if and only if $\phi (y^*) = 0$ and

$$\begin{aligned} y^* \in \arg \min _y\left\{ \phi (y): (y_{-i}, y_i) \in H_i, i \in \left\{ 0,1,\dots ,m \right\} \right\} . \end{aligned}$$

(13)

At this point, we could define minimum disequilibrium for the game $\widehat{{\mathcal {G}}}(\left( h_i,H_i \right) _{i\in \left\{ 0,1,\dots ,m \right\} })$ as a point $y^*$ satisfying (13), regardless of its objective value. We will see that this agrees with, and leads to, the concept introduced in the main text.

To use this characterization of equilibrium, consider the problem of finding a scPNE of ${\mathcal {G}}(G,\left( g_i,Y_i \right) _{i\in I})$, and following the discussion in Sect. 2.2.2 we obtain an equivalent game $\widehat{{\mathcal {G}}}(\left( h_i,H_i \right) _{i\in \left\{ 0,1,\dots ,m \right\} })$ where $H_0 \equiv \left\{ (y_{-0}, y_0): (y_0, y_1, \dots , y_m) \in G \right\}$, $H_i \equiv \left\{ (y_{-i}, y_i): y_i \in Y_i \right\}$, $h_i(y_{-i}, y_i) \equiv g_i(y_0, y_i)$, and $h_0$ is identically zero. With this, we can express the feasible set of (13) in terms of the set G and data of the (${\mathcal {A}}_i$) problems as

$$\begin{aligned} \begin{aligned} \Phi \equiv \big \{&(y_0,y_1, \dots , y_m): \\&(y_0, y_1, \dots , y_m) \in G, \\&y_i \in Y_i, \forall i \in \left\{ 1,\dots ,m \right\} \big \}, \end{aligned} \end{aligned}$$

which coincides with the feasible set of Problem (${{\mathcal {M}}}{{\mathcal {D}}}$) (defining $x \equiv y_0$). Using the definition of the $h_i$ (and the fact that $h_0$ is identically zero), we transform $\phi$ into

$$\begin{aligned} \begin{aligned} \phi (y) = {{\,\mathrm{\textstyle {\sum }}\,}}_{i=1}^{m} g_i(y_0,y_i) - \inf _z \big \{&{{\,\mathrm{\textstyle {\sum }}\,}}_{i=1}^{m} g_i(y_0,z_i): \\&(z_0, y_1, \dots , y_m) \in G,\\&z_i \in Y_i, \forall i \in \left\{ 1,\dots ,m \right\} \big \}. \end{aligned} \end{aligned}$$

For $y \in \Phi$, the infimum above is over a nonempty set, and furthermore, does not depend on the $z_0$ variable. Consequently, we can ignore the side constraints encoded in G and decompose the minimization. Thus, for $y \in \Phi$, the expression for $\phi (y)$ simplifies to

$$\begin{aligned} \phi (y) = {{\,\mathrm{\textstyle {\sum }}\,}}_{i=1}^{m} g_i(y_0,y_i) - {{\,\mathrm{\textstyle {\sum }}\,}}_{i=1}^{m} g_i^*(y_0) \end{aligned}$$

where we recall the optimal player value function $g_i^*$ defined in Equation (1). Finally, note that this expression equals the objective function of Problem (${{\mathcal {M}}}{{\mathcal {D}}}$) when $\mu : w \mapsto {{\,\mathrm{\textstyle {\sum }}\,}}_i w_i$. Thus, Problems (${{\mathcal {M}}}{{\mathcal {D}}}$) and (13) coincide, and the statement that $y^*$ is an equilibrium iff $\phi (y^*) = 0$ and $y^* \in \arg \min \left\{ \phi (y): y \in \Phi \right\}$, essentially provides an alternate proof of Proposition 1.

B Proof of Theorem 1

One effect of solving the player problems inexactly is that the lower bounds (at worst) converge to the optimal value of the relaxation

$$\begin{aligned} {\widetilde{\delta }} = \inf _{x,y,w}&{{\,\mathrm{\textstyle {\sum }}\,}}_{i\in I} (g_i(x,y_i) - w_i) \nonumber \\ \mathrm {s.t.}\;&(x,y_1,\dots ,y_m) \in G, \nonumber \\&y_i \in Y_i, \quad \forall i \in I, \nonumber \\&w_i \le g_i^*(x) + \eta _i^*, \quad \forall i \in I. \end{aligned}$$

(14)

By defining $w_i' = w_i - \eta _i^*$ and re-writing the objective as ${{\,\mathrm{\textstyle {\sum }}\,}}_{i\in I} (g_i(x,y_i) - w_i' - \eta _i^*)$ we see that

$$\begin{aligned} {\widetilde{\delta }} = \delta - {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*. \end{aligned}$$

Similarly, the upper bounds will only reach within $\sum _i \eta _i^*$ of $\delta$. The following proof makes this precise. We repeat the statement of Theorem 1 here for convenience.

Theorem

Assume that the set

$$\begin{aligned} F\equiv \left\{ (x,y): (x,y_1,\dots ,y_m) \in G, y_i \in Y_i,\forall i \in I \right\} \end{aligned}$$

is compact and nonempty. Assume that for each i, $g_i$ is continuous and $Y_i$ is compact. Let $\epsilon ^* = \limsup _{k \rightarrow \infty } \epsilon ^k$ and $\eta _i^* = \limsup _{k\rightarrow \infty } \eta _i^k$ for each i. Then for any $\varepsilon > \epsilon ^* + 2 {{\,\mathrm{\textstyle {\sum }}\,}}_{i \in I} \eta _i^*$, Algorithm 1 produces an $\varepsilon$-optimal solution $(x^*,y^*)$ of Problem (${{\mathcal {M}}}{{\mathcal {D}}}'$) in finite iterations.

Proof

We establish that the upper and lower bounds converge to some values $\delta ^{U,*}$ and $\delta ^{L,*}$, respectively, such that $\delta ^{L,*} \le \delta \le \delta ^{U,*}$ and $\delta ^{U,*} - \delta ^{L,*} \le \epsilon ^* + 2{{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*$.

To begin, we show the the approximate solutions of the lower bounding problem (4) have a subsequence that converge to a feasible point of the relaxation (14). Let $\left( (x^k,y^k,w^k) \right) _{k \in {\mathbb {N}}}$ be the sequence of feasible solutions of Problem (4) produced by Algorithm 1. Since more elements are added to $Y_i^{L,k}$ at each iteration, the part of the solution sequence $\left( w_i^k \right) _{k}$ is non-increasing, but bounded below by the minimum of $g_i$ on $F$; continuity and compactness ensure that this is finite (specifically, $\inf _{x,y,z_i} \left\{ g_i(x,z_i): z_i \in Y_i, (x,y) \in F \right\} > -\infty$). Thus, for each i, $\left\{ w_i^k: k \in {\mathbb {N}} \right\}$ is contained in a compact set, and so the entire solution sequence is in a compact set. For each i, let $\left( z_i^k \right) _{k \in {\mathbb {N}}}$ be the corresponding sequence of approximate solutions to the player problem (${\mathcal {A}}_i$); these must exist at each iteration k by continuity of $g_i$ and compactness of $Y_i$. Again, the image of this sequence $\left\{ z_i^k: k \in {\mathbb {N}} \right\}$ is in a compact set ($Y_i$) for each i. Consequently, we have that a subsequence of solutions converges to some point. Abusing notation, we have that $(x^k,y^k,w^k) \rightarrow (x^*,y^*,w^*)$ and $z^k \rightarrow z^*$. Note that we have $(x^*,y^*) \in F$.

Now, we establish that $(x^*,y^*,w^*)$ is feasible in the relaxation (14). Since $z_i^k$ is added to $Y_i^{L,k}$ at the end of each iteration, we have for each i

$$\begin{aligned} w_i^{\ell } \le g_i(x^{\ell }, z_i^k), \quad \forall \ell ,k: \ell > k. \end{aligned}$$

By taking the limit over $\ell$, and then the limit over k, we get for each i

$$\begin{aligned} w_i^* \le g_i\big (x^*, z_i^*\big ). \end{aligned}$$

(15)

Now, for a contradiction, assume that for some i, $w_i^* > g_i^*(x^*) + \eta _i^*$, indicating that $(x^*,y^*,w^*)$ is not feasible in Problem (14). This means that there exists $z_i^{\dagger } \in Y_i$ (feasible in the player problem) with

$$\begin{aligned} w_i^* > g_i\big (x^*, z_i^{\dagger }\big ) + \eta _i^*. \end{aligned}$$

(16)

By definition of $z_i^k$ as an approximate minimizer of (${\mathcal {A}}_i$) for $x = x^k$, we have $g_i(x^k, z_i^{\dagger }) + \eta _i^k \ge g_i(x^k, z_i^{k})$ for all k, and taking the limit superior over k we get

$$\begin{aligned} g_i\big (x^*, z_i^{\dagger }\big ) + \eta _i^* = \limsup _{k\rightarrow \infty }\big (g_i\big (x^k, z_i^{\dagger }\big ) + \eta _i^k\big ) \ge \limsup _{k\rightarrow \infty } g_i\big (x^k, z_i^{k}\big ) = g_i\big (x^*,z_i^*\big ). \end{aligned}$$

Combined with Inequality (16) this gives

$$\begin{aligned} w_i^* > g_i\big (x^*,z_i^*\big ), \end{aligned}$$

which contradicts (15). Thus, $(x^*,y^*,w^*)$ is feasible in Problem (14), and in particular, satisfies for each $i \in I$

$$\begin{aligned} w_i^* \le g_i^*(x^*) + \eta _i^*. \end{aligned}$$

(17)

Next, we focus on the lower bounds. The algorithm’s lower bound is constructed as

and so forms a non-decreasing sequence. From the approximate solution of the lower bounding problem (4), we have . A simple induction argument establishes that $\delta ^{L,k} \le \delta$ for all k, and so

$$\begin{aligned}\begin{aligned} \delta ^{L,*}&\equiv \lim _{k \rightarrow \infty } \delta ^{L,k} \\&\le \delta . \end{aligned}\end{aligned}$$

Since we have for all k, it follows for all k. By construction, for all k, and so we have $\limsup _{k\rightarrow \infty } ({\bar{\delta }}^{L,k} - \delta ^{L,k}) \le \limsup _{k\rightarrow \infty } \epsilon ^k \equiv \epsilon ^*$. Note that ${\bar{\delta }}^{L,k} = {{\,\mathrm{\textstyle {\sum }}\,}}_{i} (g_i(x^k,y_i^k) - w_i^k)$ has a subsequential limit

$$\begin{aligned} {\bar{\delta }}^{L,*} \equiv {{\,\mathrm{\textstyle {\sum }}\,}}_{i} \big (g_i\big (x^*,y_i^*\big ) - w_i^*\big ). \end{aligned}$$

Combining this,

$$\begin{aligned} {\bar{\delta }}^{L,*} - \delta ^{L,*}&\le \limsup _{k\rightarrow \infty } \big ({\bar{\delta }}^{L,k} - \delta ^{L,k}\big ) \nonumber \\&\le \epsilon ^*. \end{aligned}$$

(18)

Next, we focus on the upper bounds. Note that for each i, $g_i^*: {\mathbb {R}}^{n_0} \rightarrow {\mathbb {R}}$ is a continuous function by [1, Theorem 1.4.16]. Thus,

$$\begin{aligned} {\bar{\delta }}^k \equiv {{\,\mathrm{\textstyle {\sum }}\,}}_i \big (g_i\big (x^k,y_i^k\big ) - g_i^*\big (x^k\big )\big ) \end{aligned}$$

(19)

has a subsequential limit ${{\,\mathrm{\textstyle {\sum }}\,}}_i (g_i(x^*,y_i^*) - g_i^*(x^*))$ and by Inequality (17)

$$\begin{aligned}\begin{aligned} {\bar{\delta }}^*&\equiv {{\,\mathrm{\textstyle {\sum }}\,}}_i \big (g_i\big (x^*,y_i^*\big ) - g_i^*\big (x^*\big )\big ) \\&\le {{\,\mathrm{\textstyle {\sum }}\,}}_i \big (g_i\big (x^*,y_i^*\big ) - w_i^* + \eta _i^*\big ) \\&= {\bar{\delta }}^{L,*} + {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*. \end{aligned}\end{aligned}$$

For each k, ${\bar{\delta }}^k$ is an upper bound: ${\bar{\delta }}^k \ge \delta$. Consequently, so is ${\bar{\delta }}^*$: ${\bar{\delta }}^* \ge \delta$. Rearranging Inequality (18), we have ${\bar{\delta }}^{L,*} - \epsilon ^* \le \delta ^{L,*}$. Combining these relations, we get:

$$\begin{aligned} {\bar{\delta }}^{L,*} - \epsilon ^* \le \delta ^{L,*} \le \delta \le {\bar{\delta }}^* \le {\bar{\delta }}^{L,*} + {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*. \end{aligned}$$

(20)

Thus, $\left( {\bar{\delta }}^k \right) _{k}$ converges to within $\epsilon ^* + {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*$ of both $\delta$ and the lower bound limit $\delta ^{L,*}$. It remains to show that the upper bounds that are actually calculated, $\delta ^{U,k}$, also converge within a reasonable value.

To this end, note that

$$\begin{aligned} \delta ^{U,k} = \min \left\{ {\bar{\delta }}^{U,k}, \delta ^{U,k-1} \right\} \end{aligned}$$

where ${\bar{\delta }}^{U,k} = {{\,\mathrm{\textstyle {\sum }}\,}}_{i\in I} (g_i(x^k,y_i^k) - g_i^{L,k})$. Combining this with Equation (19), we have ${\bar{\delta }}^{U,k} - {\bar{\delta }}^k = {{\,\mathrm{\textstyle {\sum }}\,}}_i g_i^*(x^k) - g_i^{L,k}$. By construction of $g_i^{L,k}$, $0 \le g_i^*(x^k) - g_i^{L,k} \le \eta _i^k$ and so $0 \le {\bar{\delta }}^{U,k} - {\bar{\delta }}^k \le {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^k$. It is simple to see that $\delta ^{U,k} \ge \delta$ for all k, and that it is a non-increasing sequence, and so it must converge to some value greater than $\delta$:

$$\begin{aligned}\begin{aligned} \delta ^{U,*}&\equiv \lim _{k \rightarrow \infty } \delta ^{U,k}\\&\ge \delta . \end{aligned}\end{aligned}$$

Further, $\delta ^{U,k} \le {\bar{\delta }}^{U,k}$ for all k and so $\delta ^{U,k} - {\bar{\delta }}^k \le {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^k$. Consequently,

$$\begin{aligned}\begin{aligned} \delta ^{U,*} - {\bar{\delta }}^*&\le \limsup _{k\rightarrow \infty } \big (\delta ^{U,k} - {\bar{\delta }}^k\big )\\&\le \limsup _k {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^k\\&\le {{\,\mathrm{\textstyle {\sum }}\,}}_i \limsup _k \eta _i^k\\&= {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^* \end{aligned}\end{aligned}$$

Finally, using this and Inequality (20), if $\delta ^{U,*}$ is greater than ${\bar{\delta }}^*$, then $\delta ^{U,*} - \delta ^{L,*} \le \epsilon ^* + 2{{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*$. Otherwise, we have $\delta ^{U,*} - \delta ^{L,*} \le \epsilon ^* + {{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*$ (since $\delta ^{U,*} \ge \delta$). In either case, we have the conclusion

$$\begin{aligned} \delta ^{U,*} - \delta ^{L,*} \le \epsilon ^* + 2{{\,\mathrm{\textstyle {\sum }}\,}}_i \eta _i^*. \end{aligned}$$

$\square$

C Data for and solution of gas network price equilibrium example

In this section we specify the data used for the example in Sect. 6.3, as well as the equilibrium solution found by the primal-dual approach. The topology of the network is in Fig. 1. See Tables 4, 5, 6 and 7 for the data and solution of the overall nodes, pipelines/transmission, supplies, and demands, respectively. In addition, we have lower and upper bounds on the squared pressure variable $b_k^p = 900$ (bar$^2$) and $B_k^p = 4900$ (bar$^2$), for each node $k \in K$. Finally, based on the costs of supply and marginal utilities, we enforce the bounds [0, 12.1] for each price $x_k$ when solving the dual problem. From Table 4, we note that these bounds are not binding at the equilibrium solution.

Table 4 Price and squared pressure solution

Table 5 Pipeline data and solution

Table 6 Supply bounds, costs data, and solution

Table 7 Demand bounds, objective function data, and solution