Bandgap optimization in combinatorial graphs with tailored ground states: application in quantum annealing

Abstract

A mixed-integer linear programming (MILP) formulation is presented for parameter estimation of the Potts model. Two algorithms are developed; the first method estimates the parameters such that the set of ground states replicate the user-prescribed data set; the second method allows the user to prescribe the ground states multiplicity. In both instances, the optimization process ensures that the bandgap is maximized. Consequently, the model parameter efficiently describes the user data for a broad range of temperatures. This is useful in the development of energy-based graph models to be simulated on Quantum annealing hardware where the exact simulation temperature is unknown. Computationally, the memory requirement in this method grows exponentially with the graph size. Therefore, this method can only be practically applied to small graphs. Such applications include learning of small generative classifiers and spin-lattice model with energy described by Ising hamiltonian. Learning large data sets poses no extra cost to this method; however, applications involving the learning of high dimensional data are out of scope.

Appendices

Appendices

The codes are available at https://github.com/sidsriva/PEP.

Appendix 1: Proof of theorem

(a) Since $\mathcal {S}_{G}(\varvec{\theta },\beta ) = \mathcal {S}_D$, the Negative Log Likelhood, $\eta (\varvec{\theta }_D,\beta )$, is estimated as:

$$\begin{aligned} \eta (\varvec{\theta }_D,\beta ) = N_{GS} \beta E_0 + N_{GS} \log Z \end{aligned}$$

The derivative is estimated as::

$$\begin{aligned} \frac{d\eta }{d\beta }&= N_{GS} \left( (E_{0} - \mathbb {E}(E) \right) \end{aligned}$$

(10)

where

$$\begin{aligned} \mathbb {E}(E) = \sum _{\varvec{S}\in \mathcal {S}} E(\varvec{S}) p(\varvec{S}|\varvec{\theta }_D,\beta ) \end{aligned}$$

Since $\Delta E > 0$, the expected energy is strictly bounded below as $\mathbb {E}(E) > E_0$. Consequently:

$$\begin{aligned} \frac{d\eta }{d\beta } < 0 \end{aligned}$$

In the low temperature limit, Eq. (2) estimates that the probability of all excited states approaches 0 while all ground states are equally likely with probability $(N_{GS})^{-1}$. Therefore, the value of $\eta$ in this limit is estimated as Eq. (4).

(b) Let $\varvec{S}_G \in \mathcal {S}_G$ and $P = p(\varvec{S}_G|\varvec{\theta }_D,\beta )$ so that $\eta (\varvec{\theta }_D,\beta ) = - N_{GS} \log {P}$. The probability of occurrence of a ground state is given by $N_{GS}P$ and occurrence of a excited state is given as $\left( 1-N_{GS}P\right)$. Moreover, for any finite value of $\beta$ both of these probabilities are finite. Therefore, the expectation of energy, $\mathbb {E}$, can be bounded as

$$\begin{aligned} \mathbb {E} = N_{GS} P E_0 + \sum _{\varvec{S}\in \mathcal {S}_E} E(\varvec{S}) p(\varvec{S}|\varvec{\theta }_D,\beta ) \le N_{GS} P E_0 + (1- N_{GS} P) E_1 \end{aligned}$$

Substituting in Eq. (10),

$$\begin{aligned} \frac{d \eta }{d\beta } = E_{0} - \mathbb {E}(E)&\le \left( N_{GS}P - 1\right) N_{GS} \Delta E \end{aligned}$$

Substituting $P = e^{-\eta /N_{GS}}$ gives the following differential inequality

$$\begin{aligned} \frac{d \eta }{d\beta } \le \left( N_{GS} e^{-\eta /N_{GS}} - 1\right) N_{GS} \Delta E \end{aligned}$$

(11)

Consider the differential equation for $\beta \in [0,\infty )$,

$$\begin{aligned} \frac{d \xi }{d\beta } = \left( N_{GS} e^{-\xi /N_{GS}} - 1\right) N_{GS} \Delta E \end{aligned}$$

(12)

with initial condition $\xi (\varvec{\theta }_D,0) = \eta (\varvec{\theta }_D,0) = N_{GS}\log N_{TS}$. Noting that $N_{GS} e^{-\xi /N_{GS}} - 1 = N_{GS}P-1 > 0$, this ODE is integrated to give the following solution:

$$\begin{aligned} \xi (\varvec{\theta }_D,\beta ) = N_{GS} \log { \left( N_{GS} + N_{ES} e^{-\beta \Delta E} \right) } \end{aligned}$$

(13)

Using Comparison Lemma (Khalil 2002), for all $0<\beta <\infty$,

$$\begin{aligned} \eta (\varvec{\theta }_D,\beta ) \le \xi (\varvec{\theta }_D,\beta ) \end{aligned}$$

(14)

This proves the upper bound. The lower bound is a direct consequence from monotonicity proved in part 1.

(c) For any $\beta < \infty$

$$\begin{aligned} \eta (\varvec{\theta }_D,\beta ) - N_{GS} \log {N_{GS}} \le N_{GS} \log \left( 1+ \frac{N_{ES}}{N_{GS}} e^{-\beta \Delta E} \right) \end{aligned}$$

For any $\epsilon >0$, choose a $\beta > \beta ^*(\epsilon )$ using Eq. (6) and observe that,

$$\begin{aligned} N_{GS}\log \left( 1+ \frac{N_{ES}}{N_{GS}} e^{-\beta \Delta E} \right) < \epsilon \end{aligned}$$

This proves the third statement.

Appendix 2: Optimized graphs

1.1 Appendix 2.1: K-3 graph

A fully connected 3-noded graph is optimized for 4 data states. The energy of the graph is modeled using Ising model Eq. (9) with $|H|\le 1$ and $|J|\le 1$. The optimized parameters using the (1) Minimization of Negative Log-likelihood, and (2) PEPDAS method are presented in Fig. 4

Fig. 4

a Training data set of states with green representing a ‘+1’ state and red representing a ‘-1’ state. b Optimized graph using minimization of Negative Log-likelhood at $\beta =1$ c Optimized graph using PEPDAS method. The field terms are mentioned in blue color and interaction terms are mentioned in red color

1.2 Appendix 2.2: Peterson graph

A Peterson graph is first optimized for upto 3 user prescribed data states using PEPDAS method. Then it is optimized for 3 ground states using PEPGSM method. The energy of the graph is modeled using Ising model Eq. (9) with $|H|<1$ and $|J|<1$. The optimized graphs are presented in Fig. 5 and their respective Negative log likelhood is presented in Fig. 6.

Fig. 5

Optimal Ising parameters of a Peterson graph found using PEPDAS method (left) and PEPGSM method (right). The ground states are presented as the colored graph in the top right corner of each image. A green label denotes the ‘$+1$’ state and the red label denotes the ‘$-1$’ state

Fig. 6

Normalized Negative log likelihood and their respective bounds for Peterson graphs trained using PEPDAS method