Bayesian multi-objective optimization of process design parameters in constrained settings with noise: an engineering design application

Abstract

The use of adhesive joints in various industrial applications has become increasingly popular due to their beneficial characteristics, including their high strength-to-weight ratio, design flexibility, limited stress concentrations, planar force transfer, good damage tolerance, and fatigue resistance. However, finding the best process parameters for adhesive bonding can be challenging. This optimization problem is inherently multi-objective, aiming to maximize break strength while minimizing cost and constrained to avoid any visual damage to the materials and ensure that stress tests do not result in adhesion-related failures. Additionally, testing the same process parameters several times may yield different break strengths, making the optimization process uncertain. Conducting physical experiments in a laboratory setting is costly, and traditional evolutionary approaches like genetic algorithms are not suitable due to the large number of experiments required for evaluation. Bayesian optimization is suitable in this context, but few methods simultaneously consider the optimization of multiple noisy objectives and constraints. This study successfully applies advanced learning techniques to emulate the objective and constraint functions based on limited experimental data. These are incorporated into a Bayesian optimization framework, which efficiently detects Pareto-optimal process configurations under strict budget constraints.

Appendices

Appendix A: Constrained expected improvement (CEI)

The Constrained Expected Improvement (CEI [36]) uses one OK metamodel to approximate the expensive objective and one metamodel to approximate each expensive constraint independently. Then, for a constrained optimization problem with c constraints

$$\begin{aligned} \begin{array}{ll} min &{} \ y(\textbf{x}), \textbf{x} \in {\mathbb {R}} \\ \\ \mathrm{s.t.} &{} \ g_i(\textbf{x}) \le 0, \, i=1, 2, \dots , c. \end{array} \end{aligned}$$

(A1)

the objective value of point $\textbf{x}$ can be treated as a Gaussian random variable ${\mathcal {N}}\left( \widehat{y}(\textbf{x}), \widehat{s}\mathbf (x) \right)$ and the ith constraint value of $\textbf{x}$ can be also treated as a Gaussian random variable ${\mathcal {N}}\left( \widehat{g}_i(\textbf{x}), \widehat{e}_i\mathbf (x) \right) , \; i=1, 2, \dots , c$. For this, $\widehat{y}$ and $\widehat{s}$ are the GP prediction and standard error of the objective function, respectively, and $\widehat{g}_i$ and $\widehat{e}_i$ are the GP prediction and standard error of the ith constraint function respectively.

Then, from Eq. 2 we can transform the constraint and derive the Probability of Feasibility as

$$\begin{aligned} g(\textbf{x})&=0.5-Pf(\textbf{x}) \le 0 = Pf(\textbf{x}) \ge 0.5 \nonumber \\&= \frac{Pf(\textbf{x}) - \widehat{g} (\textbf{x})}{\widehat{e}(\textbf{x})}\ge \frac{0.5 - \widehat{g} (\textbf{x})}{\widehat{e}(\textbf{x})} \end{aligned}$$

(A2)

$$\begin{aligned} PoF(\textbf{x})&= Prob\left( \frac{Pf(\textbf{x}) - \widehat{g}(\textbf{x})}{\widehat{e}(\textbf{x})} \ge \frac{0.5 - \widehat{g}(\textbf{x})}{\widehat{e} (\textbf{x})} \right) \nonumber \\&= 1-Prob\left( \frac{Pf(\textbf{x}) - \widehat{g} (\textbf{x})}{\widehat{e}(\textbf{x})} \le \frac{0.5 - \widehat{g}(\textbf{x})}{\widehat{e} (\textbf{x})} \right) \nonumber \\&= 1-\Phi \left( \frac{0.5 - \widehat{g}(\textbf{x})}{\widehat{e} (\textbf{x})} \right) . \end{aligned}$$

(A3)

In case the GP standing for the objective and constraint function are mutually independent, the CEI can be obtained by combining the EI (to be consistent with our work we used MEI instead and the predictors defined in Eq. 12 and Eq. 13) and PoF as

$$\begin{aligned} \text {CMEI-SK}(\textbf{x})&= \text {MEI}(\textbf{x}) \times \text {PoF}(\textbf{x}) \nonumber \\&=\left[ \left( \widehat{f}_{SK}(\textbf{x}_{min}) -\widehat{f}_{SK}(\textbf{x})\right) \Phi \left( {\mathcal {D}} \right) + \widehat{s}_{OK}(\textbf{x})\phi \left( {\mathcal {D}} \right) \right] \nonumber \\&\quad \times \left[ 1-\Phi \left( \frac{0.5 - \widehat{g} (\textbf{x})}{\widehat{e}(\textbf{x})}\right) \right] \end{aligned}$$

(A4)

and

$$\begin{aligned} {\mathcal {D}} = \frac{\widehat{f}_{SK}(\textbf{x}_{min}) -\widehat{f}_{SK}(\textbf{x})}{\widehat{s}_{OK}(\textbf{x})}. \end{aligned}$$

(A5)

Fig. 12

Evolution of the mean hypervolume throughout the optimization (of 50 macro-replications). The reference point [production cost = 3, break strength = 4] is used to compute this metric

Note that this equation is different from our proposed Eq. 19 in the derivation of the PoF. Figure 12 shows that our proposed cMEI-SK got on average better Pareto fronts (higher hypervolume values) than that of CEI. This suggests that considering the uncertainty predicted by the GP may lead the optimization to points that do not cause an increase in the hypervolume. This was more evident when the improvement was measured with MEI (and fitting only one GP to the scalarized objectives).

Appendix B: Wilcoxon test results

See Fig. 13

Fig. 13

Wilcoxon test results for significant differences between algorithms, using a hypervolume and b IGD+ metrics. Pink color indicate no significant differences ($p\_value \ge 5 \%$)