Construction of optimal designs for quantile regression model via particle swarm optimization

Abstract

As an extension of mean regression and being robust against outliers, quantile regression has been used in many fields such as biomedicine, ecology, economics. However, it is theoretically and computationally challenging to find the optimal experimental design for quantile regression due to the complexity of the optimization problem. The purpose of this paper is to provide theoretical necessary conditions for A- and c-optimality of a design separately, and a numerical algorithm to find optimal designs for quantile regression models. The algorithm is constructed through particle swarm optimization so as to solve the problem of non-convexity of optimality criteria. In this paper, the algorithm is applied to obtain locally as well as Bayesian optimal designs for Michaelis–Menten, Emax and Exponential quantile regression models. We demonstrate that this technique can be applied to a variety of optimality criteria and scale functions without making any further assumption.

Appendix

Proof of Theorem 1

Because locally optimality is considered here, for fixed initial guess for $\theta$, $\theta$ is omitted in $D_i(\xi ,\theta )$ and $H(\xi ,\theta )$ next. Assume $\xi ^*$ is locally A-optimal for quantile regression model (1), i.e. $\xi ^*$ minimizes the trace $\text {tr}(H(\xi ))$. For $\alpha \in [0,1]$ and a further design $\xi _\alpha =(1-\alpha )\xi ^*+\alpha \xi$. The directional derivative of the function,

$$\begin{aligned} m(\xi _\alpha )=\text {tr}(H(\xi _\alpha ))=\text {tr}(D_1^{-1}(\xi _\alpha )D_0(\xi _\alpha )D_1^{-1}(\xi _\alpha )) =\text {tr}(D_1^{-2}(\xi _\alpha )D_0(\xi _\alpha )), \end{aligned}$$

at $\xi ^*$ in the direction of $\xi -\xi ^*$ is

$$\begin{aligned}{} & {} \frac{d}{d\alpha }m(\xi _\alpha ) \bigg |_{\alpha =0}=\frac{d}{d\alpha }\text {tr}(D_1^{-2}(\xi _\alpha )D_0(\xi _\alpha ))\bigg |_{\alpha =0}\\{} & {} \quad =\frac{d}{d\alpha }\text {tr}\left( \left( (1-\alpha )D_1(\xi ^*)+\alpha D_1(\xi )\right) ^{-2}\left( (1-\alpha )D_0(\xi ^*)+\alpha D_0(\xi )\right) \right) \bigg |_{\alpha =0}\\{} & {} \quad =\text {tr}\left( \frac{d}{d\alpha }\left( (1-\alpha )D_1(\xi ^*)+\alpha D_1(\xi )\right) ^{-2}\right. \\{} & {} \quad \left. \bigg |_{\alpha =0}D_0(\xi ^*)+D_1^{-2}(\xi ^*)\left( D_0(\xi )-D_0(\xi ^*)\right) \right) \\{} & {} \quad =-\text {tr}\left( D_1^{-2}(\xi ^*)D_1(\xi )D_1^{-1}(\xi ^*)D_0(\xi ^*)\right) -\text {tr} \left( D_1^{-1}(\xi ^*)D_1(\xi )D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) \\{} & {} \qquad +2\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) +\text {tr}\left( D_1^{-2}(\xi ^*)D_0 (\xi )\right) -\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) \\{} & {} \quad =-\text {tr}\left( D_1(\xi )D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-2}(\xi ^*)\right) -\text {tr}\left( D_1(\xi )D_1^{-2}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\right) \\{} & {} \qquad +\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi )\right) +\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) \ge 0, \end{aligned}$$

for all designs $\xi$. So the inequality holds for $\xi =\delta _x$ where $\delta _x$ denotes Dirac measure with mass 1 at points $x\in {\mathcal {X}}$. That is,

$$\begin{aligned}{} & {} -\text {tr}\left( D_1(\delta _x)D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-2}(\xi ^*)\right) -\text {tr}\left( D_1(\delta _x)D_1^{-2}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\right) \\{} & {} \qquad +\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\delta _x)\right) +\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) \\{} & {} \quad =-\frac{1}{\sigma (x,\theta )}\dot{g}^T(x,\theta )D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-2}(\xi ^*)\dot{g}(x,\theta )\\{} & {} \qquad -\frac{1}{\sigma (x,\theta )}\dot{g}^T(x,\theta )D_1^{-2}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\dot{g}(x,\theta )\\{} & {} \qquad +\dot{g}^T(x,\theta )D_1^{-2}(\xi ^*)\dot{g}(x,\theta )+\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) \ge 0, \end{aligned}$$

for all $x\in {\mathcal {X}}$. From (2) and (3) it follows that

$$\begin{aligned}{} & {} \int \frac{1}{\sigma (x,\theta )}\dot{g}^T(x,\theta )D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-2}(\xi ^*)\dot{g}(x,\theta )d\xi ^*(x)\\{} & {} \quad =\int \frac{1}{\sigma (x,\theta )}\dot{g}^T(x,\theta )D_1^{-2}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\dot{g}(x,\theta )d\xi ^*(x)\\{} & {} \quad =\int \dot{g}^T(x,\theta )D_1^{-2}(\xi ^*)\dot{g}(x,\theta )d\xi ^*(x)=\text {tr}\left( D_1^{-2}(\xi ^*)D_0(\xi ^*)\right) . \end{aligned}$$

It then follows that the equality is attained at each support point of $\xi ^*$, which concludes the proof. $\square$

Proof of Theorem 2

Similarly, $\theta$ is omitted in g $D_i(\xi ,\theta )$ and $H(\xi ,\theta )$ next. Assume $\xi ^*$ is locally c-optimal for quantile regression model (1), i.e. $\xi ^*$ minimizes $c^TH(\xi )c$. For $\alpha \in [0,1]$ and a further design $\xi _\alpha =(1-\alpha )\xi ^*+\alpha \xi$, the directional derivative of $H(\xi _\alpha )$ at $\xi ^*$ in the direction of $\xi -\xi ^*$ is

$$\begin{aligned}{} & {} \frac{d}{d\alpha }H(\xi _\alpha )\bigg |_{\alpha =0}= \frac{d}{d\alpha }D_1^{-1}(\xi _\alpha )D_0(\xi _\alpha )D_1^{-1}(\xi _\alpha )\bigg |_{\alpha =0}\\{} & {} \quad =\frac{d}{d\alpha }D_1^{-1}(\xi _\alpha )\bigg |_{\alpha =0}D_0(\xi ^*) D_1^{-1}(\xi ^*)+ D_1^{-1}(\xi ^*)\left( D_0(\xi )-D_0(\xi ^*)\right) D_1^{-1}(\xi ^*)\\{} & {} \qquad +D_1^{-1}(\xi ^*)D_0(\xi ^*)\frac{d}{d\alpha }D_1^{-1}(\xi _\alpha )\bigg |_{\alpha =0}\\{} & {} \quad =-D_1^{-1}(\xi ^*)\left( D_1(\xi )-D_1(\xi ^*)\right) D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\\{} & {} \qquad +D_1^{-1}(\xi ^*)\left( D_0(\xi )-D_0(\xi ^*)\right) D_1^{-1}(\xi ^*)\\{} & {} \qquad -D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\left( D_1(\xi )-D_1(\xi ^*)\right) D_1^{-1}(\xi ^*)\\{} & {} \quad =-D_1^{-1}(\xi ^*)\left( D_1(\xi )D_1^{-1}(\xi ^*)D_0(\xi ^*)+D_0(\xi ^*)\right. \\{} & {} \qquad \left. D_1^{-1}(\xi ^*)D_1(\xi )-D_0(\xi )-D_0(\xi ^*)\right) D_1^{-1}(\xi ^*). \end{aligned}$$

Consequently, the directional derivative of $c^TH(\xi _\alpha )c$ at $\xi ^*$ in the direction of $\xi -\xi ^*$ is

$$\begin{aligned}{} & {} \frac{d}{d\alpha }c^TH(\xi _\alpha )c \bigg | _{\alpha =0}\\{} & {} \quad =-c^TD_1^{-1}(\xi ^*)\left( D_1(\xi )D_1^{-1}(\xi ^*)D_0(\xi ^*)\right. \\{} & {} \qquad \left. + D_0(\xi ^*)D_1^{-1}(\xi ^*)D_1(\xi )-D_0(\xi )-D_0(\xi ^*)\right) D_1^{-1}(\xi ^*)c\\{} & {} \quad =-c^TD_1^{-1}(\xi ^*)\frac{1}{\sigma (x,\theta )}\dot{g}(x,\theta ) \dot{g}^T(x,\theta )D_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)c\\{} & {} \qquad -c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\frac{1}{\sigma (x,\theta )}\dot{g}(x,\theta )\dot{g}^T(x,\theta )D_1^{-1}(\xi ^*)c\\{} & {} \qquad +c^TD_1^{-1}(\xi ^*)\dot{g}(x,\theta )\dot{g}^T(x,\theta )D_1^{-1}(\xi ^*)c+c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)c\ge 0 \end{aligned}$$

for all designs $\xi$. So the inequality holds for $\xi =\delta _x$ where $\delta _x$ denotes Dirac measure with mass 1 at points $x\in {\mathcal {X}}$. That is,

$$\begin{aligned}{} & {} -\frac{2}{\sigma (x,\theta )}\left( c^TD_1^{-1}(\xi ^*)\dot{g}(x,\theta )\right) \left( c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\dot{g}(x,\theta )\right) \\{} & {} \quad +\left( c^TD_1^{-1}(\xi ^*)\dot{g}(x,\theta )\right) ^2+c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)c\ge 0 \end{aligned}$$

for all $x\in {\mathcal {X}}$. From (2) and (3), it follows that

$$\begin{aligned}{} & {} \int \frac{1}{\sigma (x,\theta )}\left( c^TD_1^{-1}(\xi ^*)\dot{g}(x,\theta ) \right) \left( c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)\dot{g}(x,\theta )\right) d\xi ^*(x)\\{} & {} \quad =\int \left( c^TD_1^{-1}(\xi ^*)\dot{g}(x,\theta )\right) ^2d\xi ^*(x) \\{} & {} \quad =c^TD_1^{-1}(\xi ^*)D_0(\xi ^*)D_1^{-1}(\xi ^*)c. \end{aligned}$$

It then follows that the equality is attained at each support point of $\xi ^*$, which concludes the proof. $\square$