Single-index composite quantile regression for ultra-high-dimensional data

Abstract

Composite quantile regression (CQR) is a robust and efficient estimation method. This paper studies CQR method for single-index models with ultra-high-dimensional data. We propose a penalized CQR estimator for single-index models and combine the debiasing technique with the CQR method to construct an estimator that is asymptotically normal, which enables the construction of valid confidence intervals and hypothesis testing. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

Appendices

A Conditions

To establish the asymptotic properties of the proposed estimators, the following technical conditions are imposed.

C1. The kernel $K(\cdot )$ is a symmetric density function with bounded support and Lipschitz continuous second-order derivative.

C2. The density function of $\mathbf{U}=\mathbf{X}^{\top }{\gamma }$ is positive and uniformly continuous for ${\gamma }$ in a neighborhood of ${\gamma }_{0}$. Further the density of $\mathbf{X}^{\top }{\gamma }_{0}$ is continuous and bounded away from 0 and $\infty$ on its support.

C3. The function $g_{0}(\cdot )$ has a continuous and bounded second derivative.

C4. Assume that the model error $\varepsilon$ has a positive density $f(\cdot )$, that is $\sup _{t\in R}f(t)\le C_{f}$ and $\min _{k\in \{1,\ldots ,K\}}f(c_k)\ge C_{f_k}$, for some constants $C_{f}$ and $C_{f_k}$. Moreover, $f(\cdot )$ is differentiable with the first order derivative uniformly bounded by a positive constant $C_{f'}$.

C5. $\max _{1\le j\le p}{} \mathbf{S}_{jj}\le C_x$, for a constant $C_x$.

C6. For a constant $C_{\lambda }$, the matrix $\mathbf{S}$ satisfies $C_{\lambda }=\inf _{\theta \in A,\theta \ne \mathbf{0}}\frac{\theta ^{\top }{} \mathbf{S}\theta }{\theta ^{\top }\theta },$ where $A=\{\theta \in R^p: \Vert \theta _{T^c}\Vert _1 \le 3\Vert \theta _T\Vert _1\}$, and $T=\{j\in (1,\ldots ,p):\gamma _{0,j}\ne 0 \}$ to be the true support of $\gamma _0$.

C7. There exists a constant $C_{\psi }$ such that $\psi (n/\log (p\vee n))\le C_{\psi }$ with probability intend to 1, where

$$\begin{aligned} \psi (q)=\sup _{\Vert \theta \Vert _0\le q,\theta \ne \mathbf{0}} \frac{ \theta ^{\top }{} \mathbf{S}\theta }{\Vert \theta \Vert _2^2}. \end{aligned}$$

Remark 1

Conditions C1–C4 are standard conditions, which are commonly used in single-index regression model, see Wu et al. (2010) and Jiang et al. (2016). Conditions C5–C7 are used for high-dimensional data. Condition C5 is used in van de Geer et al. (2014). Conditions C6–C7 are the restricted eigenvalue, see condition D4 in Belloni et al. (2011).

B Proof of main results

Lemma 1

Suppose C1–C5 hold. Then for the choice $\lambda \ge 2CK\sqrt{C_x}\sqrt{(\log p)/n}$, we have with probability at least $1-2Kp^{1-C^2}-\delta _n$ that

$$\begin{aligned} \Vert {\hat{\gamma }}_{T^c}\Vert _1\le 3\Vert ({\hat{\gamma }}-\gamma _0)_{T}\Vert _1. \end{aligned}$$

Lemma 2

Suppose C1–C6 hold and $p>3$. Then, we have with probability at least $1-6p^{1-C^2}-3\delta _n$ that

$$\begin{aligned}&\sup _{\theta \in A,\Vert \theta \Vert _\mathbf{S}\le \xi }\left| \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K\left\{ {\tilde{L}}_{i,k}(\gamma _0+\theta ) -{\tilde{L}}_{i,k}(\gamma _0)\right\} -\frac{1}{n}\sum _{i=1}^n\sum _{k=1}^KE\left\{ {\tilde{L}}_{i,k}(\gamma _0+\theta ) -{\tilde{L}}_{i,k}(\gamma _0)\right\} \right| \\&\qquad \quad \le C_BK\xi \sqrt{\frac{s\log p}{n}}, \end{aligned}$$

where $A= \{\gamma \in R^p: \Vert {\hat{\gamma }}_{T^c}\Vert _1 \le 3\Vert ({\hat{\gamma }}-\gamma _0)_T\Vert _1\}$ and $C_B=\max \{\sqrt{12},64\sqrt{2}CC_{\lambda }^{-1/2}\sqrt{C_x} \}$.

Lemma 3

Suppose that condition C4 holds and that $\max _{1\le k\le K}\sup _{|\tau -\tau _k|\le {\tilde{h}}}|Q'''(\tau )|=O(1),$ where $Q'''(\tau )$ is the cubic derivative of the $\tau$-quantile of the error. Then, with the choice of the bandwidth ${\tilde{h}}=\left( s\log \left( p\vee n\right) /{n} \right) ^{1/6}$, we have

$$\begin{aligned} \left| {\hat{\theta }}_K-{\theta }_K\right| =O_p\left( \left( \frac{s\log \left( p\vee n\right) }{n} \right) ^{1/3} \right) . \end{aligned}$$

Proof of Theorem 2.1

Define $\pi _2=\left\{ {\hat{\gamma }}- \gamma _0\in A \right\}$, where A is defined in Lemma 2 and

$$\begin{aligned} \begin{aligned} \pi _3=\bigg \{&\sup _{\theta \in A,\Vert \theta \Vert _\mathbf{S}\le \xi }\left| \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K\left\{ {\tilde{L}}_{i,k}(\gamma _0+\theta ) -{\tilde{L}}_{i,k}(\gamma _0)\right\} -\frac{1}{n}\sum _{i=1}^n\sum _{k=1}^KE\left\{ {\tilde{L}}_{i,k}(\gamma _0+\theta ) -{\tilde{L}}_{i,k}(\gamma _0)\right\} \right| \\&\le C_BK\xi \sqrt{\frac{s\log p}{n}}\bigg \}. \end{aligned} \end{aligned}$$

By Lemma 1 and Lemma 2, when $\lambda \ge 2CK\sqrt{C_x}\sqrt{(\log p)/n}$, we have

$$\begin{aligned} P(\pi _2^c\cup \pi _3^c)\le P(\pi _2^c)+P(\pi _3^c)\le 2Kp^{1-C^2}+6p^{1-C^2}+4\delta _n. \end{aligned}$$

The following derivations are based on the condition that events $\pi _2$ and $\pi _3$ hold. Let $\xi =\Vert {\hat{\gamma }}- \gamma _0\Vert _\mathbf{S}$. By optimality condition, we have

$$\begin{aligned} \begin{aligned} \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K{\tilde{L}}_{i,k}({\hat{\gamma }})-\frac{1}{n}\sum _{i=1}^n \sum _{k=1}^K{\tilde{L}}_{i,k}(\gamma _0) -\lambda \left( \Vert \gamma _0\Vert _1 - \Vert {\hat{\gamma }}\Vert _1 \right) \le 0. \end{aligned} \end{aligned}$$

(B.1)

On the events $\pi _2$ and $\pi _3$,

$$\begin{aligned} \left| \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K\left\{ {\tilde{L}}_{i,k}({\hat{\gamma }}) -{\tilde{L}}_{i,k}(\gamma _0)\right\} -\frac{1}{n}\sum _{i=1}^n\sum _{k=1}^KE\left\{ {\tilde{L}}_{i,k}({\hat{\gamma }}) -{\tilde{L}}_{i,k}(\gamma _0)\right\} \right| \le C_BK\xi \sqrt{\frac{s\log p}{n}}. \end{aligned}$$

So it follows that

$$\begin{aligned} \begin{aligned} \frac{1}{n}\sum _{i=1}^n\sum _{k=1}^K\left\{ {\tilde{L}}_{i,k}({\hat{\gamma }}) -{\tilde{L}}_{i,k}(\gamma _0)\right\} \ge \frac{1}{2}\sum _{k=1}^Kf(c_k)\Vert {\hat{\gamma }}- \gamma _0\Vert _\mathbf{S}^2-C_BK\xi \sqrt{\frac{s\log p}{n}}. \end{aligned} \end{aligned}$$

(B.2)

Furthermore, as

$$\begin{aligned} \xi ^2=\Vert {\hat{\gamma }}- \gamma _0\Vert _\mathbf{S}^2=({\hat{\gamma }}- \gamma _0)^{\top }{} \mathbf{S}({\hat{\gamma }}-\gamma _0) \ge C_{\lambda }\Vert {\hat{\gamma }}- \gamma _0\Vert _2^2. \end{aligned}$$

Thus, $\Vert {\hat{\gamma }}- \gamma _0\Vert _2\le C_{\lambda }^{-1/2}\xi$. Therefore, on the event $\pi _2$, it holds that,

$$\begin{aligned} \begin{aligned}&\big \Vert \gamma _0\big \Vert _1-\big \Vert {\hat{\gamma }}\big \Vert _1\le \big \Vert {\hat{\gamma }}- \gamma _0\big \Vert _1=\big \Vert ({\hat{\gamma }}-\gamma _0)_T\big \Vert _1+\big \Vert ({\hat{\gamma }}- \gamma _0)_{T^c}\big \Vert _1\\&\quad \le 4\big \Vert ({\hat{\gamma }}- \gamma _0)_T\big \Vert _1\le 4\sqrt{s}\big \Vert {\hat{\gamma }}- \gamma _0\big \Vert _2 \le 4C^{-1/2}_{\lambda }\sqrt{s}\xi . \end{aligned} \end{aligned}$$

(B.3)

Then, by (B.1)–(B.3), we get

$$\begin{aligned} \frac{1}{2}\sum _{k=1}^Kf(c_k)\xi ^2-C_BK\xi \sqrt{\frac{s\log p}{n}}-4\lambda C^{-1/2}_{\lambda }\sqrt{s}\xi \le 0. \end{aligned}$$

As $\lambda \ge 2CK\sqrt{C_x}\sqrt{(\log p)/n}$, we can obtain

$$\begin{aligned} \begin{aligned} \Vert {\hat{\gamma }}- \gamma _0\Vert _\mathbf{S}&=\xi \le C_A\sqrt{\frac{s\log p}{n}},\\ \Vert {\hat{\gamma }}- \gamma _0\Vert _2&\le C_{\lambda }^{-1/2}\xi \le C_AC_{\lambda }^{-1/2}\sqrt{\frac{s\log p}{n}}, \end{aligned} \end{aligned}$$

where $C_A=2C_BC_{f_k}^{-1}+8CC_{f_k}^{-1}\sqrt{C_{x}}C_{\lambda }^{-1/2}$. $\square$

Proof of Theorem 3.1

By (3.1), we have

$$\begin{aligned} \begin{aligned} \sqrt{n}(\hat{{\gamma }}_d-{\gamma }_0)&= \sqrt{n}(\hat{{\gamma }}-{\gamma }_0)- \sqrt{n}\hat{\mathbf{\varSigma }}\hat{\mathbf{M}} = \sqrt{n}(\hat{{\gamma }}-{\gamma }_0)- \sqrt{n}{\hat{\theta }}_K^{-1}{} \mathbf{D}\hat{\mathbf{M}}\\&= \sqrt{n}(\hat{{\gamma }}-{\gamma }_0)-{\theta }_K^{-1}\sqrt{n}{} \mathbf{D}\hat{\mathbf{M}} -({\hat{\theta }}_K^{-1}-{\theta }_K^{-1})\sqrt{n}{} \mathbf{D}\hat{\mathbf{M}}. \end{aligned} \end{aligned}$$

(B.4)

Note that ${H}_{\tau _k}(\mathbf{X}^{\top }_i{{\gamma }}_0 )=c_k+g_0(\mathbf{X}^{\top }_i{{\gamma }}_0 )$, we have

$$\begin{aligned} \begin{aligned} \sqrt{n}{} \mathbf{D}\hat{\mathbf{M}}&=\mathbf{D}\frac{1}{\sqrt{n}}\sum _{i=1}^n \sum _{k=1}^K\left\{ {\hat{H}}'(\mathbf{X}^{\top }_{i}{\hat{{\gamma }}} )\right\} ^{\top }\left[ I\left\{ Y_i\le {\hat{H}}_{\tau _k}(\mathbf{X}^{\top }_i\hat{{\gamma }} )\right\} - {\tau _k} \right] \\&=\mathbf{D}\frac{1}{\sqrt{n}}\sum _{i=1}^n \sum _{k=1}^K\left\{ {\hat{H}}'(\mathbf{X}^{\top }_{i}{\hat{{\gamma }}} )\right\} ^{\top }\left[ I\left\{ \varepsilon _i\le {\hat{H}}_{\tau _k}(\mathbf{X}^{\top }_i\hat{{\gamma }} )-{H}_{\tau _k}(\mathbf{X}^{\top }_i{{\gamma }}_0 )+c_k\right\} - {\tau _k}\right] \\&=\mathbf{D}\frac{1}{\sqrt{n}}\sum _{i=1}^n \sum _{k=1}^K\left\{ {\hat{H}}'(\mathbf{X}^{\top }_{i}{\hat{{\gamma }}} )\right\} ^{\top }\left\{ I\left( \varepsilon _i\le c_k\right) - {\tau _k}\right\} \\&\quad +\mathbf{D}\frac{1}{\sqrt{n}}\sum _{i=1}^n \sum _{k=1}^K\left\{ {\hat{H}}'(\mathbf{X}^{\top }_{i}{\hat{{\gamma }}} )\right\} ^{\top }\left[ I\left\{ \varepsilon _i\le {\hat{H}}_{\tau _k}(\mathbf{X}^{\top }_i\hat{{\gamma }} )-{H}_{\tau _k}(\mathbf{X}^{\top }_i{{\gamma }}_0 )+c_k\right\} - I\left( \varepsilon _i\le c_k\right) \right] \\&\equiv \mathbf{T}_1+\mathbf{T}_2. \end{aligned} \end{aligned}$$

(B.5)

Thus, by (B.4) and (B.5), we have

$$\begin{aligned} \begin{aligned} \sqrt{n}(\hat{{\gamma }}_d-{\gamma }_0)= \sqrt{n}(\hat{{\gamma }}-{\gamma }_0)-{\theta }_K^{-1}\mathbf{T}_1-{\theta }_K^{-1}{} \mathbf{T}_2 -({\hat{\theta }}_K^{-1}-{\theta }_K^{-1})\sqrt{n}{} \mathbf{D}\hat{\mathbf{M}}. \end{aligned} \end{aligned}$$

(B.6)

We can proof that

$$\begin{aligned} \begin{aligned}&{\theta }_K^{-1}{} \mathbf{T}_1|\mathbf{U}\sim N\left( \mathbf{0},R\mathbf{D}{\hat{\mathbf{S}}}{} \mathbf{D}^{\top }\right) ,\\&{\theta }_K^{-1}{} \mathbf{T}_2=\mathbf{D} \hat{\mathbf{S}}\sqrt{n}(\hat{{\gamma }}-{{\gamma }}_0)+o_p(1),\\&\left\| ({\hat{\theta }}_K^{-1}-{\theta }_K^{-1})\sqrt{n}\mathbf{D}\hat{\mathbf{M}}\right\| _{\infty }=o_p(1), \end{aligned} \end{aligned}$$

(B.7)

where $R=\theta _K^{-2}\sum _{k=1}^{K}\sum _{k'=1}^{K}\tau _{kk'}$. Then, by (B.6) and (B.7), and condition $\alpha _1s\sqrt{\log p}=o(1)$,

$$\begin{aligned} \begin{aligned} \sqrt{n}(\hat{{\gamma }}_d-{\gamma }_0)&= \sqrt{n}(\hat{{\gamma }}-{\gamma }_0)-{\theta }_K^{-1}\mathbf{T}_1-\mathbf{D} \hat{\mathbf{S}}\sqrt{n}(\hat{{\gamma }}-{{\gamma }}_0)+o_p(1)\\&= -{\theta }_K^{-1}{} \mathbf{T}_1-\left( \mathbf{D} \hat{\mathbf{S}}-\mathbf{I}\right) \sqrt{n}(\hat{{\gamma }}-{{\gamma }}_0)+o_p(1)\\&= -{\theta }_K^{-1}{} \mathbf{T}_1-O_p(\alpha _1s\sqrt{\log p})+o_p(1)\\&= -{\theta }_K^{-1}{} \mathbf{T}_1+o_p(1). \end{aligned} \end{aligned}$$

(B.8)

By (B.8), we have

$$\begin{aligned} \frac{\sqrt{n}\left( \hat{{\gamma }}_{d,j}-{\gamma }_{0,j} \right) }{\left( {\hat{R}}{\hat{\mathbf{d}}}_j{\hat{\mathbf{S}}}{\hat{\mathbf{d}}}_j^{\top }\right) ^{1/2}} =\frac{-{\theta }_K^{-1}{} \mathbf{T}_{1,j}}{\left( {\hat{R}}{\hat{\mathbf{d}}}_j{\hat{\mathbf{S}}}{\hat{\mathbf{d}}}_j^{\top }\right) ^{1/2}} +\frac{o_p(1)}{\left( {\hat{R}}{\hat{\mathbf{d}}}_j{\hat{\mathbf{S}}}{\hat{\mathbf{d}}}_j^{\top }\right) ^{1/2}}. \end{aligned}$$

By the proof of Theorem 3 in Jiang et al. (2016) and condition C4, we can obtain

$$\begin{aligned} {\hat{R}}^{-1}\le 2(K+1)\left\{ \sum _{k=1}^Kf^2(c_k)- \sum _{k=1}^Kf(c_k) f(c_{k+1}) \right\} ^2\le 2K^2(K+1)C^4_{f}. \end{aligned}$$

By Lemma 3.1 in Javanmard and Montanari (2014), we have

$$\begin{aligned} {\hat{\mathbf{d}}_j}{\hat{\mathbf{S}}}{\hat{\mathbf{d}}_j}^{\top }\ge {(1-\alpha _1)^2}/{{\hat{\mathbf{S}}}_{jj}}, \end{aligned}$$

and based on condition C5, with probability tending to 1, we have ${\hat{\mathbf{S}}}_{jj}\le 2C_x$ for all $j\in \{1,\ldots ,p\}$. Then, we have with probability tending to 1 that

$$\begin{aligned} {\left( {\hat{R}}_1{\hat{\mathbf{d}}_j}{\hat{\mathbf{S}}}{\hat{\mathbf{d}}_j}^{\top }\right) ^{-1/2}} \le \frac{2C_xC^2_{f}\sqrt{2K^2(K+1)}}{(1-\alpha _1)^2}. \end{aligned}$$

Hence, with condition $\alpha _1s\sqrt{\log p}=o(1)$, for any $j\in \{1,\ldots ,p\}$,

$$\begin{aligned} \left( {\hat{R}}{\hat{\mathbf{d}}_j}^{\top }{\hat{\mathbf{S}}}{\hat{\mathbf{d}}_j}\right) ^{-1/2}\sqrt{n}\left( \hat{{\gamma }}_{d,j}-{\gamma }_{0,j} \right) \xrightarrow {L} N(0,1). \end{aligned}$$

which concludes the proof. $\square$