Information quantity evaluation of multivariate SETAR processes of order one and applications

Abstract

The Self-Exciting Threshold Autoregressive model (SETAR) is non-linear and considers threshold values to model time series affected by regimes. It is extended through the Multivariate SETAR (MSETAR) model, where the threshold variable can also be a multivariate process. The stationary marginal density (smd) of an MSETAR process of order one corresponds to a Unified Skew-Normal density. In this paper, the smd of an MSETAR of order one process was considered to compute explicit expressions of differential entropy and Kullback–Leibler and Jeffrey’s divergences between two MSETAR(1) processes. In addition, two asymptotic tests based on divergences were built for statistical significance testing of the disparity between MSETAR(1) processes and the threshold coefficient matrix. Information measures considered involved high-dimensional integrals that likewise depended on multivariate cumulative density normal function. To solve these integrals, Genz’s algorithm was considered based on Cholesky decomposition and Monte Carlo approximation. Some numerical experiments and applications to fish condition factor and Chilean economic perception time series illustrate performance.

Appendix

Proof of Proposition 2.2

The differential entropy (Cover and Thomas 2006) of a random vector $\textbf{X}$ with pdf (2.1) is defined by

$$\begin{aligned} H(\textbf{X})= & {} -\mathbb {E}\left[ \textrm{log}f_{\textbf{X}}(\textbf{x}|\varvec{\xi },\varvec{\Omega },\varvec{\Lambda },\varvec{\tau },\varvec{\Gamma })\right] \nonumber \\= & {} -\int _{\mathbb {R}^{k_1}}f_{\textbf{X}}(\textbf{x}|\varvec{\xi },\varvec{\Omega },\varvec{\Lambda },\varvec{\tau },\varvec{\Gamma })\textrm{log}\{f_{\textbf{X}}(\textbf{x}|\varvec{\xi },\varvec{\Omega },\varvec{\Lambda },\varvec{\tau },\varvec{\Gamma })\}d\textbf{x}. \end{aligned}$$

(5.1)

Based on (5.1), the logarithm is

$$\begin{aligned} \log f_{\textbf{X}}(\textbf{x}|\varvec{\xi },\varvec{\Omega },\varvec{\Lambda },\varvec{\tau },\varvec{\Gamma })=\log \phi _{k_1}(\varvec{\Omega }^{-1/2}(\textbf{x}-\varvec{\xi })) + \log \left[ \frac{\Phi _{k_2}(\varvec{\Lambda }^{\top }(\textbf{x}-\varvec{\xi })+\varvec{\tau }|\varvec{\Gamma })}{\Phi _{k_2}(\varvec{\tau }|\varvec{\Gamma }+\varvec{\Lambda }^{\top }\varvec{\Omega }\varvec{\Lambda })}\right] . \end{aligned}$$

Considering that differential entropy is invariant under translations of $\textbf{x}$ (Cover and Thomas 2006), through Proposition 3 of Arellano-Valle et al. (2013) we get

$$\begin{aligned} H(\textbf{X})=\underbrace{\frac{1}{2}\log \{\textrm{det}(\varvec{\Omega })\} - \mathbb {E}[\log \phi _{k_1}(\textbf{Z})]}_{H(\textbf{X}_N)} - \mathbb {E}\left[ \log \left\{ \frac{\Phi _{k_2}(\varvec{\Lambda }^{\top }(\textbf{X}-\varvec{\xi })+\varvec{\tau }|\varvec{\Gamma })}{\Phi _{k_2}(\varvec{\tau }|\varvec{\Gamma }+\varvec{\Lambda }^{\top }\varvec{\Omega }\varvec{\Lambda })}\right\} \right] ,\nonumber \\ \end{aligned}$$

(5.2)

where $\textbf{Z}=\varvec{\Omega }^{-1/2}(\textbf{X}-\varvec{\xi })\sim N(\textbf{0},\textbf{I})$ and $\textbf{X}_N\sim N(\textbf{0},\varvec{\Omega })$. Thus, $H(\textbf{X}_N)=\frac{1}{2}\log \{(2\pi e)^{k_1}\textrm{det}(\varvec{\Omega })\}$ and the result is obtained. $\square$

Proof of Proposition 2.3

Consider the pdf of $\textbf{X}_i$ given by (2.3), $f_{\textbf{X}_i}(\textbf{x}_i|\textbf{0},\textbf{P}_i,-\textbf{B}_i,\textbf{0},\textbf{L}_i^{-1})$, $i=1,2$. The KL divergence between $\textbf{X}_1$ and $\textbf{X}_2$ is

$$\begin{aligned} K(\textbf{X}_1,\textbf{X}_2)= & {} \int _{\mathbb {R}^{k_1}} f_{\textbf{X}_1}(\textbf{x}_2|\textbf{0},\textbf{P}_1,-\textbf{B}_1,\textbf{0},\textbf{L}_1^{-1}) \log \left\{ \frac{f_{\textbf{X}_1}(\textbf{y}|\textbf{0},\textbf{P}_1,-\textbf{B}_1,\textbf{0},\textbf{L}_1^{-1})}{f_{\textbf{X}_2}(\textbf{y}|\textbf{0},\textbf{P}_2,-\textbf{B}_2,\textbf{0},\textbf{L}_2^{-1})}\right\} d\textbf{y}\\= & {} \underbrace{\mathbb {E}[\log f_{\textbf{X}_1}(\textbf{X}_1|\textbf{0},\textbf{P}_1,-\textbf{B}_1,\textbf{0},\textbf{L}_1^{-1})\}]}_{-H(\textbf{X}_1)}+\,\mathbb {E}[-\log f_{\textbf{X}_2}(\textbf{X}_1|\textbf{0},\textbf{P}_2,-\textbf{B}_2,\textbf{0},\textbf{L}_2^{-1})\}]. \end{aligned}$$

The second term of the last equation is the so-called cross-entropy and depends on the logarithm of the pdf of $\textbf{X}_2$ due to (2.3),

$$\begin{aligned} \log f_{\textbf{X}_2}(\textbf{x}_1|\textbf{0},\textbf{P}_2,-\textbf{B}_2,\textbf{0},\textbf{L}_2^{-1})= & {} \log \phi _k(\textbf{P}_2^{-1}\textbf{x}_1) + \log \left[ \frac{\Phi _{k}(-\textbf{B}_2^{\top }\textbf{x}_1|\textbf{L}_2^{-1})}{2^k}\right] \\= & {} -\frac{1}{2}[k\log (2\pi )+\log \{\textrm{det}(\textbf{P}_2)\}+\textbf{x}_1^{\top }\textbf{P}_2^{-1}\textbf{x}_1]\\{} & {} \,+ \log \left[ \frac{\Phi _{k}(-\textbf{B}_2^{\top }\textbf{x}_1|\textbf{L}_2^{-1})}{2^k}\right] , \end{aligned}$$

and

$$\begin{aligned} \mathbb {E}[-\log f_{\textbf{X}_2}(\textbf{x}_1|\textbf{0},\textbf{P}_2,-\textbf{B}_2,\textbf{0},\textbf{L}_2^{-1})\}]= & {} \frac{1}{2}\{k\log (2\pi )+\log \{\textrm{det}(\textbf{P}_2)\}+\mathbb {E}[\textbf{X}_1^{\top }\textbf{P}_2^{-1}\textbf{X}_1]\}\\{} & {} \,- \mathbb {E}\left[ \log \left\{ \frac{\Phi _{k}(-\textbf{B}_2^{\top }\textbf{X}_1|\textbf{L}_2^{-1})}{2^k}\right\} \right] .\\ \end{aligned}$$

Given that mean vector of $\textbf{X}_i$ is zero, by part (ii) of Lemma 2.1 the expected value of the quadratic form of the latter equation is $\mathbb {E}[\textbf{X}_1^{\top }\textbf{P}_2^{-1}\textbf{X}_1]=\textrm{tr}(\textbf{P}_2^{-1}\textbf{P}_1)$ and

$$\begin{aligned} K(\textbf{X}_1,\textbf{X}_2)= & {} \frac{1}{2}\{k\log (2\pi )+\log \{\textrm{det}(\textbf{P}_2)\}+\textrm{tr}(\textbf{P}_2^{-1}\textbf{P}_1)\} - \mathbb {E}\left[ \log \left\{ \frac{\Phi _{k}(-\textbf{B}_2^{\top }\textbf{X}_1|\textbf{L}_2^{-1})}{2^k}\right\} \right] \\{} & {} -\frac{1}{2}\log \{(2\pi e)^k\textrm{det}(\textbf{P}_1)\} + \mathbb {E}\left[ \log \left\{ \frac{\Phi _{k}(-\textbf{B}_1^{\top }\textbf{X}_1|\textbf{L}_1^{-1})}{2^k}\right\} \right] .\,\Box \end{aligned}$$

$\square$

Proof of Proposition 3.1

Considering the proof of Lemma 6 of Cai et al. (2010), it is proved that the KL divergence based on covariance matrix estimates $\textbf{P}_1$ and $\textbf{P}_2$ has the following upper bound:

$$\begin{aligned} K({\varvec{\Theta }}_1,{\varvec{\Theta }}_2)= & {} \frac{1}{2}\left[ \textrm{tr}(\textbf{P}_2^{-1}\textbf{P}_1)-\log \{\textrm{det}(\textbf{P}_2^{-1}\textbf{P}_1)\} - k\right] \\\le & {} c_1 a b^2,\nonumber \end{aligned}$$

where $c_1$ is a positive constant, $a=n^{1/(2\gamma +1)}$ and $b=a^{-(\gamma +1)}$. $\gamma$ is a smoothness parameter whose optimal value found by Cai et al. (2010) was $\gamma =1/2$. Thus,

$$\begin{aligned} K({\varvec{\Theta }}_1,{\varvec{\Theta }}_2)\le c_1 n^{-1/2}. \end{aligned}$$

(5.3)

Considering Corollary 2.4, the J divergence (3.8) can be expressed in terms of KL divergences of two multivariate normal random variables given by (see also Arellano-Valle et al. 2017). Therefore,

$$\begin{aligned} |J(\widehat{{\varvec{\Theta }}}_1,\widehat{{\varvec{\Theta }}}_2) - J({\varvec{\Theta }}_1,{\varvec{\Theta }}_2)|\le & {} |K(\widehat{{\varvec{\Theta }}}_1,\widehat{{\varvec{\Theta }}}_2) - K({\varvec{\Theta }}_1,{\varvec{\Theta }}_2)|+|K(\widehat{{\varvec{\Theta }}}_2,\widehat{{\varvec{\Theta }}}_1) - K({\varvec{\Theta }}_2,{\varvec{\Theta }}_1)|\nonumber \\{} & {} \,+ \sum _{i=1}^2 |\widehat{I}_{ii}-I_{ii}| + \sum _{\begin{array}{c} i,j=1\\ i\ne j \end{array}}^2 |\widehat{I}_{ij} - I_{ij}|, \end{aligned}$$

(5.4)

where the triangular inequality was used considering that KL divergence is non-negative. Using (3.4) and (5.3) and applying the expected value to both sides of (5.4), we get

$$\begin{aligned} \mathbb {E}\left[ |J(\widehat{{\varvec{\Theta }}}_1,\widehat{{\varvec{\Theta }}}_2) - J({\varvec{\Theta }}_1,{\varvec{\Theta }}_2)|\right] \le 2(c_1 n^{-1/2}+ c_2 N^{-1/2})\\ = c_3(n^{-1/2}+N^{-1/2}), \end{aligned}$$

where $c_2$ and $c_3$ are two positive constants. Thus, the result is obtained. $\square$