Predicting Extreme Financial Risks on Imbalanced Dataset: A Combined Kernel FCM and Kernel SMOTE Based SVM Classifier

Abstract

Extreme financial risk prediction is an important component of risk management in financial markets. In this study, taking the China Securities Index 300 (CSI300) as an example, we set out to introduce the kernel method into fuzzy c-mean algorithm (FCM) and synthetic minority over-sampling technique (SMOTE) and combine them with support vector machine (SVM) to propose a hybrid model of KFCM-KSMOTE-SVM for predicting extreme financial risks, which is compared with other various prediction models. In addition, we investigate the influence on the prediction performance of KFCM-KSMOTE-SVM exerted by its parameters. The empirical results present that KFCM-KSMOTE-SVM outperforms other various prediction models significantly, which verifies that KFCM-KSMOTE-SVM can solve the class imbalance problem in financial markets and is more appropriate for predicting extreme financial risks. Meanwhile, parameter set plays an important role in constructing KFCM-KSMOTE-SVM prediction model. Besides, the experiment on Shanghai Stock Exchange Composite Index also proves that KFCM-KSMOTE-SVM has strong robustness on predicting extreme financial risks.

Appendix A: The Process of Kernel Map** in KSMOTE

Given positive instances $x_{q}$ and $x_{a}$ where $q = 1,2, \ldots ,\left| P \right|$ and $a = 1,2, \ldots ,\left| P \right|$, in order to find $k$ nearest neighbors of $x_{q}$, the distance between $x_{q}$ and $x_{a}$ in the high dimensional feature spaces given by:

$$ d_{qa} = \left\| {\varPhi (x_{q} ) - \varPhi (x_{a} )} \right\| = \sqrt {k(x_{q} ,x_{q} ) - 2k(x_{q} ,x_{a} ) + k(x_{a} ,x_{a} )} $$

(29)

Formula (30) can be simplified by substituting formula (8) to formula (29):

$$ d_{qa} = \left\| {\varPhi (x_{q} ) - \varPhi (x_{a} )} \right\| = \sqrt {2 - 2\exp \left( {{{ - \left\| {x_{q} - x_{a} } \right\|^{2} } \mathord{\left/ {\vphantom {{ - \left\| {x_{q} - x_{a} } \right\|^{2} } {\left( {2\sigma^{2} } \right)}}} \right. \kern-0pt} {\left( {2\sigma^{2} } \right)}}} \right)} $$

(30)

where the distance $d_{qa}$ is converted into $+ \infty$ when $q = a$ and all $d_{qa}$ construct a $\left| P \right| \times \left| P \right|$ matrix, every row of which is denoted as the distance $d_{q}$ between $x_{q}$ and all instances of $x_{a}$. For $d_{q}$,all instances of $x_{a}$ are sorted by $d_{q}$ in ascending order row by row, which constitutes a new instance set $S$ being a $\left| P \right| \times \left| P \right|$ matrix. Finally, the first $k$ instances from each row of $S$ are selected as $k$ nearest neighbors of $x_{q}$ in the high dimensional feature space, which consists of a nearest neighbor set $D = \left\{ {x_{v}^{q} ,v = 1,2, \ldots ,k} \right\}$ of $x_{q}$ in the vector space.

In addition, in order to find original instances corresponding to generated instances, the relationship of distance between the vector space and the high dimensional feature space should be obtained firstly.

For the nearest neighbor set $D = \left\{ {x_{v}^{q} ,v = 1,2, \ldots ,k} \right\}$ of $x_{q}$, the distance between $x_{v}^{q}$ in the high dimensional feature space and $O_{j}^{q}$ is given by:

$$ \begin{aligned} d_{q}^{2} \left({O_{j}^{q},\varPhi (x_{v}^{q})} \right) & = d_{v}^{2} \left({\varPhi (x_{q}) + \lambda_{j}^{q} \times \left({\varPhi (x_{j}) - \varPhi (x_{q})} \right),\varPhi (x_{v}^{q})} \right) \\ & = \left\| {\varPhi (x_{q}) + \lambda_{j}^{q} \times \left({\varPhi (x_{j}) - \varPhi (x_{q})} \right) - \varPhi (x_{v}^{q})} \right\|^{2} \\ & = (1 + 2\lambda_{j}^{q})k(x_{v}^{q},x_{v}^{q}) - 2k(x_{v}^{q},x_{q}) - 2\lambda_{j}^{q} k(x_{v}^{q},x_{j}) \\ & \quad + (\lambda_{j}^{q} - 1)^{2} k(x_{q},x_{q}) + 2\lambda_{j}^{q} (1 - \lambda_{j}^{q})k(x_{q},x_{j}) + \left({\lambda_{j}^{q}} \right)^{2} k(x_{j},x_{j}) \\ & = 2\left[{\left({\lambda_{j}^{q}} \right)^{2} + 1 - k(x_{v}^{q},x_{q}) - \lambda_{j}^{q} k(x_{v}^{q},x_{j}) + \lambda_{j}^{q} (1 - \lambda_{j}^{q})k(x_{q},x_{j})} \right] \\ & = 2\left[{\left({\lambda_{j}^{q}} \right)^{2} + 1 - \exp \left({- \left\| {x_{v}^{q} - x_{q}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right) }\right. \\ &\left. \quad {-\, \lambda_{j}^{q} \exp \left({- \left\| {x_{v}^{q} - x_{j}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right)} \right. \\ &\left. \quad +\, \lambda_{j}^{q} (1 - \lambda_{j}^{q})\exp \left({- \left\| {x_{q} - x_{j}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right) \vphantom{\left({\lambda_{j}^{q}} \right)^{2}}\right] \\ \end{aligned} $$

(31)

Simultaneously, the distance between $x_{v}^{q}$ and the original instance $u_{j}^{q}$ in the vector space of $O_{j}^{q}$ in the high dimensional feature space is obtained by:

$$ \begin{aligned} d_{q}^{2} \left( {\varPhi (u_{j}^{q} ),\varPhi (x_{v}^{q} )} \right) & = \left\| {\varPhi (u_{j}^{q} ) - \varPhi (x_{v}^{q} )} \right\|^{2} = k(u_{j}^{q} ,u_{j}^{q} ) - 2k(u_{j}^{q} ,x_{v}^{q} ) + k(x_{v}^{q} ,x_{v}^{q} ) \\ & = 2 - 2\exp \left( { - \left\| {u_{j}^{q} - x_{v}^{q} } \right\|^{2} /\left( {2\sigma^{2} } \right)} \right) \\&= 2 - 2\exp \left( { - d_{q}^{2} (u_{j}^{q} ,x_{v}^{q} )/\left( {2\sigma^{2} } \right)} \right) \\ \end{aligned} $$

(32)

and hence,

$$ d_{q}^{2} (u_{j}^{q} ,x_{v}^{q} ) = - 2\sigma^{2} \ln \left( {1 - \frac{1}{2}d_{q}^{2} \left( {\varPhi (u_{j}^{q} ),\varPhi (x_{v}^{q} )} \right)} \right) $$

(33)

From formula (33), the relationship of distance between the vector space and the high dimensional feature space can be obtained. Moreover, because $d_{q}^{2} \left( {\varPhi (u_{j}^{q} ),\varPhi (x_{v}^{q} )} \right) = d_{q}^{2} \left( {O_{j}^{q} ,\varPhi (x_{v}^{q} )} \right)$, formula (34) can be simplified by substituting formula (31) to formula (33):

$$ \begin{aligned} d_{q}^{2} (u_{j}^{q},x_{v}^{q}) & = - 2\sigma^{2} \ln \left({1 - \frac{1}{2}d_{q}^{2} \left({\varPhi (u_{j}^{q}),\varPhi (x_{v}^{q})} \right)} \right) \\ & = - 2\sigma^{2} \ln \left({- \left({\lambda_{j}^{q}} \right)^{2} + \exp \left({- \left\| {x_{v}^{q} - x_{q}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right)} \right. \\ & \left.\quad + \lambda_{j}^{q} \exp \left({- \left\| {x_{v}^{q} - x_{j}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right) - \lambda_{j}^{q} (1 - \lambda_{j}^{q})\right.\\&\quad\left.\times\exp \left({- \left\| {x_{q} - x_{j}} \right\|^{2}/\left({2\sigma^{2}} \right)} \right) \right) \\ \end{aligned} $$

(34)

Generally, the distance between an instance and its nearest neighbors plays an important role in the process of locating the instance. Therefore, the vector $d^{2}$ is denoted as the distance between the original instance $u_{j}^{q}$ of $O_{j}^{q}$ and the nearest neighbor set $D = \left\{ {x_{v}^{q} ,v = 1,2, \ldots ,k} \right\}$ of $x_{q}$ is shown as follows:

$$ d_{v}^{2} = \left[ {d_{1}^{2} ,d_{2}^{2} , \ldots ,d_{k}^{2} } \right]^{T} $$

(35)

where $d_{v}^{2} ,v = 1,2, \ldots ,k$ is the distance between $u_{j}^{q}$ and nearest neighbors $x_{v}^{q}$ in the vector space.

In the light of Kwok and Tsang (2004) and Gower (1968) that the coordinate of an instance is defined by the distance constraint between the instance and other instances, the original instance $u_{j}^{q}$ of $O_{j}^{q}$ can be located. For $D = \left\{ {x_{v}^{q} ,v = 1,2, \ldots ,k} \right\}$, the mean value $\bar{x} = (1/k)\sum\nolimits_{v = 1}^{k} {x_{v}^{q} }$ of original instances $\left\{ {x_{1}^{q} ,x_{2}^{q} , \ldots ,x_{k}^{q} } \right\}$ corresponding to $k$ nearest neighbors $\left\{ {\varPhi (x_{1}^{q} ),\varPhi (x_{2}^{q} ), \ldots ,\varPhi (x_{k}^{q} )} \right\}$ of $x_{q}$ in the high dimensional feature space is denoted as the centroid of $\left\{ {x_{1}^{q} ,x_{2}^{q} , \ldots ,x_{k}^{q} } \right\}$ and a new coordinate system can be defined.

First, construct a matrix $X_{v}^{q} = \left[ {x_{1}^{q} ,x_{2}^{q} , \ldots ,x_{k}^{q} } \right]$ and a $k \times k$ centering matrix $H$ given by:

$$ H = I - \frac{1}{k}LL^{T} $$

(36)

where $I$ is a $k \times k$ identity matrix, $L = \left[ {1,1, \ldots ,1} \right]^{T}$ is a $k \times 1$ vector. So $X_{v}^{q} H$ is a centering matrix with $\bar{x}$-centered as follows:

$$ X_{v}^{q} H = \left[ {x_{1}^{q} - \bar{x},x_{2}^{q} - \bar{x}, \ldots ,x_{k}^{q} - \bar{x}} \right] $$

(37)

Assuming that the rank of the centering matrix $X_{v}^{q} H$ is $p$, the singular value decomposition (SVD) of $X_{v}^{q} H$ can be obtained as:

$$ X_{v}^{q} H = \left[ {U_{1} ,U_{2} } \right]\left[ {\begin{array}{*{20}c} {\varLambda_{1} } & O \\ O & O \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {V_{1}^{T} } \\ {V_{2}^{T} } \\ \end{array} } \right] = U_{1} \varLambda_{1} V_{1}^{T} = U_{1} \varGamma $$

(38)

where $U_{1} = \left[ {e_{1} ,e_{2} , \ldots ,e_{p} } \right]$ is a matrix with orthonormal columns $e_{i}$,$i = 1,2, \ldots ,p$, $\varGamma = \varLambda_{1} V_{1}^{T} = \left[ {c_{1} ,c_{2} , \ldots ,c_{k} } \right]$ is a matrix with columns $c_{v}$ being the projections of $x_{v}^{q} - \bar{x}$ onto the $U_{1}$’s with $\left\| {c_{v} } \right\|^{2} = \left\| {x_{v}^{q} - \bar{x}} \right\|^{2}$. Again, a $k \times 1$ vector $d_{c}^{2} = \left[ {\left\| {c_{1} } \right\|^{2} ,\left\| {c_{2} } \right\|^{2} , \ldots ,\left\| {c_{k} } \right\|^{2} } \right]^{T}$ is obtained. Obviously, for the sake of gaining the approximate original instance $u_{j}^{q}$, the distance $d_{q}^{2} (u_{j}^{q} ,x_{v}^{q} )$ is as close to those values obtained in formula (35) as possible, i.e.,

$$ d_{q}^{2} (u_{j}^{q} ,x_{v}^{q} ) \approx d_{v}^{2} $$

(39)

Then, define $c \in R^{p \times 1}$ with $U_{1} c = u_{j}^{q} - \bar{x}$, so

$$ \begin{aligned} d_{v}^{2} \approx d_{q}^{2} (u_{j}^{q} ,x_{v}^{q} ) = \left\| {u_{j}^{q} - x_{v}^{q} } \right\|^{2} & = \left\| {(u_{j}^{q} - \bar{x}) - (x_{v}^{q} - \bar{x})} \right\|^{2} \\ & = \left\| c \right\|^{2} + \left\| {c_{v} } \right\|^{2} - 2(u_{j}^{q} - \overline{x} )(x_{v}^{q} - \overline{x} ) \\ \end{aligned} $$

(40)

Considering that the centering matrix $X_{v}^{q} H$ can make the cumulative of the inner product zero in formula (40), formula (41) can be simplified by the cumulative of formula (40):

$$ \sum\limits_{v = 1}^{k} {d_{v}^{2} } = k\left\| c \right\|^{2} + \sum\limits_{v = 1}^{k} {\left\| {c_{v} } \right\|^{2} } \Rightarrow \left\| c \right\|^{2} = \frac{1}{k}\sum\limits_{v = 1}^{k} {\left( {d_{v}^{2} - \left\| {c_{v} } \right\|^{2} } \right)} $$

(41)

Substitute for $\left\| c \right\|^{2}$ in formula (40) with formula (41):

$$ 2(u_{j}^{q} - \bar{x})^{T} (x_{v}^{q} - \bar{x}) = \left\| {c_{v} } \right\|^{2} - d_{v}^{2} - \frac{1}{k}\sum\limits_{v = 1}^{k} {\left( {\left\| {c_{v} } \right\|^{2} - d_{v}^{2} } \right)} $$

(42)

Formula (42) is transformed into the matrix form satisfying:

$$ 2\varGamma^{T} c = \left( {d_{c}^{2} - d_{v}^{2} } \right) - \frac{1}{k}LL^{T} \left( {d_{c}^{2} - d_{v}^{2} } \right) $$

(43)

Now, $\varGamma LL^{T} = 0$ because $\varGamma$ is the centering matrix. Hence, formula (43) is transformed further into:

$$ c = \frac{1}{2}\left( {\varGamma \varGamma^{T} } \right)^{ - 1} \varGamma \left( {d_{c}^{2} - d_{v}^{2} } \right) = \frac{1}{2}\varLambda_{1}^{ - 1} V_{1}^{T} \left( {d_{c}^{2} - d_{v}^{2} } \right) $$

(44)

At last, $u_{j}^{q}$ can be obtained by transforming $c$ in formula (44) back to the original coordinate system in the vector space:

$$ u_{j}^{q} = \frac{1}{2}U_{1} \varLambda_{1}^{ - 1} V_{1}^{T} \left( {d_{c}^{2} - d_{v}^{2} } \right) + \bar{x} $$

(45)

After obtaining $u_{j}^{q}$ in step8 via above, synthetic positive instances $u_{j}^{q}$ can be added to original positive instance set $P = \{ x_{q} ,q = 1,2, \ldots ,\left| P \right|\}$ and a new positive instance set $P_{new}$ is gained.