Tucker-3 decomposition with sparse core array using a penalty function based on Gini-index

Abstract

Tucker-3 decomposition is a dimension reduction method for tensor data, similar to principal component analysis. One of the characteristics of Tucker-3 is the core array, which represents the interactions between low-dimensional spaces. However, it is difficult to interpret the result when the number of elements in the core array is large. One solution to this problem is using sparse estimation, such as the L1 regularization method, for the core array. However, some regularization methods often sacrifice the model fit too much. To solve this issue, we propose a novel estimation method for Tucker-3 decomposition with a penalty function based on the Gini index, which is a measure of sparsity and variance. Maximizing the Gini index is expected to obtain an estimated value for the core array that is easy to interpret. Moreover, the model fitted to the data will not involve shrinkage much because the Gini index is a measure of variance, which is one of the model fit measures of Tucker-3. The nonconvex problem poses a challenge when using the proposed penalty function based on the Gini index. To address this problem, we develop a majorization–minimization algorithm. From a numerical example, we revealed that the performance (the precision and accurate prediction of zero cells) of our method is superior to that of the estimation method with existing penalties, such as the L1 penalty, smoothly clipped absolute deviation, and minimax concave penalty.

Appendix A: Deriving the updated formula of the core array

Let $\varvec{a} \in {\mathbb {R}}^{p}$ be the parameter vector and $\varvec{x}\in {\mathbb {R}}^{p}$ be the constant vector. We consider minimizing the objective function f defined as follows:

$$\begin{aligned}&f(\varvec{a}|\varvec{x},\lambda , \alpha , \{w_i \mid i = 1, \,2,\, \dotsc ,\, p\},\{\beta _i \mid i = 1, \,2,\, \dotsc ,\, p\})\\&\quad = \dfrac{1}{2}\Vert \varvec{x}-\varvec{a}\Vert ^2_2 + \lambda \left( \dfrac{1}{\alpha }\sum _{i=1}^{p} w_i{|a_{i}|}-\sum _{i=1}^p \beta _i \log ({|a_i|}+\epsilon )\right) \\&\quad =\sum _{i=1}^{p}\left( \dfrac{1}{2}( x_i -a_i\right) ^2 + \lambda \left( \dfrac{1}{\alpha } w_i{|a_{i}|}-\beta _i \log ({|a_i|}+\epsilon )) \right) , \end{aligned}$$

where $\alpha> 0,w_i \ge 0, \beta _i \in [0,1], \epsilon >0$ is constant. For minimizing f, we minimize each $( ( x_i -a_i)^2/2 + \lambda (\dfrac{1}{\alpha } w_i{|a_{i}|}-\beta _i \log ({|a_i|}+\epsilon )) )$. We set $g(a_i|x_i) =( ( x_i -a_i)^2/2 + \lambda (\dfrac{1}{\alpha } w_i{|a_{i}|}-\beta _i \log ({|a_i|}+\epsilon )) )$. The second term of $g(a_i|x_i)$ is independent the sign of $a_i$. Thus, the sign of the optimal solution of $g(a_i|x_i)$ is the same as $x_i$ when $x_i \ne 0$ because $-2x_ia_i \le -2x_i c$under the same sign of x and $a_i$ and $|a_i| = |c|$.

For the case $x_i >0$, the candidate set of optimal solution of g is $a_i>0$. $g(a_i|x_i)$ is the convex function if $a_i>0$. When setting the derivative of $g(a_i|x_i)$ as 0, we obtain the equation as follows:

$$\begin{aligned}&a_i -x + \lambda \left( \dfrac{w_i}{\alpha }-\dfrac{\beta _i}{\left( a_i + \epsilon \right) }\right) = 0 \\&\quad \Longleftrightarrow (a_i-x)(a_i + \epsilon ) + \lambda \dfrac{w_i}{\alpha } (a_i + \epsilon ) -\lambda \beta _i= 0\\&\quad \Longleftrightarrow a^2_i+ \left( -x +\epsilon + \lambda \dfrac{w_i }{\alpha }\right) a_i + \left( -x \epsilon +\lambda \left( \dfrac{w_i \epsilon - \beta _i\alpha }{\alpha }\right) \right) = 0. \end{aligned}$$

In the case $a_i>0$, the candidates of extremal value of g are obtained as follows:

$$\begin{aligned} a_i&= \dfrac{1}{2}\left( x- \left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) \pm \sqrt{\left( x+\epsilon -\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}\right) . \end{aligned}$$

Because $x- (\epsilon + \lambda {w_i }/{\alpha }) <\sqrt{(x+\epsilon -\lambda {w_i }/{\alpha })^2 +4 \lambda \beta _i}$ holds, in the case $a_i>0$, the candidate of extremal value of g is

$$\begin{aligned} a_i&= \dfrac{1}{2}\max \left\{ 0,\,{x- \left( \epsilon + \lambda {w_i }{\alpha }\right) + \sqrt{\left( x+\epsilon -\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}} \right\} . \end{aligned}$$

For the case $x_i <0$, by the same way as the case $x_i>0$, the candidate of extremal value of g is obtained as follows:

$$\begin{aligned} a_i&= \dfrac{1}{2}\left( x +\left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) \pm \sqrt{\left( x -\epsilon +\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}\right) . \end{aligned}$$

$x + (\epsilon + \lambda w_i /\alpha ) +\sqrt{(x -\epsilon +\lambda {w_i }/{\alpha })^2 +4 \lambda \beta _i}$ is also positive because $x + (\epsilon + \lambda w_i /\alpha ) + |x-\epsilon +\lambda {w_i }/{\alpha }|$ is a positive value. Thus, in the case $a_i<0$, the candidate of extremal value of g as

$$\begin{aligned} a_i&= \dfrac{1}{2}\min \left\{ 0,{x + (\epsilon + \lambda {w_i }{\alpha }) - \sqrt{\left( x -\epsilon +\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}}\right\} . \end{aligned}$$

If $x_i=0$, the optimal points of $g(a_i|x_i)$ are

$$\begin{aligned} \dfrac{ \left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) - \sqrt{\left( \epsilon -\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}}{2} \mathrm {and} \dfrac{ -\left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) + \sqrt{\left( -\epsilon +\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}}{2}. \end{aligned}$$

Thus, we obtain the optimal points of $g(a_i|x_i)$ as

$$\begin{aligned} a_i = {\left\{ \begin{array}{ll} \dfrac{1}{2}\min \left\{ 0,{x + \left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) - \sqrt{\left( x -\epsilon +\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}}\right\} &{}(x<0)\\ \\ \dfrac{1}{2}\max \left\{ 0,\,{x- \left( \epsilon + \lambda \dfrac{w_i }{\alpha }\right) + \sqrt{\left( x+\epsilon -\lambda \dfrac{w_i }{\alpha }\right) ^2 +4 \lambda \beta _i}}\right\} &{}(x\ge 0) \end{array}\right. }. \end{aligned}$$

Therefore, we obtain the updated formula of $\varvec{G}_i$ when we set $\varvec{x} = \varvec{\eta }+\varvec{M}'_g(\mathrm {Vec}(\varvec{X}_1)- \varvec{M}_g\varvec{\eta })$, $\epsilon = \epsilon /R$, and $\lambda = \lambda /L$.