Abstract
Independent component analysis (ICA) is a popular unsupervised learning method. This paper extends it to multilinear modewise ICA (MMICA) for tensors and explores two architectures in learning and recognition. MMICA models tensor data as mixtures generated from modewise source matrices that encode statistically independent information. Its sources have more compact representations than the sources in ICA. We embed ICA into the multilinear principal component analysis framework to solve for each source matrix alternatively with a few iterations. Then we obtain mixing tensors through regularized inverses of the source matrices. Simulations on synthetic data show that MMICA can estimate hidden sources accurately from structured tensor data. Moreover, in face recognition experiments, it outperforms competing solutions with both architectures.
Chapter PDF
Similar content being viewed by others
Keywords
References
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons (2001)
Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer Series in Statistics (2002)
Hyvärinen, A., Oja, E.: Independent component analysis: Algorithms and applications. Neural Networks 13(4-5), 411–430 (2000)
Bartlett, M.S., Movellan, J.R., Sejnowski, T.J.: Face recognition by independent component analysis. IEEE Transactions on Neural Networks 13(6), 1450–1464 (2002)
Shakhnarovich, G., Moghaddam, B.: Face recognition in subspaces. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, pp. 141–168 (2004)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: A survey of multilinear subspace learning for tensor data. Pattern Recognition 44(7), 1540–1551 (2011)
Faloutsos, C., Kolda, T.G., Sun, J.: Mining large time-evolving data using matrix and tensor tools. In: International Conference on Machine Learning 2007 Tutorial (2007), http://www.cs.cmu.edu/~christos/TALKS/ICML-07-tutorial/ICMLtutorial.pdf
Ye, J.: Generalized low rank approximations of matrices. Machine Learning 61(1-3), 167–191 (2005)
Ye, J., Janardan, R., Li, Q.: GPCA: An efficient dimension reduction scheme for image compression and retrieval. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 354–363 (2004)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: Multilinear principal component analysis of tensor objects. IEEE Transactions on Neural Networks 19(1), 18–39 (2008)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Uncorrelated multilinear principal component analysis through successive variance maximization. In: International Conference on Machine Learning, pp. 616–623 (2008)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning. IEEE Transactions on Neural Networks 20(11), 1820–1836 (2009)
Ye, J., Janardan, R., Li, Q.: Two-dimensional linear discriminant analysis. In: Advances in Neural Information Processing Systems (NIPS), pp. 1569–1576 (2004)
Yan, S., Xu, D., Yang, Q., Zhang, L., Tang, X., Zhang, H.J.: Discriminant analysis with tensor representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 526–532 (2005)
Tao, D., Li, X., Wu, X., Maybank, S.J.: General tensor discriminant analysis and gabor features for gait recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(10), 1700–1715 (2007)
Tao, D., Li, X., Wu, X., Maybank, S.J.: Tensor rank one discriminant analysis-a convergent method for discriminative multilinear subspace selection. Neurocomputing 71(10-12), 1866–1882 (2008)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition. IEEE Transactions on Neural Networks 20(1), 103–123 (2009)
Lee, S.H., Choi, S.: Two-dimensional canonical correlation analysis. IEEE Signal Processing Letters 14(10), 735–738 (2007)
Kim, T.K., Cipolla, R.: Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(8), 1415–1428 (2009)
Lu, H.: Learning canonical correlations of paired tensor sets via tensor-to-vector projection. In: The 23rd International Joint Conference on Artificial Intelligence, pp.1516–1522 (2013)
Lathauwer, L.D., Vandewalle, J.: Dimensionality reduction in higher-order signal processing and rank-(R 1,R 2,., R N ) reduction in multilinear algebra. Linear Algebra and its Applications 391, 31–55 (2004)
Beckmann, C.F., Smith, S.M.: Tensorial extensions of independent component analysis for multisubject fMRI analysis. NeuroImage 25(1), 294–311 (2005)
Vasilescu, M.A.O., Terzopoulos, D.: Multilinear independent components analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 547–553 (2005)
Raj, R.G., Bovik, A.C.: MICA: A multilinear ICA decomposition for natural scene modeling. IEEE Transactions on Image Processing 17(3), 259–271 (2009)
Zhang, L., Gao, Q., Zhang, D.: Directional independent component analysis with tensor representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2008)
Gao, Q., Zhang, L., Zhang, D., Xu, H.: Independent components extraction from image matrix. Pattern Recognition Letters 31(3), 171–178 (2010)
Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10(3), 626–634 (1999)
Lathauwer, L.D., Moor, B.D., Vandewalle, J.: On the best rank-1 and rank-(R 1,R 2,.,R N ) approximation of higher-order tensors. SIAM Journal of Matrix Analysis and Applications 21(4), 1324–1342 (2000)
Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singualr value decomposition. SIAM Journal of Matrix Analysis and Applications 21(4), 1253–1278 (2000)
Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review 51(3), 455–500 (2009)
Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)
Friedman, J.H.: Regularized discriminant analysis. Journal of the American Statistical Association 84(405), 165–175 (1989)
Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Boosting discriminant learners for gait recognition using MPCA features. EURASIP Journal on Image and Video Processing 2009, Article ID 713183, 11 pages (2009), doi:10.1155/2009/713183
Oja, E., Yuan, Z.: The FastICA algorithm revisited: convergence analysis. IEEE Transactions on Neural Networks 17(6), 1370–1381 (2006)
Scholz, M., Gatzek, S., Sterling, A., Fiehn, O., Selbig, J.: Metabolite fingerprinting: detecting biological features by independent component analysis. Bioinformatics 20(15), 2447–2454 (2004)
Zuo, X.N., Kelly, C., Adelstein, J.S., Klein, D.F., Castellanos, F.X., Milham, M.P.: Reliable intrinsic connectivity networks: test-retest evaluation using ICA and dual regression approach. Neuroimage 49(3), 2163–2177 (2010)
Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(12), 1615–1618 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lu, H. (2013). Learning Modewise Independent Components from Tensor Data Using Multilinear Mixing Model. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013. Lecture Notes in Computer Science(), vol 8189. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40991-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-40991-2_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40990-5
Online ISBN: 978-3-642-40991-2
eBook Packages: Computer ScienceComputer Science (R0)