Abstract
In this article we present the supervised iterative projections and rotations (s-ipr) algorithm, a method for learning discriminative incoherent subspaces from data. We derive s-ipr as a supervised extension of our previously proposed iterative projections and rotations (ipr) algorithm for incoherent dictionary learning, and we employ it to learn incoherent sub-spaces that model signals belonging to different classes. We test our method as a feature transform for supervised classification, first by visualising transformed features from a synthetic dataset and from the ‘iris’ dataset, then by using the resulting features in a classification experiment.
Similar content being viewed by others
Notes
Here e is a vector of ones.
Note that the term cluster implies that a this stage the algorithm needs to make an unsupervised decision, since there is no any a-priori reason to assign a given atom to any particular class.
The Matlab code used to generate the results in this Section is available from https://github.com/danieleb/2014-SJSPS
References
Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3), 337–404.
Bair, E., Paul, D., Tibshirani, R. (2006). Prediction by supervised principal components. Journal of the American Statistical Association, 101, 119–137.
Barchiesi, D., & Plumbley, M.D. (2013). Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Transactions on Signal Processing, 61(8), 2055–2065.
Barshan, E., Ghodsi, A., Azimifar, Z., Jahromi, M.Z. (2011). Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds. Pattern Recognition, 44(7), 1357–1371.
Duda, R., & Hart, P.E. (1973). Pattern classification and scene analysis. New York: Wiley.
Elad, M. (2010). Sparse and redundant representations. New York: Springer.
Elhamifar, E., & Vidal, R. (2013). Sparse subspace clustering: algorithm, theory, and applications. To appear in IEEE transactions on pattern analysis and machine intelligence.
Gretton, A., Bousquet, O., Smola, A., Schölkopf, B. (2005). Measuring statistical dependence with hilbert-schmidt norms. Algorithmic learning theory (pp. 63–77). New York: Springer.
Li, K.-C. (1991). Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414), 316–327.
Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, Sixth Series, 2, 559–572.
Rubinstein, R., Bruckstein, A., Elad, M. (2010). Dictionaries for sparse representation modeling. In Proceedings of the IEEE, 98(6), 1045–1057.
Schnass, K., & Vandergheynst, P. (2010). A union of incoherent spaces model for classification. In Proceedings of the IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 5490–5493).
Van Der Maaten, L., Postma, E., Van Den Herik, J. (2009). Dimensionality reduction: A comparative review. Tech. Rep., TiCC, Tilburg University.
**ng, E.P., Jordan, M.I., Russell, S., Ng, A. (2002). Distance metric learning with application to clustering with side-information. In Advances in neural information processing systems (pp. 505–512).
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the Platform Grant EP/K009559/1 and the Leadership Fellowship EP/G007144/1, both from the UK Engineering and Physical Sciences Research Council (EPSRC).
Rights and permissions
About this article
Cite this article
Barchiesi, D., Plumbley, M.D. Learning Incoherent Subspaces: Classification via Incoherent Dictionary Learning. J Sign Process Syst 79, 189–199 (2015). https://doi.org/10.1007/s11265-014-0937-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-014-0937-5