Abstract
In this paper, we propose a model for natural images to learn topographic representations and complex cell properties. Different from the estimation of traditional models, e.g., pooling the outputs of filters in neighboring regions, our method maximizes a simple form of binary relations between two adjacent complex cells—“pairwise cumulant”, which contains the favorable nonlinearity as high order cumulant, and can exploit the “sparseness” and “correlation” of cells in primary visual cortex. By means of choosing nonlinearity properly, our model is related to cumulant-based ICA model, and the derived fixed-point algorithm is close to the well-known FastICA algorithm. The local convergence analysis proves that our fixed-point algorithm is cubic convergence and experiments on nature images show its high efficiency than traditional algorithms. Besides, simulations demonstrate the effectiveness of our model in capturing nonlinear dependencies among these neighboring complex cells. The learnt filters preserve properties of complex cells, and their orientation, spatial frequency and location change smoothly over the topographic map. In addition, these learnt filters can be used as feature descriptors. They produce features that are invariant to object transformations, and achieve better results than traditional models on digit recognition tasks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hubel D, Wiesel T (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol 195: 215–243
Hyvärinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3): 626–634
Olshausen BA (2003) Principles of image representation in visual cortex. In: Chalupa LM, Werner JS (eds) The visual neurosciences. MIT Press, Cambridge
Lee H, Ekanadham C, Ng AY (2008) Sparse deep belief net model for visual area v2. In: Advances in neural information processing systems, vol 20. MIT Press, Cambridge
Alonso JM, Martinez LM (1998) Functional connectivity between simple cells and complex cells in cat striate cortex. Nat Neurosci 1(5): 395–403
Hyvärinen A, Köster U (2007) Complex cell pooling and the statistics of natural images. Comput Neural Syst 18(2): 81–100
Hyvärinen A, Hoyer PO (2002) A two-layer sparse coding model learn simple and complex cell receptive fields and topography from natural images. Vis Res 41(18): 2413–2423
Zoran D, Weiss Y (2009) The ‘tree-dependent components’ of natural scenes are edge filters. In: Advances in neural information processing systems. MIT Press, Cambridge
Osindero S, Welling M, Hinton GE (2006) Topographic product models applied to natural scene statistics. Neural Comput 18: 381–414
Kavukcuoglu K, Ranzato M, Fergus R, LeCun Y (2009) Learning invariant features through topographic filter maps. In: Proceedings of computer vision and pattern recognition conference, Miami, Florida, USA, June 20–26
Karklin Y, Lewicki MS (2005) A hierarchical Bayesian model for learning non-linear statistical regularities in non-stationary natural signals. Neural Comput 17(2): 397–423
Wiskott L, Sejnowski T (2002) Slow feature analysis: unsupervised learning of invariances. Neural Comput 14(4): 715–770
Oja E, Yuan Z (2006) The FastICA algorithm revisited: convergence analysis. IEEE Trans Neural Netw 17(6): 1370–1381
Giannakopoulos X, Karhunen J, Oja E (1999) Experimental comparison of neural algorithms for independent component analysis and blind separation. Int J Neural Syst 9(2): 651–656
Hyvärinen A, Köster U (2006) FastISA: a fast fixedpoint algorithm for independent subspace analysis. In: Proceedings of the 14th European symposium on artificial neural networks, Bruges, Belgium
Serre T, Wolf L, Poggio T (2005) Object recognition with features inspired by visual cortex. In: Proceedings of computer vision and pattern recognition, San Diego, California, USA, June 20–26
Ranzato M, Boureau Y, LeCun Y (2007) Sparse feature learning for deep belief networks. In: Advances in neural information processing systems. MIT Press, Cambridge
Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: Proceedings of the international conference on computer vision, Kyoto, Japan, Sept. 27–Oct. 4
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324
Pinto N, Cox D, DiCarlo J (2008) Why is real-world visual object recognition hard. PLoS Comput Biol 4(1):151–156
Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2): 91–110
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of computer vision and pattern recognition, San Diego, California, USA, June 20–26
Inki M (2004) Natural image patch statistics conditioned on activity of an independent component. Comput Inf Sci A79: 28p
Hurri J, Hyvärinen A (2003) Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Comput 15: 663–691
Köster U, Hyvärinen A (2007) A two-layer ICA-like model estimated by score matching. In: Proceedings of the international conference on artificial neural networks, pp 798–807, Porto, Portugal, Sept. 9–13
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70: 489–501
Huang GB, Ding XJ, Zhou HM (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3): 155–163
Luenberger D (1969) Optimization by vector space methods. Wiley, New York
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Z., Huang, Y., Luo, X. et al. Learning Topographic Representations of Nature Images with Pairwise Cumulant. Neural Process Lett 34, 155–175 (2011). https://doi.org/10.1007/s11063-011-9189-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-011-9189-6