SpringerLink
Log in

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition

  • Original Article
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://sites.google.com/uci.edu/neurovision2022

References

  • Ales JM, Appelbaum LG, Cottereau BR et al (2013) The time course of shape discrimination in the human brain. Neuroimage 67:77–88

    Article  PubMed  Google Scholar 

  • Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Netw 48:109–24

    Article  PubMed  Google Scholar 

  • Beyeler M, Dutt N, Krichmar JL (2016) 3D visual response properties of MSTd emerge from an efficient, sparse population code. J Neurosci 36(32):8399–8415

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Beyeler M, Rounds E, Carlson K et al (2019) Neural correlates of sparse coding and dimensionality reduction. PLoS Comput Biol 15(6):e1006908

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bi GQ, Poo MM (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18(24):10,464-10,472

  • Bing Z, Baumann I, Jiang Z et al (2019) Supervised learning in snn via reward-modulated spike-timing-dependent plasticity for a target reaching vehicle. Front Neurorobot 13:18

    Article  PubMed  PubMed Central  Google Scholar 

  • Brzosko Z, Mierau SB, Paulsen O (2019) Neuromodulation of spike-timing-dependent plasticity: past, present, and future. Neuron 103(4):563–581

    Article  CAS  PubMed  Google Scholar 

  • Campbell, Fergus W. The transmission of spatial information through the visual system. From Theoretical Physics to Biology. Karger Publishers, 1973. 374–384

  • Caporale N, Dan Y et al (2008) Spike timing-dependent plasticity: a hebbian learning rule. Annu Rev Neurosci 31(1):25–46

    Article  CAS  PubMed  Google Scholar 

  • Chang L, Tsao DY (2017) The code for facial identity in the primate brain. Cell 169(6):1013–1028

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Chauhan T, Masquelier T, Montlibert A et al (2018) Emergence of binocular disparity selectivity through Hebbian learning. J Neurosci 38(44):9563–9578

  • Chauhan T, Masquelier T, Cottereau BR (2021) Sub-optimality of the early visual system explained through biologically plausible plasticity. Front Neurosci 15:727448

    Article  PubMed  PubMed Central  Google Scholar 

  • Cichy RM, Pantazis D, Oliva A (2016) Similarity-based fusion of meg and fmri reveals spatio-temporal dynamics in human cortex during visual object recognition. Cereb Cortex 26(8):3563–3579

    Article  PubMed  PubMed Central  Google Scholar 

  • De Valois RL, Albrecht DG, Thorell LG (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vis Res 22(5):545–559

    Article  PubMed  Google Scholar 

  • De Valois RL, Albrecht DG, Thorell LG (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vis Res 22(5):545–559

    Article  PubMed  Google Scholar 

  • Delorme A, Thorpe SJ (2001) Face identification using one spike per neuron: resistance to image degradations. Neural Netw 14(6–7):795–803

    Article  CAS  PubMed  Google Scholar 

  • Derrington A, Lennie P (1982) The influence of temporal frequency and adaptation level on receptive field organization of retinal ganglion cells in cat. J Physiol 333(1):343–366

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Derrington A, Lennie P, Wright M (1979) The mechanism of peripherally evoked responses in retinal ganglion cells. J Physiol 289(1):299–310

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • DiCarlo J, Zoccolan D, Rust N (2012) How does the brain solve visual object recognition? Neuron 73(3):415–434

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Diehl PU, Cook M (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci. https://doi.org/10.3389/fncom.2015.00099

    Article  PubMed  PubMed Central  Google Scholar 

  • Enroth-Cugell C, Robson JG (1966) The contrast sensitivity of retinal ganglion cells of the cat. J Physiol 187(3):517–552

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Falez P, Tirilly P, Bilasco IM, et al (2019) Multi-layered spiking neural network with target timestamp threshold adaptation and stdp. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

  • Feldman DE (2012) The spike-timing dependence of plasticity. Neuron 75(4):556–571

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. Josa A 4(12):2379–2394

    Article  CAS  Google Scholar 

  • Fu Q, Dong H (2021) An ensemble unsupervised spiking neural network for objective recognition. Neurocomputing 419:47–58

    Article  Google Scholar 

  • Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Ginsburg AP (1986) Spatial filtering and visual form perception. Handbook of Perception and Human Performance, Vol 2 Cognitive Processes and Performance

  • Goel A, Tung C, Lu YH, et al (2020) A survey of methods for low-power deep learning and computer vision. In: 2020 IEEE 6th world forum on internet of things (WF-IoT). IEEE, pp 1–6

  • Gütig R, Aharonov R, Rotter S et al (2003) Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J Neurosci 23(9):3697–3714. https://doi.org/10.1523/JNEUROSCI.23-09-03697.2003

    Article  PubMed  PubMed Central  Google Scholar 

  • Gütig R, Aharonov R, Rotter S et al (2003) Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J Neurosci 23(9):3697–3714

    Article  PubMed  PubMed Central  Google Scholar 

  • Hao Y, Huang X, Dong M et al (2020) A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Netw 121:387–395

    Article  PubMed  Google Scholar 

  • He K, Zhang X, Ren S, et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  • Henriksson L, Nurminen L, Hyvärinen A et al (2008) Spatial frequency tuning in human retinotopic visual areas. J Vis 8(10):5–5

    Article  Google Scholar 

  • Hughes HC, Nozawa G, Kitterle F (1996) Global precedence, spatial frequency channels, and the statistics of natural images. J Cognit Neurosci 8(3):197–230

    Article  CAS  Google Scholar 

  • Jiang P, Ergu D, Liu F et al (2022) A review of yolo algorithm developments. Procedia Comput Sci 199:1066–1073

    Article  Google Scholar 

  • Kauffmann L, Ramanoël S, Peyrin C (2014) The neural bases of spatial frequency processing during scene perception. Front Integr Neurosci 8:37

    Article  PubMed  PubMed Central  Google Scholar 

  • Kheradpisheh SR, Ganjtabesh M, Thorpe SJ et al (2018) Stdp-based spiking deep convolutional neural networks for object recognition. Neural Netw 99:56–67

    Article  PubMed  Google Scholar 

  • Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Tech. Rep. 0, University of Toronto, Toronto, Ontario

  • LeCun Y (1998) The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/

  • Liu D, Yue S (2016) Visual pattern recognition using unsupervised spike timing dependent plasticity learning. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 285–292

  • Liu Q, Pan G, Ruan H et al (2020) Unsupervised aer object recognition based on multiscale spatio-temporal features and spiking neurons. IEEE Trans Neural Netw Learn Syst 31(12):5300–5311

    Article  PubMed  Google Scholar 

  • Maass W (2000) On the computational power of winner-take-all. Neural Comput 12(11):2519–2535

    Article  CAS  PubMed  Google Scholar 

  • Majaj NJ, Hong H, Solomon EA et al (2015) Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J Neurosci 35(39):13,402-13,418

    Article  CAS  Google Scholar 

  • Masquelier T, Thorpe S (2007) Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput Biol 3(2):e31

    Article  PubMed  PubMed Central  Google Scholar 

  • Mozafari M, Ganjtabesh M, Nowzari-Dalini A et al (2019) Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks. Pattern Recognit 94:87–95

    Article  Google Scholar 

  • Nassi JJ, Callaway EM (2009) Parallel processing strategies of the primate visual system. Nat Rev Neurosci 10(5):360–372

  • Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res 37(23):3311–3325. https://doi.org/10.1016/S0042-6989(97)00169-7

    Article  CAS  PubMed  Google Scholar 

  • Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 1994 IEEE workshop on applications of computer vision. IEEE, pp 138–142

  • Sanchez-Garcia M, Chauhan T, Cottereau BR, et al (2022) Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition. ar**v:2212.00081

  • Shapley R, Lennie P et al (1985) Spatial frequency analysis in the visual system. Annu Rev Neurosci 8(1):547–581

    Article  CAS  PubMed  Google Scholar 

  • Solomon SG, White AJ, Martin PR (2002) Extraclassical receptive field properties of parvocellular, magnocellular, and koniocellular cells in the primate lateral geniculate nucleus. J Neurosci 22(1):338–349

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Stivaktakis R, Tsagkatakis G, Tsakalides P (2019) Deep learning for multilabel land cover scene categorization using data augmentation. IEEE Geosci Remote Sens Lett 16(7):1031–1035

    Article  Google Scholar 

  • Stuijt J, Sifalakis M, Yousefzadeh A et al (2021) \(\mu \)brain: an event-driven and fully synthesizable architecture for spiking neural networks. Front Neurosci 15:538

    Article  Google Scholar 

  • Sun Y, Liang D, Wang X, et al (2015) Deepid3: face recognition with very deep neural networks. ar**v:1502.00873

  • Tolhurst DJ, Tadmor Y, Chao T (1992) Amplitude spectra of natural images. Ophthalmic Physiol Opt 12(2):229–232

    Article  CAS  PubMed  Google Scholar 

  • Vigneron A, Martinet J (2020) A critical survey of stdp in spiking neural networks for pattern recognition. In: 2020 international joint conference on neural networks (IJCNN). IEEE, pp 1–9

  • Vinje WE, Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287(5456):1273–1276

    Article  CAS  PubMed  Google Scholar 

  • **ao H, Rasul K, Vollgraf R (2017) Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. ar**v:1708.07747

  • Yu Q, Tang H, Tan KC et al (2013) Rapid feedforward computation by temporal encoding and learning with spiking neurons. IEEE Trans Neural Netw Learn Syst 24(10):1539–1552

    Article  PubMed  Google Scholar 

  • Zhou Q, Li X (2022) A bio-inspired hierarchical spiking neural network with reward-modulated stdp learning rule for aer object recognition. IEEE Sens J 22(16):16,323-16,338

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by a UCSB Academic Senate Faculty Research Grant to MB and by FLAG-ERA project JTC-2019 DOMINO to BRC. TC was partially supported by the grants DE-SC0022997 (US Department of Energy) and FRM:SPF20170938752 (Fondation pour la Recherche Médical, France), and a Picower Fellowship from The JPB Foundation.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of California, Santa Barbara, CA, USA

    Melani Sanchez-Garcia & Michael Beyeler

  2. The Picower Institute for Learning and Memory, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Boston, MA, USA

    Tushar Chauhan

  3. CerCo CNRS UMR5549, Université de Toulouse III-Paul Sabatier, Toulouse, France

    Tushar Chauhan & Benoit R. Cottereau

  4. IPAL, CNRS IRL 2955, Singapore, Singapore

    Benoit R. Cottereau

  5. Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA

    Michael Beyeler

Authors
  1. Melani Sanchez-Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tushar Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benoit R. Cottereau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Beyeler
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TC and BRC conceived and designed the original study, which was subsequently extended by MSG and MB. TC wrote all the code and MSG ran all the simulations. MSG and MB analyzed and interpreted the results. MSG drafted the manuscript. All authors reviewed and approved the final version of the manuscript.

Corresponding author

Correspondence to Melani Sanchez-Garcia.

Additional information

Communicated by Benjamin Lindner.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Melani Sanchez-Garcia and Tushar Chauhan are co-first authors. Benoit R. Cottereau and Michael Beyeler are co-last authors.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanchez-Garcia, M., Chauhan, T., Cottereau, B.R. et al. Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition. Biol Cybern 117, 95–111 (2023). https://doi.org/10.1007/s00422-023-00956-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00422-023-00956-x

Keywords

Search

Navigation