Abstract
This paper introduces a spiking neural network able to learn multiple tasks using their unique characteristic, namely, that their behavior can be changed based on the modulation of the firing threshold of spiking neurons. We designed and tested a threshold-modulated spiking neural network (TM-SNN) to solve multiple classification tasks using the approach of learning only one task at a time. The task to be performed is determined by a firing threshold: with one threshold the network learns one task, with the second threshold another task, etc. TM-SNN was implemented on Intel’s Loihi2 neuromorphic computer and tested on neuromorphic NMNIST data. The results show that TM-SNN can actually learn different tasks through modifying its dynamics via modulation of the neurons’ firing threshold. It is the first application of spiking neural networks to multi-task classification problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The lava and lava-dl library are available at https://lava-nc.org.
- 2.
References
Bilen, H., Vedaldi, A.: Universal representations: the missing link between faces, text, planktons, and cat breeds. CoRR abs/1701.07275 (2017). http://arxiv.org/abs/1701.07275
Cios, K.J., Shin, I.: Image recognition neural network: IRNN. Neurocomputing 7(2), 159–185 (1995). https://doi.org/10.1016/0925-2312(93)E0062-I
Crawshaw, M.: Multi-task learning with deep neural networks: a survey. CoRR abs/2009.09796 (2020). https://arxiv.org/abs/2009.09796
Davies, M., et al.: Advancing neuromorphic computing with Loihi: a survey of results and outlook. Proc. IEEE 109(5), 911–934 (2021). https://doi.org/10.1109/JPROC.2021.3067593
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML 2015, vol. 37, pp. 1180–1189. JMLR.org (2015)
Gerstner, W., Kistler, W.M., Naud, R., Paninski, L.: Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, New York (2014)
Kaiser, J., Mostafa, H., Neftci, E.: Synaptic plasticity dynamics for deep continuous local learning (DECOLLE). Front. Neurosci. 14 (2020). https://doi.org/10.3389/fnins.2020.00424
Liu, P., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. CoRR abs/1704.05742 (2017). http://arxiv.org/abs/1704.05742
Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997). https://doi.org/10.1016/S0893-6080(97)00011-7. https://www.sciencedirect.com/science/article/pii/S0893608097000117
Maninis, K., Radosavovic, I., Kokkinos, I.: Attentive single-tasking of multiple tasks. CoRR abs/1904.08918 (2019). http://arxiv.org/abs/1904.08918
Marder, E.: Neuromodulation of neuronal circuits: back to the future. Neuron 76(1), 1–11 (2012). https://doi.org/10.1016/j.neuron.2012.09.010
Orchard, G., et al.: Efficient neuromorphic signal processing with Loihi 2. CoRR abs/2111.03746 (2021). https://arxiv.org/abs/2111.03746
Orchard, G., Jayawant, A., Cohen, G.K., Thakor, N.: Converting static image datasets to spiking neuromorphic datasets using saccades. Front. Neurosci. 9 (2015). https://doi.org/10.3389/fnins.2015.00437
Rebuffi, S., Bilen, H., Vedaldi, A.: Efficient parametrization of multi-domain deep neural networks. CoRR abs/1803.10082 (2018). http://arxiv.org/abs/1803.10082
Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: NIPS 2005 Workshop on Transfer Learning, vol. 898 (2005)
Ruder, S.: An overview of multi-task learning in deep neural networks (2017). https://doi.org/10.48550/ARXIV.1706.05098. https://arxiv.org/abs/1706.05098
Schröder, F., Biemann, C.: Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2971–2985. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.268. https://aclanthology.org/2020.acl-main.268
Shin, J., et al.: Recognition of partially occluded and rotated images with a network of spiking neurons. IEEE Trans. Neural Netw. 21(11), 1697–1709 (2010). https://doi.org/10.1109/TNN.2010.2050600
Shrestha, S.B., Orchard, G.: SLAYER: spike layer error reassignment in time. CoRR abs/1810.08646 (2018). http://arxiv.org/abs/1810.08646
Standley, T., Zamir, A.R., Chen, D., Guibas, L., Malik, J., Savarese, S.: Which tasks should be learned together in multi-task learning? (2019). https://doi.org/10.48550/ARXIV.1905.07553. https://arxiv.org/abs/1905.07553
Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., Masquelier, T., Maida, A.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019). https://doi.org/10.1016/j.neunet.2018.12.002. https://www.sciencedirect.com/science/article/pii/S0893608018303332
Wang, Z., Dai, Z., Póczos, B., Carbonell, J.: Characterizing and avoiding negative transfer (2018). https://doi.org/10.48550/ARXIV.1811.09751. https://arxiv.org/abs/1811.09751
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cachi, P.G., Soto, S.V., Cios, K.J. (2023). TM-SNN: Threshold Modulated Spiking Neural Network for Multi-task Learning. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14135. Springer, Cham. https://doi.org/10.1007/978-3-031-43078-7_53
Download citation
DOI: https://doi.org/10.1007/978-3-031-43078-7_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43077-0
Online ISBN: 978-3-031-43078-7
eBook Packages: Computer ScienceComputer Science (R0)