SpringerLink
Log in

A novel capsule network based on deep routing and residual learning

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Capsule network is a novel network structure based on vector, which has excellent performance in image classification and attracted the attention of researchers. However, the capsule network has a large amount of computation and does not perform well in the task of classifying complex datasets. Therefore, we propose a capsule network based on deep routing and residual learning. The network reduces the number of capsules to reduce the computation during routing, adds local routing layers to extract deep hidden capsule information and constructs residual connections to assist parameters training. In addition, we propose a new capsule dropout method to enhance the robustness of the network and reduce the computation of the network. We conducted several experiments on multiple benchmark datasets (CIFAR-10, SVHN, Fashion-MNIST and MNIST). The experimental results show that the proposed method achieves competitive results on multiple benchmark datasets. For example, the accuracy of 81.61% is achieved on the CIFAR-10 dataset, and the number of training parameters is reduced by 72.09%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  • Afshar P, Naderkhani F, Oikonomou A et al (2021) Mixcaps: a capsule network-based mixture of experts for lung nodule malignancy prediction. Pattern Recognit 116:107942

    Article  Google Scholar 

  • Amer M, Maul T (2020) Path capsule networks. Neural Process Lett 52:545–559

    Article  Google Scholar 

  • Cheng X, He J, He J et al (2019) Cv-CapsNet: complex-valued capsule network. IEEE Access 7:85492–85499

    Article  Google Scholar 

  • Choi J, Seo H, Im S (2019) Attention routing between capsules. In: proceedings of the IEEE international conference on computer vision workshops, Seoul, pp 1981–1989

  • Deliege A, Cioppa A, Droogenbroeck MV (2018) Hitnet: a neural network with capsules embedded in a hit-or-miss layer, extended with hybrid data augmentation and ghost capsules. ar**v preprint ar**v:1806.06519

  • Dhanaraj RK, Lalitha K, Anitha S et al (2021a) Hybrid and dynamic clustering based data aggregation and routing for wireless sensor networks. J Intell Fuzzy Syst 40(6):10751–10765

    Article  Google Scholar 

  • Dhanaraj RK, Lalitha K, Oana G et al (2021b) Black hole and sink hole attack detection in wireless body area networks. CMC-Comput Mater Con 68(2):1949–1965

    Google Scholar 

  • Dhanaraj RK, Ramakrishnan V, Poongodi M et al (2021c) Random forest bagging and X-means clustered antipattern detection from SQL query log for accessing secure mobile data. Wirel Commun Mob Comput 2021:2730246

    Article  Google Scholar 

  • Duarte K, Rawat Y S, Shah M (2018) VideoCapsuleNet: a simplified network for action detection. In: Proceedings of the 32nd international conference on neural information processing systems, Montréal, pp 7621–7630

  • Guo X (2017) A Keras implementation of CapsNet in NIPS2017 paper "Dynamic routing between capsules". https://github.com/**fengGuo/CapsNet-Keras

  • He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, pp 770–778

  • Hinton GE, Krizhevsky A, Wang S D (2011) Transforming auto-encoders. In: International conference on artificial neural networks, Berlin, Heidelberg, pp 44–51

  • Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with EM routing. In: International conference on learning representations, Vancouver, pp 1–15

  • Hua Q, Wei L, Dong C et al (2020) Improved variational inference with dynamic routing flow. Int J Mach Learn Cyb 11(2):301–312

    Article  Google Scholar 

  • Kosiorek AR, Sabour S, Teh YW et al (2019) Stacked capsule autoencoders. Adv Neural Inf Process Syst 32:1–11

    Google Scholar 

  • Krishnamoorthi S, Jayapaul P, Dhanaraj RK et al (2021) Design of pseudo-random number generator from turbulence padded chaotic map. Nonlinear Dyn 104(2):1627–1643

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  • LaLonde R, Xu Z, Irmakci I et al (2021) Capsules for biomedical image segmentation. Med Image Anal 68:101889

    Article  Google Scholar 

  • Mazzia V, Salvetti F, Chiaberge M (2021) Efficient-CapsNet: capsule network with self-attention routing. ar**v preprint ar**v:2101.12491

  • Nair P, Doshi R, Keselj S (2021) Pushing the limits of capsule networks. ar**v preprint ar**v:2103.08074

  • Phaye SSR, Sikka A, Dhall A, Bathula DR (2018) Multi-level dense capsule networks. In: Asian conference on computer vision, Perth, pp 577–592

  • Rajasegaran J, Jayasundara V, Jayasekara S et al (2019) Deepcaps: going deeper with capsule networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, pp 10725–10733

  • Ren Q, Shang S, He L (2019) Adaptive routing between capsules. ar**v preprint ar**v:1911.08119

  • Rosario VMD, Borin E, Breternitz M (2019) The multi-lane capsule network. IEEE Signal Process Lett 26(7):1006–1010

    Article  Google Scholar 

  • Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Proceedings of the 31st international conference on neural information processing systems, California, pp 3859–3869

  • Saravanakumar C, Senthilvel P, Thirupurasundari D et al (2021) Plant syndrome recognition by Gigapixel Image using convolutional neural network. In: Proceedings of the first international conference on advanced scientific innovation in science, Engineering and Technology (ICASISET), Chennai

  • Sathyamoorthy M, Kuppusamy S, Dhanaraj RK et al (2022) Improved K-means based Q learning algorithm for optimal clustering and node balancing in WSN. Wireless Pers Commun 122:2745–2766

    Article  Google Scholar 

  • Sun G, Ding S, Sun T et al (2021a) SA-CapsGAN: using capsule networks with embedded self-attention for generative adversarial network. Neurocomputing 423:399–406

    Article  Google Scholar 

  • Sun K, Xu H, Yuan L et al (2021b) A tiny deep capsule network. Int J Mach Learn Cyb 11:301–312

    Google Scholar 

  • Sun G, Ding S, Sun T et al (2022) A novel dense capsule network based on dense capsule layers. Appl Intell 52(3):3066–3076

    Article  Google Scholar 

  • Vinothsaravanan R, Palanisamy C, Rajesh KD et al (2021) Bootstrap aggregative mean shift clustering for big data anti-pattern detection analytics in 5G/6G communication networks. Comput Electr Eng 95:107380

    Article  Google Scholar 

  • Wang Y, **ao W, Tan Z et al (2021) Caps-OWKG: a capsule network model for open-world knowledge graph. Int J Mach Learn Cyb 12(6):1627–1637

    Article  Google Scholar 

  • Wu Y, Li J, Wu J et al (2020) Siamese capsule networks with global and local features for text classification. Neurocomputing 390:88–98

    Article  Google Scholar 

  • ** E, Bing S, ** Y (2017) Capsule network performance on complex data. ar**v preprint ar**v:1712.03480

  • **ang C, Zhang L, Tang Y et al (2018) MS-CapsNet: a novel multi-scale capsule network. IEEE Signal Process Lett 25(12):1850–1854

    Article  Google Scholar 

  • **ong Y, Su G, Ye S et al (2019) Deeper capsule network for complex data. In: International joint conference on neural networks, Rio, pp 1–8

  • Zhao Z, Kleinhans A, Sandhu G, et al (2019) Capsule networks with max-min normalization. ar**v preprint ar**v:1903.09662

Download references

Acknowledgements

This work is supported by the National Natural Science Foundations of China (no.61976216, no.62276265 and no.62206297).

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Jian Zhang, Qinghai Xu, Lili Guo & Shifei Ding

  2. College of Intelligence and Computing, Tian** University, Tian**, 300350, China

    Ling Ding

  3. Mine Digitization Engineering Research Center of Ministry of Education of the People’s Republic of China, Xuzhou, 221116, China

    Shifei Ding

Authors
  1. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinghai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lili Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ling Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the writing of the manuscript and final approval.

Corresponding author

Correspondence to Shifei Ding.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.

Human and animal rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Xu, Q., Guo, L. et al. A novel capsule network based on deep routing and residual learning. Soft Comput 27, 7895–7906 (2023). https://doi.org/10.1007/s00500-023-08018-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-08018-x

Keywords

Search

Navigation