SpringerLink
Log in

I2DKPCN: an unsupervised deep learning network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we proposed an incremental two-dimensional kernel PCA-based convolutional network (I2DKPCN) which is a novel unsupervised deep learning network. In our architecture, I2DKPCN consists of several feature extraction stages and one output stage, and each feature extraction stage includes a convolutional layer, a feature pooling layer and a feature fusion layer. In the output stage, binary hashing and blockwise histograms are exploited for the generation of the final features. In particular, the filters of the convolutional layer are learned by using incremental two-dimensional kernel principal component analysis(I2DKPCA) rather than the gradient-based optimization. Due to the fact that the back propagation is not used to learn the parameters of the filter, the calculation speed of I2DKPCN is much faster than that of the existing deep network. We have extensively tested I2DKPCN on multiple data sets for three challenging tasks, including hand-written digit recognition, texture classification and face recognition. It can be seen from the results that I2DKPCN performs competitively or even better compared with other networks in all tests. Because the filters are learned by I2DKPCA, which combined the 2DPCA with kernel method and incremental learning, the non-linear problem was solved and the computational time was reduced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Fei L, Lu G, Jia W, Teng S, Zhang D (2018) Feature extraction methods for palmprint recognition: A survey and evaluation. IEEE Transactions on Systems, Man, and Cybernetics: Systems 49(2):346–363

    Article  Google Scholar 

  2. Fei L, Zhang B, Jia W, Wen J, Zhang D (2020) Feature extraction for 3-d palmprint recognition: A survey. IEEE Trans Instrum Meas 69(3):645–656

    Article  Google Scholar 

  3. Ma W, Wu Y, Cen F, Wang G (2020) Mdfn: Multi-scale deep feature learning network for object detection. Pattern Recogn 100:107149

    Article  Google Scholar 

  4. Galzán E, Mooney P (2021) Neuroevolution in deep neural networks: Current trends and future challenges. IEEE Transactions on Artificial Intelligence

  5. Zhou L, Min W, Lin D, Han Q, Liu R (2020) Detecting motion blurred vehicle logo in iov using filter-deblurgan and vl-yolo. IEEE Trans Veh Technol 69(4):3604–3614

    Article  Google Scholar 

  6. Arunachalam A, Ravi V, Acharya V, Pham TD (2021) Toward data-model-agnostic autonomous machine-generated data labeling and annotation platform: Covid-19 autoannotation use case. IEEE Trans Eng Manag

  7. Long B, Zhu Z, Yang W, Chong KT, Rodriguez J, Guerrero JM (2021) Gradient descent optimization based parameter identification for fcs-mpc control of lcl-type grid connected converter. IEEE Trans Ind Electron

  8. deVos BD, Berendsen FF, Viergever MA, Sokooti H, Staring M, Išgum I (2019) A deep learning framework for unsupervised affine and deformable image registration. Medical image analysis 52:128–143

    Article  Google Scholar 

  9. Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 132–149

  10. Bi X, Li S, **ao B, Li Y, Wang G, Ma X (2020) Computer aided alzheimer’s disease diagnosis by an unsupervised deep learning technology. Neurocomputing 392:296–304

    Article  Google Scholar 

  11. Wu Z, **ong Y, Yu SX, Lin D (2018) Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3733–3742

  12. Yu Y, Chan K HR, You C, Song C, Ma Y (2020) Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in Neural Information Processing Systems, 33

  13. Mei S, Yang H, Yin Z (2018) An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Trans Instrum Meas 67(6):1266–1277

    Article  Google Scholar 

  14. Hussein S, Kandel P, Bolan CW, Wallace MB, Bagci U (2019) Lung and pancreatic tumor characterization in the deep learning era: novel supervised and unsupervised learning approaches. IEEE transactions on medical imaging 38(8):1777–1787

    Article  Google Scholar 

  15. Hsu C-C, Lin C-W (2017) Cnn-based joint clustering and representation learning with feature drift compensation for large-scale image data. IEEE Transactions on Multimedia 20(2):421–429

    Article  Google Scholar 

  16. Huang J, Dong Q, Gong S, Zhu X (2019) Unsupervised deep learning by neighbourhood discovery. In: International Conference on Machine Learning, PMLR, pp 2849–2858

  17. **. IEEE Trans Knowl Data Eng 30(6):1065–1080

    Article  Google Scholar 

  18. Kim D, Cho D, Kweon IS (2019) Self-supervised video representation learning with space-time cubic puzzles. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 8545–8552

  19. Xu P, Song Z, Yin Q, Song Y-Z, Wang L (2020) Deep self-supervised representation learning for free-hand sketch. IEEE Transactions on Circuits and Systems for Video Technology 31(4):1503–1513

    Article  Google Scholar 

  20. Wang X, He K, Gupta A (2017) Transitive invariance for self-supervised visual representation learning. In: Proceedings of the IEEE international conference on computer vision, pp 1329–1338

  21. Ansari AF, Scarlett J, Soh H (2020) A characteristic function approach to deep implicit generative modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7478–7487

  22. Dornaika F (2021) Flexible data representation with feature convolution for semi-supervised learning. Appl Intell, pp 1–15

  23. Shen Y, Qin J, Chen J, Yu M, Liu L, Zhu F, Shen F, Shao L (2020) Auto-encoding twin-bottleneck hashing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2818–2827

  24. Tkach A, Tagliasacchi A, Remelli E, Pauly M, Fitzgibbon A (2017) Online generative model personalization for hand tracking. ACM Transactions on Graphics (ToG) 36(6):1–11

    Article  Google Scholar 

  25. Ye M, Ji C, Chen H, Lei L, Lu H, Qian Y (2020) Residual deep pca-based feature extraction for hyperspectral image classification. Neural Comput & Applic 32(18):14287–14300

    Article  Google Scholar 

  26. Zhang B, Qian J (2021) Autoencoder-based unsupervised clustering and hashing. Appl Intell 51(1):493–505

    Article  Google Scholar 

  27. Chan T-H, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) Pcanet: A simple deep learning baseline for image classification?. IEEE transactions on image processing 24(12):5017–5032

    Article  MathSciNet  Google Scholar 

  28. Yu H, Dai Q (2021) Dwe-il: a new incremental learning algorithm for non-stationary time series prediction via dynamically weighting ensemble learning. Appl Intell, pp 1–21

  29. Wu J, Qiu S, Kong Y, Jiang L, Chen Y, Yang W, Senhadji L, Shu H (2018) Pcanet: An energy perspective. Neurocomputing 313:271–287

    Article  Google Scholar 

  30. Gao Q, Ma L, Liu Y, Gao X, Nie F (2017) Angle 2dpca: A new formulation for 2dpca. IEEE transactions on cybernetics 48(5):1672–1678

    Article  Google Scholar 

  31. **ao X, Zhou Y (2018) Two-dimensional quaternion pca and sparse pca. IEEE transactions on neural networks and learning systems 30(7):2028–2042

    Article  Google Scholar 

  32. Sahoo TK, Banka H, Negi A (2020) Novel approaches to one-directional two-dimensional principal component analysis in hybrid pattern framework. Neural Comput & Applic 32(9):4897–4918

    Article  Google Scholar 

  33. Yu D, Wu X-J (2018) 2dpcanet: a deep leaning network for face recognition. Multimedia Tools and Applications 77(10):12919–12934

    Article  Google Scholar 

  34. Choi Y, Ozawa S, Lee M (2014) Incremental two-dimensional kernel principal component analysis. Neurocomputing 134:280–288

    Article  Google Scholar 

  35. Bhattacharya S, Maddikunta P KR, Kaluri R, Singh S, Gadekallu TR, Alazab M, Tariq U, et al. (2020) A novel pca-firefly based xgboost classification model for intrusion detection in networks using gpu. Electronics 9(2):219

    Article  Google Scholar 

  36. Yang X, Liu W, Tao D, Cheng J (2017) Canonical correlation analysis networks for two-view image recognition. Inf Sci 385:338–352

    Article  Google Scholar 

  37. Mi J-X, Zhu Q, Lu J (2019) Principal component analysis based on block-norm minimization. Appl Intell 49(6):2169–2177

    Article  Google Scholar 

  38. Jiang Q, Yan X (2018) Parallel pca–kpca for nonlinear process monitoring. Control Eng Pract 80:17–25

    Article  Google Scholar 

  39. Li X, Wu H, Yang X, Xue P, Tan S (2021) Multiview machine vision research of fruits boxes handling robot based on the improved 2d kernel principal component analysis network. Journal of Robotics, 2021

  40. Fujiwara T, Chou J-K, Shilpika S, Xu P, Ren L, Ma K-L (2019) An incremental dimensionality reduction method for visualizing streaming multidimensional data. IEEE transactions on visualization and computer graphics 26(1):418– 428

    Article  Google Scholar 

  41. Charte D, Charte F, García S, del Jesus MJ, Herrera F (2018) A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines. Information Fusion 44:78–96

    Article  Google Scholar 

  42. Wang R, Thakur CS, Cohen G, Hamilton TJ, Tapson J, Schaik AV (2017) Neuromorphic hardware architecture using the neural engineering framework for pattern recognition. IEEE Trans Biomed Circuits Syst, pp 574–584

  43. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  44. Ballester P, Araujo R (2016) On the performance of googlenet and alexnet applied to sketches. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 30

  45. Zeng R, Wu J, Shao Z, Chen Y, Chen B, Senhadji L, Shu H (2016) Color image classification via quaternion principal component analysis network. Neurocomputing 216:416–428

    Article  Google Scholar 

  46. Vishwakarma VP, Dalal S (2020) A novel non-linear modifier for adaptive illumination normalization for robust face recognition. Multimedia Tools and Applications 79(17):11503– 11529

    Article  Google Scholar 

  47. Deng W, Hu J, Guo J (2017) Face recognition via collaborative representation: Its discriminant nature and superposed representation. IEEE transactions on pattern analysis and machine intelligence 40 (10):2513–2521

    Article  Google Scholar 

  48. MacAllister A, Kohl A, Winer E (2020) Using high-fidelity meta-models to improve performance of small dataset trained bayesian networks. Expert Syst Appl 139:112830

    Article  Google Scholar 

  49. Sudars K (2017) Face recognition face2vec based on deep learning: Small database case. Autom Control Comput Sci 51(1):50–54

    Article  Google Scholar 

  50. Zhang Q, Zhang M, Chen T, Sun Z, Ma Y, Yu B (2019) Recent advances in convolutional neural network acceleration. Neurocomputing 323:37–51

    Article  Google Scholar 

  51. Shao Z, Wu W, Wang Z, Du W, Li C (2018) Seaships: A large-scale precisely annotated dataset for ship detection. IEEE transactions on multimedia 20(10):2593–2604

    Article  Google Scholar 

  52. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Shanghai Agriculture Applied Technology Development Program, China (2020-02-08-00-07-F01480).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Tongji University, Shanghai, 201804, China

    Ruyi Zhao & Fanhuai Shi

Authors
  1. Ruyi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fanhuai Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fanhuai Shi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by Shanghai Agriculture Applied Technology Development Program, China (2020-02-08-00-07-F01480).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, R., Shi, F. I2DKPCN: an unsupervised deep learning network. Appl Intell 52, 9938–9951 (2022). https://doi.org/10.1007/s10489-021-03007-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03007-9

Keywords

Search

Navigation