SpringerLink
Log in

Integrating self-organizing feature map with graph convolutional network for enhanced superpixel segmentation and feature extraction in non-Euclidean data structure

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep learning has been widely used on Euclidean data type, and the deep learning architecture has made a breakthrough by the development of technology. The common neural network architectures include Deep Neural Network (DNN), Convolutional Neural Network (CNN) and Long-short Term Memory (LSTM). The achievements of these models have above the standard. But in various fields not all data can be shown by Euclidean data type, so Graph Convolutional Network (GCN) was proposed to solve this problem. GCN is applied to non-Euclidian data structure and presents in the graph data type, which is composed of nodes and edges, such as chemical compound, a subset of the web. The graph data type can be able the relationship between nodes and nodes, making it not lose the important features. Therefore, our paper converts the image into graph data type to retain the complete feature information of image, which is different from CNN requiring multiple convolution layers of different dimensions to retain the features information of image. In the paper, we use the superpixel segmentation algorithm to convert the image to the graph data type. The problem of superpixel block disappearance is prone to occur in the previous superpixel algorithm, and the missing block must be used with zero-padding to correct the dimensional error. The purpose of this thesis is to propose the Self-Organizing Feature Map (SOM) for superpixel segmentation combined with graph convolutional network to solve the problem of incorrect feature extraction caused by superpixel segmentation algorithm. Most of the superpixel segmentation algorithm uses the RGB or CIELAB color space to segment the pixels in the image, which is unexplainable features. Therefore, in this paper combins with image processing to explain the feature meaning and proposed the explainable features with the graph data type.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Algorithm 1
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Masoum S, Malabat C, Jalali-Heravi M, Guillou C, Rezzi S, Rutledge D (2007) Application of support vector machines to 1h nmr data of fish oils: methodology for the confirmation of wild and farmed salmon and their origins. Anal Bioanal Chem 387:1499–510. https://doi.org/10.1007/s00216-006-1025-x

    Article  Google Scholar 

  2. Liao B-K, Goh AP, Lio CI, Hsiao H-I (2024) Kinetic models applied to quality change and shelf-life prediction of fresh-cut pineapple in food cold chain. Food Chem 437:137803. https://doi.org/10.1016/j.foodchem.2023.137803

    Article  Google Scholar 

  3. Semyalo D, Kwon O, Wakholi C, Min HJ, Cho B-K (2024) Nondestructive online measurement of pineapple maturity and soluble solids content using visible and near-infrared spectral analysis. Postharvest Biol Technol 209:112706. https://doi.org/10.1016/j.postharvbio.2023.112706

    Article  Google Scholar 

  4. Malik R (2003) Learning a classification model for segmentation. In: Proceedings ninth IEEE international conference on computer vision, pp 10–171. https://doi.org/10.1109/ICCV.2003.1238308

  5. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Int 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120

    Article  Google Scholar 

  6. Li Z, Chen J (2015) Superpixel segmentation using linear spectral clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1356–1363. https://doi.org/10.1109/CVPR.2015.7298741

  7. Achanta R, Süsstrunk S (2017) Superpixels and polygons using simple non-iterative clustering. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4895–4904. https://doi.org/10.1109/CVPR.2017.520

  8. Twogood RE, Sommer FG (1982) Digital image processing. IEEE Trans Nucl Sci 29(3):1075–1086. https://doi.org/10.1109/TNS.1982.4336327

    Article  Google Scholar 

  9. Douglas DH, Peucker TK (1973) Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: Int J Geographic Inform Geovisualization 10:112–122

    Article  Google Scholar 

  10. Le CV, Hong QN, Quang TT, Trung ND (2016) Superpixel-based background removal for accuracy salience person re-identification. In: 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp 1–4. https://doi.org/10.1109/ICCE-Asia.2016.7804806

  11. Giraud R, Ta V-T, Papadakis N (2017) Superpixel-based color transfer. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 700–704. https://doi.org/10.1109/ICIP.2017.8296371

  12. Almero VJD, Alejandrino JD, Bandala AA, Dadios EP (2020) Segmentation of aquaculture underwater scene images based on slic superpixels merging-fast marching method hybrid. In: 2020 IEEE REGION 10 CONFERENCE (TENCON), pp 432–437. https://doi.org/10.1109/TENCON50793.2020.9293806

  13. Andrew A (2000) Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science, by j.a. sethian. Robotica 18:89–92. https://doi.org/10.1017/S0263574799212404

    Article  Google Scholar 

  14. Forcadel N, Guyader C, Gout C (2008) Generalized fast marching method: applications to image segmentation. Numer Algo 48:189–211. https://doi.org/10.1007/s11075-008-9183-x

    Article  MathSciNet  Google Scholar 

  15. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480. https://doi.org/10.1109/5.58325

    Article  Google Scholar 

  16. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  18. Liu S, Deng W (2015) Very deep convolutional neural network based image classification using small training sample size. In: 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp 730–734. https://doi.org/10.1109/ACPR.2015.7486599

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  20. Nazir A, Wani MA (2023) You only look once - object detection models: a review. In: 2023 10th International conference on computing for sustainable global development (INDIACom), pp 1088–1095

  21. Zhao Z, Fang H, ** Z, Qiu Q (2020) Gisnet: graph-based information sharing network for vehicle trajectory prediction. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp 1–7. https://doi.org/10.1109/IJCNN48605.2020.9206770

  22. Zhao L, Song Y, Zhang C, Liu Y, Wang P, Lin T, Deng M, Li H (2020) T-gcn: a temporal graph convolutional network for traffic prediction. IEEE Trans Intell Transp Syst 21(9):3848–3858. https://doi.org/10.1109/TITS.2019.2935152

    Article  Google Scholar 

  23. Lo L, **e H-X, Shuai H-H, Cheng W-H (2020) Mer-gcn: micro-expression recognition based on relation modeling with graph convolutional networks. In: 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp 79–84. https://doi.org/10.1109/MIPR49039.2020.00023

  24. Liu Z, Jiang Z, Feng W, Feng H (2020) Od-gcn: object detection boosted by knowledge gcn. In: 2020 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp 1–6. https://doi.org/10.1109/ICMEW46912.2020.9105952

  25. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, vol 2, pp 416–4232. https://doi.org/10.1109/ICCV.2001.937655

  26. Jiang B, Zhang Z, Lin D, Tang J, Luo B (2019) Semi-supervised learning with graph learning-convolutional networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 11305–11312. https://doi.org/10.1109/CVPR.2019.01157

  27. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  Google Scholar 

  28. Xhaferra E, Cina E, Toti L (2022) Classification of standard fashion mnist dataset using deep learning based cnn algorithms. In: 2022 International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp 494–498. https://doi.org/10.1109/ISMSIT56059.2022.9932737

  29. Rusch TK, Bronstein MM, Mishra S (2023) A survey on oversmoothing in graph neural networks. ar**v:2303.10993

  30. Keriven N (2022) Not too little, not too much: a theoretical analysis of graph (over)smoothing. ar**v:2205.12156 [stat.ML]

  31. Gao X-Y, Yuan Q-X, Zhang C-X (2022) 3d model classification based on gcn and svm. IEEE Access 10:121494–121507. https://doi.org/10.1109/ACCESS.2022.3223384

    Article  Google Scholar 

  32. Maturana D, Scherer S (2015) Voxnet: a 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International conference on Intelligent Robots and Systems (IROS), pp 922–928. https://doi.org/10.1109/IROS.2015.7353481

  33. **e H, Yao H, Zhou S, Zhang S, Tong X, Sun W (2021) Toward 3d object reconstruction from stereo images. Neurocomputing 463:444–453. https://doi.org/10.1016/j.neucom.2021.07.089

    Article  Google Scholar 

Download references

Acknowledgements

This study was partly supported by the National Science and Technology Council (NSTC), Taiwan, under NSTC 111-2221-E-011 -162 -MY3 and 111-2221-E- 011 -163 -MY3.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Yi-Zeng Hsieh, Chia-Hsuan Wu & Yu-Ting Chen

Authors
  1. Yi-Zeng Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chia-Hsuan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Ting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-Zeng Hsieh.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Data Sharing

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hsieh, YZ., Wu, CH. & Chen, YT. Integrating self-organizing feature map with graph convolutional network for enhanced superpixel segmentation and feature extraction in non-Euclidean data structure. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19619-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19619-5

Keywords

Search

Navigation