SpringerLink
Log in

Enhancing of polyp image segmentation in colonoscopy images: a comprehensive approach using modified UNet, hybrid color space, and ensemble learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Polyp detection in its early stages can reduce the risk of colorectal cancer. Utilizing colonoscopy image segmentation enables expedited polyp diagnosis, further enhancing the effectiveness of detection. However, achieving accurate polyp image segmentation is a challenging task due to the variability in size, shape, and location of polyps. Expert experience directly affects this process, playing a pivotal role in determining its effectiveness. In this paper, we propose a novel method for automatic segmentation of polyp areas from colonoscopy images. Initially, we modified the UNet architecture using different backbones based on the transfer learning technique. In order to achieve better separation and localization of polyp regions, we adopt a hybrid LGB color space. The proposed color space merges the primary hues of green and blue with the Lightness component derived from CIE-L × a × b, resulting in a hybrid color representation. Furthermore, we propose a modified ResUNet architecture called Xcep-MResUNet, which uses the Xception backbone for feature extraction with an additional middle decoder to determine polyp areas. The middle decoder utilizes middle features to retrieve spatial information, while the ResUNet decoder utilizes high-level features. The proposed Xcep-MResUNet architecture combines the proposed middle decoder features with the ResUNet decoder features to refine the polyp area. Evaluation of the proposed method using the Kvasir-SEG database shows that our proposed method achieves more accurate results compared to other ResUNet-based models. Finally, using the Ensemble learning technique, we integrated the output of the proposed Xcep-MResUNet architecture with two of the best architectures based on modified UNet. The segmentation results of the proposed method were evaluated using the Dice similarity coefficient, IOU, sensitivity, and positive predictive value criteria. The corresponding values for these criteria were 0.8890, 0.8249, 0.9108, and 0.8992, respectively. Furthermore, we have performed additional experiments to check the generalizability capability of the proposed architecture on the CVC-ClinicDB dataset. The results show a good performance of the proposed Ensemble models with respect to conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Brazil)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Data availability

The dataset used during the current study is available in the [8, 46].

Notes

  1. https://www.uclahealth.org/Workfiles/brochures-programs/preparing-colonoscopy-en.pdf

  2. https://keras.io/api/applications/

References

  1. Asplund J, Kauppila JH, Mattsson F, Lagergren J (2018) Survival trends in gastric adenocarcinoma: a population-based study in Sweden. Ann Surg Oncol 25:2693–2702

    Article  Google Scholar 

  2. Siegel RL, Miller KD, Goding Sauer A, Fedewa SA, Butterly LF, Anderson JC et al (2020) Colorectal cancer statistics, 2020. CA Cancer J Clin 70(3):145–164

    Article  Google Scholar 

  3. Franz M, Scholz M, Henze I, Röckl S, Gomez LI (2013) Detection of colon polyps by a novel, polymer pattern-based full blood test. J Transl Med 11(1):1–9

    Article  Google Scholar 

  4. Yang K, Chang S, Tian Z, Gao C, Du Y, Zhang X et al (2022) Automatic polyp detection and segmentation using shuffle efficient channel attention network. Alexandria Eng J 61(1):917–926

    Article  Google Scholar 

  5. Stoitsis J, Valavanis I, Mougiakakou SG, Golemati S, Nikita A, Nikita KS (2006) Computer aided diagnosis based on medical image processing and artificial intelligence methods. Nuclear Instr Methods Phys Res Sec A: Accelerators, Spectrometers, Detectors and Associated Equipment 569(2):591–595

    Article  Google Scholar 

  6. Riegler M, Lux M, Griwodz C, Spampinato C, de Lange T, Eskeland SL et al (2016) Multimedia and medicine: teammates for better disease detection and survival. In: Proceedings of the 24th ACM international conference on multimedia, pp 968–977

    Chapter  Google Scholar 

  7. Pham DL, Xu C, Prince JL (2000) Current methods in medical image segmentation. Annu Rev Biomed Eng 2(1):315–337

    Article  Google Scholar 

  8. Jha D, Smedsrud PH, Riegler MA, Halvorsen P, de Lange T, Johansen D, Johansen HD (2020) Kvasir-seg: a segmented polyp dataset. In: In MultiMedia modeling: 26th international conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, proceedings, part II 26. Springer international publishing, pp 451–462

    Chapter  Google Scholar 

  9. Gupta N, Bhatele P, Khanna P (2019) Glioma detection on brain MRIs using texture and morphological features with ensemble learning. Biomed Signal Process Ctrl 47:115–125

    Article  Google Scholar 

  10. Todd C, Kirillov M, Tarabichi M, Naghdy F, Naghdy G (2009) An analysis of medical image processing methods for segmentation of the inner ear. University of Wollongong

    Google Scholar 

  11. Rahima Z, Ahror B, Basel S, Douraied BS, Souhil T (2018) Segmentation of low-grade gliomas based on the growing region and level sets techniques. In: 2018 4th international conference on advanced Technologies for Signal and Image Processing (ATSIP). IEEE, pp 1–5

    Google Scholar 

  12. Li Q, Gao Z, Wang Q, **a J, Zhang H, Zhang, Het al. (2018) Glioma segmentation with a unified algorithm in multimodal MRI images. IEEE Access 6:9543–9553

    Google Scholar 

  13. Haque IRI, Neubert J (2020) Deep learning approaches to biomedical image segmentation. Inform Med Unlocked 18:100297

    Article  Google Scholar 

  14. Rathod J, Waghmode V, Sodha A, Bhavathankar P (2018) Diagnosis of skin diseases using convolutional neural networks. In: 2018 second international conference on electronics, communication and aerospace technology (ICECA). IEEE, pp 1048–1051

    Chapter  Google Scholar 

  15. Chen S, Urban G, Baldi P (2022) Weakly supervised polyp segmentation in colonoscopy images using deep neural networks. J Imaging 8(5):121

    Article  Google Scholar 

  16. Ahsan, M. M., Luna, S. A., & Siddique, Z. (2022). Machine-learning-based disease diagnosis: a comprehensive review. In healthcare (Vol. 10, no. 3, p. 541). MDPI.

    Google Scholar 

  17. Shen H, Zhang J, Zheng W (2017) Efficient symmetry-driven fully convolutional network for multimodal brain tumor segmentation. In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 3864–3868

    Chapter  Google Scholar 

  18. Li L, Chen Y, Shen Z, Zhang X, Sang J, Ding Y et al (2020) Convolutional neural network for the diagnosis of early gastric cancer based on magnifying narrow band imaging. Gastric Cancer 23:126–132

    Article  Google Scholar 

  19. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  20. Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y et al (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  21. Hussain S, Anwar SM, Majid M (2018) Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282:248–261

    Article  Google Scholar 

  22. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: In medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer international publishing, pp 234–241

    Google Scholar 

  23. Zhang K, Sun M, Han TX, Yuan X, Guo L, Liu T (2017) Residual networks of residual networks: multilevel residual networks. IEEE Transactions on Circuits and Systems for Video Technology 28(6):1303–1314

    Article  Google Scholar 

  24. Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, Johansen HD (2019) Resunet++: an advanced architecture for medical image segmentation. In: 2019 IEEE international symposium on multimedia (ISM). IEEE, pp 225–2255

    Chapter  Google Scholar 

  25. Jha D, Ali S, Tomar NK, Johansen HD, Johansen D, Rittscher J et al (2021) Real-time polyp detection, localization and segmentation in colonoscopy using deep learning. IEEE Access 9:40496–40510

    Article  Google Scholar 

  26. Mahmud T, Paul B, Fattah SA (2021) PolypSegNet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Comput Biol Med 128:104119

    Article  Google Scholar 

  27. Huang, C. H., Wu, H. Y., & Lin, Y. L. (2021). Hardnet-mseg: a simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps. https://arxiv.org/abs/2101.07172.

  28. Fan DP, Ji GP, Zhou T, Chen G, Fu H, Shen J, Shao L (2020) Pranet: parallel reverse attention network for polyp segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer International Publishing, Cham, pp 263–273

    Google Scholar 

  29. Tomar NK, Jha D, Ali S, Johansen HD, Johansen D, Riegler MA, Halvorsen P (2021) DDANet: dual decoder attention network for automatic polyp segmentation. In: In pattern recognition. ICPR international workshops and challenges: virtual event, January 10-15, 2021, proceedings, part VIII. Springer international publishing, pp 307–314

    Google Scholar 

  30. Rauniyar S, Jha VK, Jha RK, Jha D, Rauniyar A (2021) Improving polyp segmentation in colonoscopy using deep learning. Nordic Machine Intell 1(1):35–37

    Article  Google Scholar 

  31. Elmeslimany EM, Kishk SS, Altantawy DA (2024) Ψnet: a parallel network with deeply coupled spatial and squeezed features for segmentation of medical images. Multimedia Tools Appl 83(8):24045–24082

    Article  Google Scholar 

  32. Wu C, Long C, Li S, Yang J, Jiang F, Zhou R (2022) MSRAformer: multiscale spatial reverse attention network for polyp segmentation. Comput Biol Med 151:106274

    Article  Google Scholar 

  33. Jain Y, Saxena V, Mittal S (2022) Ensembling deep learning and CIELAB color space model for fire detection from UAV images. In: Proceedings of the second international conference on AI-ML systems, pp 1–9

    Google Scholar 

  34. Niranjana KK, Devi MK (2015) RGB to lab transformation using image segmentation. Image 3(11)

  35. Lakio S, Heinämäki J, Yliruusi J (2010) Colorful drying. Aaps Pharmscitech 11:46–53

  36. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 25

  37. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition https://arxiv.org/abs/1409.1556.

  38. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  39. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

    Google Scholar 

  40. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. https://arxiv.org/abs/1704.04861.

  41. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520

    Google Scholar 

  42. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

    Google Scholar 

  43. Tan M, Le Q (2019) Efficientnet: rethinking model scaling for convolutional neural networks. In: International conference on machine learning. PMLR, pp 6105–6114

    Google Scholar 

  44. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

    Google Scholar 

  45. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 15(5):749–753

    Article  Google Scholar 

  46. Salem MH, Li Y, Liu Z, AbdelTawab AM (2023) A transfer learning and optimized CNN based maritime vessel classification system. Appl Sci 13(3):1912

    Article  Google Scholar 

  47. Large J, Lines J, Bagnall A (2019) A probabilistic classifier ensemble weighting scheme based on cross-validated accuracy estimates. Data Min Knowl Disc 33(6):1674–1709

    Article  MathSciNet  Google Scholar 

  48. Dubey SR, Singh SK, Chaudhuri BB (2022) Activation functions in deep learning: a comprehensive survey and benchmark. Neurocomputing 503:92–108

    Article  Google Scholar 

  49. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111

    Article  Google Scholar 

  50. Dawod AY, Phaphuangwittayakul A (2021) Adaptive image segmentation for traumatic brain Haemorrhage. TEM J 10(3):1476

    Article  Google Scholar 

Download references

Funding

No organization funded this research.

Author information

Authors and Affiliations

  1. Malek-Ashtar University of Technology, Tehran, Iran

    Motahareh Aghalari & Hossein Khaleghi Bizaki

Authors
  1. Motahareh Aghalari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Khaleghi Bizaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M. Aghalari designed the model, implemented the research, and wrote the original version of the draft. H. Khalghi Bezaki validated the methodology and reviewed the final version.

Corresponding author

Correspondence to Hossein Khaleghi Bizaki.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghalari, M., Bizaki, H.K. Enhancing of polyp image segmentation in colonoscopy images: a comprehensive approach using modified UNet, hybrid color space, and ensemble learning. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19703-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19703-w

Keywords

Search

Navigation