SpringerLink
Log in

RA-Net: Region-Aware Attention Network for Skin Lesion Segmentation

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The precise segmentation of skin lesion in dermoscopic images is essential for the early detection of skin cancer. However, the irregular shapes of the lesions, the absence of sharp edges, the existence of artifacts like hair follicles, and marker color make this task difficult. Currently, fully connected networks (FCNs) and U-Nets are the most commonly used techniques for melanoma segmentation. However, as the depth of these neural network models increases, they become prone to various challenges. The most pertinent of these challenges are the vanishing gradient problem and the parameter redundancy problem. These can result in a decline in Jaccard index of the segmentation model. This study introduces a novel end-to-end trainable network designed for skin lesion segmentation. The proposed methodology consists of an encoder-decoder, a region-aware attention approach, and guided loss function. The trainable parameters are reduced using depth-wise separable convolution, and the attention features are refined using a guided loss, resulting in a high Jaccard index. We assessed the effectiveness of our proposed RA-Net on four frequently utilized benchmark datasets for skin lesion segmentation: ISIC 2016, ISIC 2017, ISIC 2018, and PH2. The empirical results validate that our method achieves state-of-the-art performance, as indicated by a notably high Jaccard index.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The ISIC 2016 [48], ISIC 2017 [49], and ISIC 2018 [50, 51] datasets are available publicly and can be accessed from: ISICDatasets. The PH2 [52] dataset is also available publicly and can be accessed from: PH2.

References

  1. Siegel RL, Miller KD, Wagle NS, Jemal A. Cancer statistics, 2023. CA Cancer J Clin. 2023;73(1):17–48.

    Article  Google Scholar 

  2. Wang X, Jiang X, Ding H, Liu J. Bi-directional dermoscopic feature learning and multi-scale consistent decision fusion for skin lesion segmentation. IEEE Trans Image Process. 2020;29:3039–51. https://doi.org/10.1109/TIP.2019.2955297.

    Article  Google Scholar 

  3. Wu H, Chen S, Chen G, Wang W, Lei B, Wen Z. FAT-Net: feature adaptive transformers for automated skin lesion segmentation. Med Image Anal. 2022;76:102327. https://doi.org/10.1016/j.media.2021.102327.

    Article  Google Scholar 

  4. Hu K, Lu J, Lee D, **ong D, Chen Z. AS-Net: attention synergy network for skin lesion segmentation. Expert Syst Appl. 2022;201:117112.

    Article  Google Scholar 

  5. Kharazmi P, AlJasser MI, Lui H, Wang ZJ, Lee TK. Automated detection and segmentation of vascular structures of skin lesions seen in dermoscopy, with an application to basal cell carcinoma classification. IEEE J Biomed Health Inform. 2016;21(6):1675–84.

    Article  Google Scholar 

  6. Bi L, Fulham M, Kim J. Hyper-fusion network for semi-automatic segmentation of skin lesions. Med Image Anal. 2022;76:102334.

  7. Yueksel ME, Borlu M. Accurate segmentation of dermoscopic images by image thresholding based on type-2 fuzzy logic. IEEE Trans Fuzzy Syst. 2009;17(4):976–82.

    Article  Google Scholar 

  8. Yu L, Chen H, Dou Q, Qin J, Heng P-A. Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging. 2016;36(4):994–1004.

    Article  Google Scholar 

  9. Cao W, Yuan G, Liu Q, Peng C, **e J, Yang X, Ni X, Zheng J. ICL-Net: Global and local inter-pixel correlations learning network for skin lesion segmentation. IEEE J Biomed Health Inform. 2022;27(1):145–56. IEEE.

  10. Zhang W, Lu F, Zhao W, Hu Y, Su H, Yuan M. ACCPG-Net: a skin lesion segmentation network with adaptive channel-context-aware pyramid attention and global feature fusion. Comput Biol Med. 2023;154:106580. Elsevier.

  11. Tang P, Liang Q, Yan X, **ang S, Sun W, Zhang D, Coppola G. Efficient skin lesion segmentation using separable-UNet with stochastic weight averaging. Comput Methods Programs Biomed. 2019;178:289–301. https://doi.org/10.1016/j.cmpb.2019.07.005.

    Article  Google Scholar 

  12. Bi L, Kim J, Ahn E, Kumar A, Feng D, Fulham M. Step-wise integration of deep class-specific learning for dermoscopic image segmentation. Pattern Recogn. 2019;85:78–89. https://doi.org/10.1016/j.patcog.2018.08.001.

    Article  Google Scholar 

  13. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Springer; 2015. pp. 234–41.

  14. Mahmud M, Kaiser MS, Hussain A, Vassanelli S. Applications of deep learning and reinforcement learning to biological data. IEEE Trans Neural Netw Learn Syst. 2018;29(6):2063–79.

    Article  MathSciNet  Google Scholar 

  15. Mahmud M, Kaiser MS, McGinnity TM, Hussain A. Deep learning in mining biological data. Cogn Comput. 2021;13:1–33.

    Article  Google Scholar 

  16. Dai D, Dong C, Xu S, Yan Q, Li Z, Zhang C, Luo N. Ms RED: a novel multi-scale residual encoding and decoding network for skin lesion segmentation. Med Image Anal. 2022;75:102293. https://doi.org/10.1016/j.media.2021.102293.

    Article  Google Scholar 

  17. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017. pp. 4700–8.

  18. Emre Celebi M, Kingravi HA, Iyatomi H, Alp Aslandogan Y, Stoecker WV, Moss RH, Malters JM, Grichnik JM, Marghoob AA, Rabinovitz HS, et al. Border detection in dermoscopy images using statistical region merging. Skin Res Technol. 2008;14(3):347–53.

    Article  Google Scholar 

  19. Emre Celebi M, Wen Q, Hwang S, Iyatomi H, Schaefer G. Lesion border detection in dermoscopy images using ensembles of thresholding methods. Skin Research and Technology. 2013;19(1):252–8.

    Article  Google Scholar 

  20. Erkol B, Moss RH, Joe Stanley R, Stoecker WV, Hvatum E. Automatic lesion boundary detection in dermoscopy images using gradient vector flow snakes. Skin Research and Technology. 2005;11(1):17–26.

    Article  Google Scholar 

  21. Ma Z, Tavares JMRS. A novel approach to segment skin lesions in dermoscopic images based on a deformable model. IEEE J Biomed Health Inform. 2016;20(2):615–23. https://doi.org/10.1109/JBHI.2015.2390032.

    Article  Google Scholar 

  22. Schmid P. Lesion detection in dermatoscopic images using anisotropic diffusion and morphological flooding. In: Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348). 1999. pp. 449–4533. https://doi.org/10.1109/ICIP.1999.817154.

  23. Yuan Y, Lo Y-C. Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. IEEE J Biomed Health Inform. 2017;23(2):519–26.

    Article  Google Scholar 

  24. Tang Y, Yang F, Yuan S, Zhan C. A multi-stage framework with context information fusion structure for skin lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). 2019. pp. 1407–10. https://doi.org/10.1109/ISBI.2019.8759535.

  25. Zhang G, Shen X, Chen S, Liang L, Luo Y, Yu J, Lu J. DSM: a deep supervised multi-scale network learning for skin cancer segmentation. IEEE Access. 2019;7:140936–45. https://doi.org/10.1109/ACCESS.2019.2943628.

    Article  Google Scholar 

  26. Nasr-Esfahani E, Rafiei S, Jafari MH, Karimi N, Wrobel JS, Samavi S, Reza Soroushmehr SM. Dense pooling layers in fully convolutional network for skin lesion segmentation. Comput Med Imaging Graph. 2019;78:101658. https://doi.org/10.1016/j.compmedimag.2019.101658.

    Article  Google Scholar 

  27. Hasan MK, Dahal L, Samarakoon PN, Tushar FI, Martí R. DSNet: automatic dermoscopic skin lesion segmentation. Comput Biol Med. 2020;120:103738. https://doi.org/10.1016/j.compbiomed.2020.103738.

    Article  Google Scholar 

  28. Abhishek K, Hamarneh G, Drew MS. Illumination-based transformations improve skin lesion segmentation in dermoscopic images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020. pp. 728–9.

  29. Lei B, **a Z, Jiang F, Jiang X, Ge Z, Xu Y, Qin J, Chen S, Wang T, Wang S. Skin lesion segmentation via generative adversarial networks with dual discriminators. Med Image Anal. 2020;64:101716. https://doi.org/10.1016/j.media.2020.101716.

    Article  Google Scholar 

  30. Chen Y, Kalantidis Y, Li J, Yan S, Feng J. A\({}^\wedge \) 2-Nets: double attention networks. Adv Neural Inf Process Syst. 2018;31.

  31. Li X, Zhong Z, Wu J, Yang Y, Lin Z, Liu H. Expectation-maximization attention networks for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. pp. 9167–76.

  32. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Proces Syst. 2017;30.

  33. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H. Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. pp. 3146–54.

  34. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D. Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal. 2019;53:197–207.

    Article  Google Scholar 

  35. Zhang S, Fu H, Yan Y, Zhang Y, Wu Q, Yang M, Tan M, Xu Y. Attention guided network for retinal image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Springer; 2019. pp. 797–805.

  36. He A, Li T, Li N, Wang K, Fu H. CABNet: category attention block for imbalanced diabetic retinopathy grading. IEEE Trans Med Imaging. 2020;40(1):143–53.

    Article  Google Scholar 

  37. Chen B, Liu Y, Zhang Z, Lu G, Kong AWK. TransATTUnet: multi-level attention-guided U-Net with transformer for medical image segmentation. IEEE Transactions on Emerging Topics in Computational Intelligence. 2023. https://doi.org/10.1109/TETCI.2023.3309626.

    Article  Google Scholar 

  38. Singh VK, Abdel-Nasser M, Rashwan HA, Akram F, Pandey N, Lalande A, Presles B, Romani S, Puig D. FCA-Net: adversarial learning for skin lesion segmentation based on multi-scale features and factorized channel attention. IEEE Access. 2019;7:130552–65. https://doi.org/10.1109/ACCESS.2019.2940418.

    Article  Google Scholar 

  39. Hu K, Lu J, Lee D, **ong D, Chen Z. AS-Net: attention synergy network for skin lesion segmentation. Expert Syst Appl. 2022;201:117112. https://doi.org/10.1016/j.eswa.2022.117112.

    Article  Google Scholar 

  40. Basak H, Kundu R, Sarkar R. MFSNet: a multi focus segmentation network for skin lesion segmentation. Pattern Recogn. 2022;128:108673. https://doi.org/10.1016/j.patcog.2022.108673.

    Article  Google Scholar 

  41. Lin M, Chen Q, Yan S. Network in network. ar**v:1312.4400 [Preprint]. 2013. Available from: http://arxiv.org/abs/1312.4400.

  42. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR; 2015. pp. 448–56.

  43. Chollet F. Xception: deep learning with depthwise separable convolutions. CVPR 2017. ar**v:1610.02357 [Preprint]. 2017. Available from: http://arxiv.org/abs/1610.02357.

  44. Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Martinez-Gonzalez P, Garcia-Rodriguez J. A survey on deep learning techniques for image and video semantic segmentation. Appl Soft Comput. 2018;70:41–65. https://doi.org/10.1016/j.asoc.2018.05.018.

    Article  Google Scholar 

  45. Jadon S. A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE; 2020. pp. 1–7.

  46. van Beers F, Lindström A, Okafor E, Wiering MA. Deep neural networks with intersection over union loss for binary image segmentation. In: ICPRAM. SciTePress; 2019. pp. 438–45.

  47. Abraham N, Khan NM. A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE; 2019. pp. 683–7.

  48. Gutman D, Codella NC, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A. Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). ar**v:1605.01397 [Preprint]. 2016. Available from: http://arxiv.org/abs/1605.01397.

  49. Codella NC, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, et al. Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE; 2018. pp. 168–72.

  50. Tschandl P, Rosendahl C, Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data. 2018;5(1):1–9.

    Article  Google Scholar 

  51. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M, et al. Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). ar**v:1902.03368 [Preprint]. 2019. Available form: http://arxiv.org/abs/1902.03368.

  52. Mendonça T, Ferreira PM, Marques JS, Marques AR, Rozeira J. PH\(^2\)-a dermoscopic image database for research and benchmarking. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE; 2013. pp. 5437–40.

  53. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017. pp. 618–26

  54. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. pp. 4510–20.

  55. Tan M, Le Q. EfficientNetV2: smaller models and faster training. In: International Conference on Machine Learning. PMLR; 2021. pp. 10096–106.

  56. Xu Q, Ma Z, Na H, Duan W. DCSAU-Net: a deeper and more compact split-attention U-Net for medical image segmentation. Comput Biol Med. 2023;154:106626.

    Article  Google Scholar 

  57. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. UNet++: a nested U-Net architecture for medical image segmentation. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer; 2018. pp. 3–11.

  58. Feng K, Ren L, Wang G, Wang H, Li Y. SLT-NET: a codec network for skin lesion segmentation. Comput Biol Med. 2022;148:105942. https://doi.org/10.1016/j.compbiomed.2022.105942.

    Article  Google Scholar 

  59. Feng S, Zhao H, Shi F, Cheng X, Wang M, Ma Y, **ang D, Zhu W, Chen X. CPFNet: context pyramid fusion network for medical image segmentation. IEEE Trans Med Imaging. 2020;39(10):3008–18. https://doi.org/10.1109/TMI.2020.2983721.

    Article  Google Scholar 

  60. Lee HJ, Kim JU, Lee S, Kim HG, Ro YM. Structure boundary preserving segmentation for medical image with ambiguous boundary. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020. pp. 4817–26. https://doi.org/10.1109/CVPR42600.2020.00487

  61. Maji D, Sigedar P, Singh M. Attention Res-Unet with guided decoder for semantic segmentation of brain tumors. Biomed Signal Process Control. 2022;71:103077.

    Article  Google Scholar 

  62. ** Q, Cui H, Sun C, Meng Z, Su R. Cascade knowledge diffusion network for skin lesion diagnosis and segmentation. Appl Soft Comput. 2021;99:106881. https://doi.org/10.1016/j.asoc.2020.106881.

    Article  Google Scholar 

  63. ** model for automated skin lesion segmentation and classification. IEEE Trans Med Imaging. 2020;39(7):2482–93. https://doi.org/10.1109/TMI.2020.2972964.

    Article  Google Scholar 

  64. Zuo B, Lee F, Chen Q. An efficient u-shaped network combined with edge attention module and context pyramid fusion for skin lesion segmentation. Med Biol Eng Comput. 2022;60(7):1987–2000. Springer.

  65. Bi L, Kim J, Ahn E, Kumar A, Feng D, Fulham M. Step-wise integration of deep class-specific learning for dermoscopic image segmentation. Pattern Recogn. 2019;85:78–89.

    Article  Google Scholar 

  66. Ji C, Deng Z, Ding Y, Zhou F, **ao Z. RMMLP: rolling MLP and matrix decomposition for skin lesion segmentation. Biomed Signal Process Control. 2023;84:104825.

    Article  Google Scholar 

  67. Goyal M, Oakley A, Bansal P, Dancey D, Yap MH. Skin lesion segmentation in dermoscopic images with ensemble deep learning methods. IEEE Access. 2019;8:4171–81.

    Article  Google Scholar 

  68. Zafar K, Gilani SO, Waris A, Ahmed A, Jamil M, Khan MN, Sohail Kashif A. Skin lesion segmentation from dermoscopic images using convolutional neural network. Sensors. 2020;20(6):1601.

    Article  Google Scholar 

  69. Wang R, Chen S, Ji C, Li Y. Cascaded context enhancement network for automatic skin lesion segmentation. Expert Syst Appl. 2022;201:117069.

    Article  Google Scholar 

  70. Qin C, Zheng B, Zeng J, Chen Z, Zhai Y, Genovese A, Piuri V, Scotti F. Dynamically aggregating MLPs and CNNs for skin lesion segmentation with geometry regularization. Comput Methods Programs Biomed. 2023;238:107601.

    Article  Google Scholar 

  71. Jiang X, Jiang J, Wang B, Yu J, Wang J. SEACU-Net: attentive ConvLSTM U-Net with squeeze-and-excitation layer for skin lesion segmentation. Comput Methods Programs Biomed. 2022;225:107076.

    Article  Google Scholar 

Download references

Funding

There is no funding for this work.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, COMSATS University Islamabad (CUI), Islamabad, 45550, Pakistan

    Asim Naveed, Syed S. Naqvi, Shahzaib Iqbal & Haroon Ahmed Khan

  2. Department of Computer Science and Engineering, University of Engineering and Technology (UET) Lahore, Narowal Campus, Narowal, 51600, Pakistan

    Asim Naveed

  3. School of Computer Science & Engineering, UNSW, Kensington, Sydney, 2052, NSW, Australia

    Imran Razzak & Tariq M. Khan

Authors
  1. Asim Naveed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Syed S. Naqvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahzaib Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Imran Razzak
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haroon Ahmed Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tariq M. Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Asim Naveed.

Ethics declarations

Ethics Approval

The authors have stated that no studies involving human participants or animals were conducted for this article.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naveed, A., Naqvi, S.S., Iqbal, S. et al. RA-Net: Region-Aware Attention Network for Skin Lesion Segmentation. Cogn Comput (2024). https://doi.org/10.1007/s12559-024-10304-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10304-1

Keywords

Search

Navigation