Abstract
Semi-supervised learning methods have been attracting much attention in medical image segmentation due to the lack of high-quality annotation. To cope with the noise problem of pseudo-label in semi-supervised medical image segmentation and the limitations of contrastive learning applications, we propose a semi-supervised medical image segmentation framework, HPFG, based on hybrid pseudo-label and feature-guiding, which consists of a hybrid pseudo-label strategy and two different feature-guiding modules. The hybrid pseudo-label strategy uses the CutMix operation and an auxiliary network to enable the labeled images to guide the unlabeled images to generate high-quality pseudo-label and reduce the impact of pseudo-label noise. In addition, a feature-guiding encoder module based on feature-level contrastive learning is designed to guide the encoder to mine useful local and global image features, thus effectively enhancing the feature extraction capability of the model. At the same time, a feature-guiding decoder module based on adaptive class-level contrastive learning is designed to guide the decoder in better extracting class information, achieving intra-class affinity and inter-class separation, and effectively alleviating the class imbalance problem in medical datasets. Extensive experimental results show that the segmentation performance of the HPFG framework proposed in this paper outperforms existing semi-supervised medical image segmentation methods on three public datasets: ACDC, LIDC, and ISIC. Code is available at https://github.com/fakerlove1/HPFG.
Graphical abstract
Similar content being viewed by others
References
Raghu M, Zhang C, Kleinberg J, et al (2019) Transfusion: understanding transfer learning for medical imaging[J]. Adv Neural Inform Process Syst 32. https://doi.org/10.48550/ar**v.1902.07208
You C, Zhao R, Liu F, et al (2022) Class-aware generative adversarial transformers for medical image segmentation[J]. ar**v preprint ar**v:2201.10737. https://doi.org/10.48550/ar**v.2201.10737
Isensee F, Petersen J, Klein A, et al (2018) nnu-net: self-adapting framework for u-net-based medical image segmentation[J]. ar**v preprint ar**v:1809.10486. https://doi.org/10.1007/978-3-658-25326-4_7
Luo X, Chen J, Song T, et al (2021) Semi-supervised medical image segmentation through dual-task consistency[C]. Proc AAAI Conf Artif Intell 35(10):8801–8809. https://doi.org/10.48550/ar**v.2009.04448
Basak H, Bhattacharya R, Hussain R, et al (2022) An embarrassingly simple consistency regularization method for semi-supervised medical image segmentation[J]. ar**v preprint ar**v:2202.00677. https://doi.org/10.48550/ar**v.2202.00677
Wu Y, Ge Z, Zhang D et al (2022) Mutual consistency learning for semi-supervised medical image segmentation[J]. Med Image Anal 81:102530. https://doi.org/10.48550/ar**v.2112.02508
Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results[J]. Adv Neural Inform Process Syst 30. https://doi.org/10.48550/ar**v.1703.01780
Lee DH (2013) Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks[C]//Workshop on challenges in representation learning. ICML 3(2):896
Zou Y, Zhang Z, Zhang H, et al (2020) Pseudoseg: designing pseudo labels for semantic segmentation[J]. ar**v preprint ar**v:2010.09713. https://doi.org/10.48550/ar**v.2010.09713
Sohn K, Berthelot D, Carlini N et al (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence[J]. Adv Neural Inform Process Syst 33:596–608. https://doi.org/10.48550/ar**v.2001.07685
Wang K, Zhan B, Zu C et al (2022) Semi-supervised medical image segmentation via a tripled-uncertainty guided mean teacher model with contrastive learning[J]. Med Image Anal 79:102447. https://doi.org/10.1016/j.media.2022.102447
Liu Y, Wang W, Luo G et al (2022) A contrastive consistency semi-supervised left atrium segmentation model[J]. Comput Med Imaging Graph 99:102092. https://doi.org/10.1016/j.compmedimag.2022.102092
You C, Zhao R, Staib L H, et al (2022) Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part IV. Cham: Springer Nature Switzerland 639-652.https://doi.org/10.1007/978-3-031-16440-8_61
Lei T, Zhang D, Du X et al (2022) Semi-supervised medical image segmentation using adversarial consistency learning and dynamic convolution network[J]. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2022.3225687
Miyato T, Maeda S, Koyama M et al (2018) Virtual adversarial training: a regularization method for supervised and semi-supervised learning[J]. IEEE Trans Pattern Anal Mach Intell 41(8):1979–1993. https://doi.org/10.1109/TPAMI.2018.2858821
Qiao S, Shen W, Zhang Z, et al (2018) Deep co-training for semi-supervised image recognition[C]. Proc Eur Conf Comput Vision (ECCV) 135–152. https://doi.org/10.48550/ar**v.1803.05984
Wang Z, Li T, Zheng J Q, et al (2023) When CNN meet with ViT: towards semi-supervised learning for multi-class medical image semantic segmentation[C]. Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII. Cham: Springer Nature Switzerland 424-441.https://doi.org/10.1007/978-3-031-25082-8_28
Yeung M, Sala E, Schönlieb CB et al (2022) Unified focal loss: generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation[J]. Comput Med Imaging Graph 95:102026. https://doi.org/10.1016/j.compmedimag.2021.102026
Hu H, Wei F, Hu H et al (2021) Semi-supervised semantic segmentation via adaptive equalization learning[J]. Adv Neural Inform Process Syst 34:22106–22118
Armato SG III, McLennan G, Bidaut L et al (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans[J]. Med Phys 38(2):915–931. https://doi.org/10.1118/1.3528204
Chen X, Yuan Y, Zeng G, et al (2021) Semi-supervised semantic segmentation with cross pseudo supervision[C]. Proc IEEE/CVF Conf Comput Vision Pattern Recog 2613–2622. https://doi.org/10.48550/ar**v.2106.01226
Luo X, Hu M, Song T, et al (2022) Semi-supervised medical image segmentation via cross teaching between CNN and transformer[C]. Int Conf Med Imaging Deep Learn PMLR 820–833. https://doi.org/10.48550/ar**v.2112.04894
Dosovitskiy A, Beyer L, Kolesnikov A, et al (2020) An image is worth 16x16 words: transformers for image recognition at scale[J]. ar**v preprint ar**v:2010.11929. https://doi.org/10.48550/ar**v.2010.11929
Shen Z, Cao P, Yang H, et al (2023) Co-training with high-confidence pseudo labels for semi-supervised medical image segmentation[J]. ar**v preprint ar**v:2301.04465. https://doi.org/10.48550/ar**v.2301.04465
Zhang Z, Tian C, Bai HX, Jiao Z, Tian X (2022) Discriminaive error prediction network for semi-supervised colon gland segmentation. Med Image Anal 79:102458. https://doi.org/10.1016/j.media.2022.102458
Zhao X, Qi Z, Wang S, et al (2023) RCPS: Rectified contrastive pseudo supervision for semi-supervised medical image segmentation[J]. ar**v preprint ar**v:2301.05500. https://doi.org/10.48550/ar**v.2301.05500
Wang X, Zhang R, Shen C, et al (2021) Dense contrastive learning for self-supervised visual pre-training[C]. Proc IEEE/CVF Conf Comput Vision Pattern Recog 3024–3033. https://doi.org/10.48550/ar**v.2011.09157
Chen T, Kornblith S, Norouzi M, et al (2020) A simple framework for contrastive learning of visual representations[C]. Int Conf Mach Learn PMLR 1597–1607. https://doi.org/10.48550/ar**v.2002.05709
He K, Fan H, Wu Y, et al (2020) Momentum contrast for unsupervised visual representation learning[C]. Proc IEEE/CVF Conf Comput Vision Pattern Recog 9729–9738. https://doi.org/10.48550/ar**v.1911.05722
Chaitanya K, Erdil E, Karani N et al (2020) Contrastive learning of global and local features for medical image segmentation with limited annotations[J]. Adv Neural Inform Process Syst 33:12546–12558. https://doi.org/10.48550/ar**v.2006.10511
Hu X, Zeng D, Xu X, et al (2021) Semi-supervised contrastive learning for label-efficient medical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24. Springer International Publishing 481-490.https://doi.org/10.1007/978-3-030-87196-3_45
Chaitanya K, Erdil E, Karani N, et al (2023) Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation[J]. Med Image Anal 102792. https://doi.org/10.1016/j.media.2023.102792
Wu Y, Wu Z, Wu Q, et al (2022) Exploring smoothness and class-separation for semi-supervised medical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V. Cham: Springer Nature Switzerland 34-43.https://doi.org/10.1007/978-3-031-16443-9_4
Wang T, Lu J, Lai Z, et al (2022) Uncertainty-guided pixel contrastive learning for semi-supervised medical image segmentation[C]. Proc Thirty-First Int Joint Conf Artif Intell IJCAI 1444–1450. https://doi.org/10.1142/S0129065722500162
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing 234-241.https://doi.org/10.1007/978-3-319-24574-4_28
Yun S, Han D, Oh S J, et al (2019) Cutmix: regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the IEEE/CVF international conference on computer vision 6023–6032. https://doi.org/10.48550/ar**v.1905.04899
Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]//2016 fourth international conference on 3D vision (3DV). IEEE 565–571. https://doi.org/10.1109/3DV.2016.79
Oord A, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding[J]. ar**v preprint ar**v:1807.03748. https://doi.org/10.48550/ar**v.1807.03748
Bernard O, Lalande A, Zotti C et al (2018) Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?[J]. IEEE Trans Med Imaging 37(11):2514–2525. https://doi.org/10.1109/TMI.2018.2837502
Codella N C F, Gutman D, Celebi M E, et al (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC)[C]//2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE 168-172.https://doi.org/10.1109/ISBI.2018.8363547
Selvaraju R R, Cogswell M, Das A, et al (2017) Grad-cam: visual explanations from deep networks via gradient-based localization[C]. Proc IEEE Int Conf Comput Vision 618–626. https://doi.org/10.1007/s11263-019-01228-7
Acknowledgements
The authors thank the editors and reviewers for their valuable comments that improve the paper’s quality.
Funding
This work is supported by the Fundamental Research Program of Shanxi Province (No.202203021211177).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, F., Jiang, A., Li, M. et al. HPFG: semi-supervised medical image segmentation framework based on hybrid pseudo-label and feature-guiding. Med Biol Eng Comput 62, 405–421 (2024). https://doi.org/10.1007/s11517-023-02946-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-023-02946-4