SpringerLink
Log in

Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Emerging semi-supervised learning methods have enabled great progress in segmentation tasks. However, popular semi-supervised segmentation models use constraints that are not strict. In this paper, we propose a new method, multi-scale cross pseudo-supervision, that introduces higher constraints by multi-scale information to improve the quality of pseudo-labels. Specifically, we extend the backbone segmentation network by adding a multi-scale feature pyramid at the decoder to extract multi-scale information. In addition, to further enhance the consistency on multiple scales, we perform perturbation operations on the original input image. Experiments show that our method achieves excellent segmentation performance on both sonar and ISIC2016 datasets. The performance gain benefits from two techniques—multi-scale constraints and perturbation consistency. And the proposed method alleviates the annotation pressure for image segmentation in real-world human-centric applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Chuang, M.C., Hwang, J.N., Ye, J.H., Huang, S.C., Williams, K.: Underwater fish tracking for moving cameras based on deformable multiple kernels. IEEE Trans. Syst. Man Cybern. Syst. 47(9), 2467–2477 (2017)

    Google Scholar 

  2. Karoui, I., Quidu, I., Legris, M.: Automatic sea-surface obstacle detection and tracking in forward-looking sonar image sequences. IEEE Trans. Geosci. Remote Sens. 53(8), 4661–4669 (2015)

    Article  Google Scholar 

  3. Renga, A., et al.: SAR bathymetry in the tyrrhenian sea by COSMO-SkyMed data: a novel approach. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 7(7), 2834–2847 (2014)

    Article  Google Scholar 

  4. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)

  5. Smirnov, E.A., Timoshenko, D.M., Andrianov, S.N.: Comparison of regularization methods for ImageNet classification with deep convolutional neural networks. AASRI Procedia 6, 89–94 (2014)

    Article  Google Scholar 

  6. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  7. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)

    Article  Google Scholar 

  8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. Acm 60(6), 84–90 (2017)

    Article  Google Scholar 

  9. Szegedy, C., et al.: Going deeper with convolutions. In: Presented at the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

  10. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: presented at the medical image computing and computer-assisted intervention, PT III (2015)

  11. He, K.M., Zhang, X.Y., Ren, S.Q., Sun, J., Ieee: Deep residual learning for image recognition. In: Presented at the 2016 IEEE conference on computer vision and pattern recognition (CVPR) (2016)

  12. Shotton, J., Johnson, M., Cipolla, R., Ieee: Semantic texton forests for image categorization and segmentation. In: Presented at the 2008 IEEE Conference on Computer Vision and Pattern Recognition, VOLS, pp 1–12 (2008)

  13. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  14. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. Siam J. Appl. Math. 66(5), 1632–1648 (2006)

    Article  MathSciNet  Google Scholar 

  15. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)

    Article  Google Scholar 

  16. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  17. Sarker, M.M.K., et al.: SLSDeep: skin lesion segmentation based on dilated residual and pyramid pooling networks. In: Presented at the Medical Image Computing and Computer Assisted Intervention—MICCAI 2018, PT II (2018)

  18. Milletari, F., Navab, F., Ahmadi, S.A., Ieee: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Presented at the 2016 Fourth International Conference On 3D Vision (3DV) (2016)

  19. Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)

    Article  Google Scholar 

  20. Mehta, D., et al.: Simple and efficient architectures for semantic segmentation. In: Presented at the 2022 IEEE conference on computer vision and pattern recognition (CVPR) (2022)

  21. Svanera, M., Savardi, M., Signoroni, A., Benini, S., Muckli, L.: Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data. Med. Image Anal. 93, 103090 (2024)

    Article  Google Scholar 

  22. Ibtehaz, N., Kihara, D.: ACC-UNet: a completely convolutional UNet model for the 2020s. In: Presented at 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), vol. 14222, pp. 692–702 (2023)

  23. Zhu, P.P., Isaacs, J., Fu, B., Ferrari, S., and Ieee.: Deep learning feature extraction for target recognition and classification in underwater sonar images. In: Presented at the 2017 IEEE 56TH Annual Conference On Decision And Control (CDC) (2017)

  24. Valdenegro-Toro, M., Ieee.: Best practices in convolutional networks for forward-looking sonar image recognition. In: Presented at the OCEANS 2017—ABERDEEN (2017)

  25. Kim, J., Cho, H., Pyo, J., Kim, B., Yu, S.C., Ieee: The convolution neural network based agent vehicle detection using forward-looking sonar image. In: Presented at the Oceans 2016 MTS/IEEE Monterey (2016)

  26. Zhang, B., Zhou, T., Shi, Z., Xu, C., Yang, K., Yu, X.: An underwater small target boundary segmentation method in forward-looking sonar images. Appl. Acoust. 207, 109341 (2023)

    Article  Google Scholar 

  27. Song, Y., He, B., Liu, P.: Real-time object detection for AUVs using self-cascaded convolutional neural networks. IEEE J. Ocean. Eng. 46(1), 56–67 (2021)

    Article  Google Scholar 

  28. Dzieciuch, I., Gebhardt, D., Barngrover, C., Parikh, K.: Non-linear convolutional neural network for automatic detection of mine-like objects in sonar imagery. In: Presented at the 4th International Conference on Applications in Nonlinear Dynamics (ICAND) (2017)

  29. Song, Y., et al.: Side scan sonar segmentation using deep convolutional neural network. In: Presented at the Oceans 2017–Anchorage (2017)

  30. Wang, Z., Guo, J., Huang, W., Zhang, S.: Side-scan sonar image segmentation based on multi-channel fusion convolution neural networks. IEEE Sens. J. 22(6), 5911–5928 (2022)

    Article  Google Scholar 

  31. Wang, Q., Zhang, Y., He, B.: Automatic seabed target segmentation of AUV via multilevel adversarial network and marginal distribution adaptation. IEEE Trans. Industr. Electron. 71, 749–759 (2023)

    Article  Google Scholar 

  32. Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Presented at the Advances in Neural Information Processing Systems 30 (NIPS 2017) (2017)

  33. Li, X., Yu, L., Chen, H., Fu, C.W., **ng, L., Heng, P.A.: Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans. Neural Netw. Learn. Syst. 32(2), 523–534 (2021)

    Article  Google Scholar 

  34. Verma, V., et al.: Interpolation consistency training for semi-supervised learning. Neural Netw. 145, 90–106 (2022)

    Article  Google Scholar 

  35. Chen, X.K., Yuan, Y.H., Zeng, G., Wang, J.D., S.O.C. IEEE Comp: Semi-supervised semantic segmentation with cross pseudo supervision. In: Presented at the 2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

  36. Luo, X., et al.: Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In: Presented at the Medical Image Computing And Computer Assisted Intervention-MICCAI 2021 (2021)

  37. Yu, L.Q., Wang, S.J., Li, X.M., Fu, C.W., Heng, P.A.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: Presented at the Medical Image Computing and Computer Assisted Intervention-MICCAI 2019, PT II (2019)

  38. Huang, W., et al.: Semi-supervised neuron segmentation via reinforced consistency learning. IEEE Trans. Med. Imaging 41(11), 3016–3028 (2022)

    Article  Google Scholar 

  39. Hung, W.-C., Tsai, Y.-H., Liou, Y.-T., Lin, Y.-Y., Yang, M.-H.: Adversarial Learning for Semi-supervised Semantic Segmentation

  40. Zhang, Y., Yang, L., Chen, J., Fredericksen, M., Hughes, D.P., Chen, D.Z.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: Presented at the Medical Image Computing and Computer Assisted Intervention-MICCAI 2017 (2017)

  41. Peiris, H., Chen, Z.L., Egan, G., Harandi, M.: Duo-SegNet: adversarial dual-views for semi-supervised medical image segmentation. In: Presented at the Medical Image Computing And Computer Assisted Intervention-MICCAI 2021, PT II (2021)

  42. Hou, J.Y., Ding, X.J., Deng, J.D., Soc, I.C.: Semi-supervised semantic segmentation of vessel images using leaking perturbations. In: Presented at the 2022 IEEE Winter Conference on Applications of Computer Vision (WACV 2022) (2022)

  43. Lei, T., Zhang, D., Du, X., Wang, X., Wan, Y., Nandi, A.K.: Semi-supervised medical image segmentation using adversarial consistency learning and dynamic convolution network. IEEE Trans. Med. Imaging 42(5), 1265–1277 (2023)

    Article  Google Scholar 

  44. Zhao, Z., Zhou, F., Xu, K., Zeng, Z., Guan, C., Zhou, S.K.: LE-UDA: label-efficient unsupervised domain adaptation for medical image segmentation. IEEE Trans. Med. Imaging 42(3), 633–646 (2023)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Marine Electrical Engineering, Dalian Maritime University, Dalian, China

    Huipu Xu, Pengfei Tong & Meixiang Zhang

Authors
  1. Huipu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengfei Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meixiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xu has control of the experimental programme, Tong conducts the experiments and writes the paper, and Zhang collaborates in writing and reviewing the paper.

Corresponding author

Correspondence to Huipu Xu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

The sonar dataset used in this paper is available from the corresponding author on reasonable request. The ISIC2016 dataset is available for download in https://challenge.isic-archive.com/data/.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Tong, P. & Zhang, M. Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation. SIViP 18, 4515–4524 (2024). https://doi.org/10.1007/s11760-024-03091-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03091-7

Keywords

Search

Navigation