SpringerLink
Log in

Image segmentation with boundary-to-pixel direction and magnitude based on watershed and attention mechanism

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

An improved image segmentation algorithm with boundary-to-pixel direction and magnitude (IS-BPDM) is proposed to deal with small regions segmentation while kee** the accuracy of edge segmentation. First, we develop a BPDM network embedded with watershed and attention module and use an adaptive loss function to learn each pixel’s robust and accurate BPDM which is defined as a two-dimensional vector, including direction and magnitude, and pointing from its nearest boundary pixel to itself. Then, we use the learned BPDMs to obtain the refined initial segmented regions by considering the pixels near boundary have shorter magnitude and near root pixels have longer magnitude, meanwhile adjacent pixels in different regions or nearby pixels on both sides of root pixel in same region have opposite directions and nearby pixels in same region have similar directions. Last, we utilize a fast grou** method according to direction similarity to combine these initial segmented regions into final segmentation. The experimental results show that compared with the state-of-the-art segmentation methods, the IS-BPDM approach proposed in this paper achieves better segmentation accuracy and high computational efficiency and outperforms in small regions segmentation on public datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

This article uses public datasets, which are available on their official website.

References

  1. Ghosh, S., Das, N., Das, I., et al.: Understanding deep learning techniques for image segmentation. ACM Comput. Surv. 52(4), 1–35 (2019)

    Article  Google Scholar 

  2. Cheng, D., Tian, F., Liu, L., et al.: Image segmentation based on multi-region multi-scale local binary fitting and Kullback–Leibler divergence[. Signal Image Video Process. 12(5), 895–903 (2018)

    Article  Google Scholar 

  3. Yue, X., Zhang, H.: A multi-level image thresholding approach using Otsu based on the improved invasive weed optimization algorithm. Signal Image Video Process. 14(3), 575–582 (2020)

    Article  Google Scholar 

  4. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  5. Achanta, R., Shaji, A., Smith, K., et al.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  6. Shi, P., Zhong, J., Rampun, A., et al.: A hierarchical pipeline for breast boundary segmentation and calcification detection in mammograms. Comput. Biol. Med. 96, 178 (2018)

    Article  Google Scholar 

  7. Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM T. Graogic. 23(3), 309–314 (2004)

    Article  Google Scholar 

  8. Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 694–699 (1995)

  9. Yu, Y., Fang, C., Liao, Z.: Piecewise flat embedding for image segmentation. IEEE Int. Conf. Comput. Vis., pp. 1368–1376 (2015)

  10. Maninis, K.K., Pont-Tuset, J., Arbelaez, P., et al.: Convolutional oriented boundaries: from image segmentation to high-level tasks. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 819–833 (2017)

    Article  Google Scholar 

  11. Pont-Tuset, J., Arbeláez, P., Barron, J.T., et al.: Multiscale combinatorial grou** for image segmentation and object proposal generation. IEEE Trans. Pattern Anal. Mach. Intell. 39(1), 128–140 (2016)

    Article  Google Scholar 

  12. Hu, P., Caba, F., Wang, O., et al.: Temporally distributed networks for fast video semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 8818–8827 (2020)

  13. Carion, N., Massa, F., Synnaeve, G., et al.: End-to-end object detection with transformers. Eur. Conf. Comput. Vis., pp. 213–229 (2020)

  14. Cheng, H.K., Chung, J., Tai, Y.W., et al.: Cascade PSP: toward class-agnostic and very high-resolution segmentation via global and local refinement. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 8890–8899 (2020)

  15. Huynh, C., Tran, A.T., Luu, K., et al.: Progressive semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 16755–16764 (2021)

  16. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3431–3440 (2015)

  17. Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  18. Mishra, P., Herrmann, I.: GAN meets chemometrics: segmenting spectral images with pixel2pixel image translation with conditional generative adversarial networks. Chemom. Intell. Lab. Syst. 215, 104362 (2021)

    Article  Google Scholar 

  19. Isola, P., Zhu, J.Y., Zhou, T., et al.: Image-to-image translation with conditional adversarial networks. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1125–1134 (2017)

  20. Zhang, X., Chen, Z., Wu, Q., et al.: Fast semantic segmentation for scene perception. IEEE Trans. Industr. Inform. 15(2), 1183–1192 (2018)

    Article  Google Scholar 

  21. Wan, J., Liu, Y., Wei, D., et al.: Super-BPD: super boundary-to-pixel direction for fast image segmentation. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 9253–9262 (2020)

  22. Carlinet, E., Géraud, T.: A comparative review of component tree computation algorithms. IEEE Trans. Image Process. 23(9), 3885–3895 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  23. Salembier, P., Oliveras, A., Garrido, L.: Antiextensive connected operators for image and sequence processing. IEEE Trans. Image Process. 7(4), 555–570 (1998)

    Article  Google Scholar 

  24. Yuan, Y., Zhu, Z., Yu, H., et al.: Watershed-based superpixels with global and local boundary marching. IEEE Trans. Image Process. 29, 7375–7388 (2020)

    Article  MATH  Google Scholar 

  25. Wolf, S., Bailoni, A., Pape, C., et al.: The mutex watershed: efficient, parameter-free image partitioning. Eur. Conf. Comput. Vis., pp. 546–562 (2018)

  26. Bai, M., Urtasun, R.: Deep watershed transform for instance segmentation. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 5221–5229 (2017)

  27. Lin, Y., Diao, Y., Du, Y., et al.: Automatic cell counting for phase-contrast microscopic images based on a combination of Otsu and watershed segmentation method. Microsc. Res. Tech. 85(1), 169–180 (2022)

    Article  Google Scholar 

  28. Guo, Q., Wang, Y., Yang, S., et al.: A method of blasted rock image segmentation based on improved watershed algorithm. Sci. Rep. 12(1), 1–21 (2022)

    Google Scholar 

  29. Corbetta, M., Shulman, G.L.: Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3(3), 201–215 (2002)

    Article  Google Scholar 

  30. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 7132–7141 (2018)

  31. Woo, S., Park, J., Lee, J.Y., et al.: Cbam: convolutional block attention module. Eur. Conf. Comput. Vis., pp. 3–19 (2018)

  32. Mottaghi, R., Chen, X., Liu, X., et al.: The role of context for object detection and semantic segmentation in the wild. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 891–898 (2014)

  33. Arbelaez, P., Maire, M., Fowlkes, C., et al.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2010)

    Article  Google Scholar 

  34. Cordts, M., Omran, M., Ramos, S., et al.: The cityscapes dataset for semantic urban scene understanding. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3213–3223 (2016)

  35. Kingma D., Ba J.: Adam: a method for stochastic optimization. ar**v preprint ar**v:1412.6980, (2014)

Download references

Acknowledgements

This work was supported by Foundation of Hubei Key Laboratory of Metallurgical Industry Process System Science (No.Y202008), National Natural Science Foundation of China (No.61671338).

Author information

Authors and Affiliations

  1. College of Science, Wuhan University of Science and Technology, Wuhan, 430081, China

    Hongyang Xu, Yuanxiu **ng & Wenbo Wang

  2. Hubei Province Key Laboratory of Systems Science in Metallurgical Process, Wuhan, 430081, China

    Hongyang Xu, Yuanxiu **ng & Wenbo Wang

Authors
  1. Hongyang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanxiu **ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HX and YX proposed the algorithm and wrote the main manuscript, and WW participated in the design of the BPDM network. All authors reviewed the manuscript.

Corresponding author

Correspondence to Yuanxiu **ng.

Ethics declarations

Ethical approval

This article does not involve ethical issues.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., **ng, Y. & Wang, W. Image segmentation with boundary-to-pixel direction and magnitude based on watershed and attention mechanism. SIViP 17, 1695–1703 (2023). https://doi.org/10.1007/s11760-022-02380-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02380-3

Keywords

Search

Navigation