SpringerLink
Log in

A Diffusion Equation for Improving the Robustness of Deep Learning Speckle Removal Model

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Speckle removal aims to smooth noise while preserving image boundaries and texture information. In recent years, speckle removal models based on deep learning methods have attracted a lot of attention. However, it was found that these models are less robust to adversarial attacks. The adversarial attack makes the image recovery of deep learning methods significantly less effective when the speckle noise distribution is almost unchanged. In purpose of addressing the above problem, we propose a diffusion equation-based speckle removal model that can improve the robustness of deep learning algorithms in this paper. The model utilizes a deep learning image prior and an image grayscale detection operator together to construct the coefficient function of the diffusion equation. Among them, there is a high possibility that the deep learning image prior is inaccurate or even incorrect, but it will not affect the performance and the properties of the proposed diffusion equation model for noise removal. Moreover, we analyze the robustness of the proposed diffusion equation model in terms of theoretical and numerical properties. Experiments show that our proposed diffusion equation speckle removal model is not affected by adversarial attacks in any way and has stronger robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://github.com/Chengli99999/IP-NDE.

  2. https://tpm-ds.eo.esa.int/smcat/TerraSAR-X/.

References

  1. Aubert, G., Aujol, J.F.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68(4), 925–946 (2008)

    Article  MathSciNet  Google Scholar 

  2. Busse, L., Crimmins, T., Fienup, J.: A model based approach to improve the performance of the geometric filtering speckle reduction algorithm. In: 1995 IEEE Ultrasonics Symposium. Proceedings. An International Symposium, vol. 2, pp. 1353–1356 (1995). https://doi.org/10.1109/ULTSYM.1995.495807

  3. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (sp), pp. 39–57. IEEE (2017)

  4. Catté, F., Lions, P.L., Morel, J.M., Coll, T.: Image selective smoothing and edge detection by nonlinear diffusion. SIAM J. Numer. Anal. 29(1), 182–193 (1992). https://doi.org/10.1137/0729012

    Article  MathSciNet  Google Scholar 

  5. Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2016). https://doi.org/10.1109/TPAMI.2016.2596743

    Article  Google Scholar 

  6. Cheng, L., Guo, Z., Li, Y., **ng, Y.: Two-stream multiplicative heavy-tail noise despeckling network with truncation loss. IEEE Trans. Geosci. Remote Sens. 61, 1–17 (2023). https://doi.org/10.1109/TGRS.2023.3302953

    Article  Google Scholar 

  7. Chierchia, G., Cozzolino, D., Poggi, G., Verdoliva, L.: Sar image despeckling through convolutional neural networks. In: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 5438–5441. IEEE (2017)

  8. Dalsasso, E., Denis, L., Tupin, F.: SAR2SAR: a semi-supervised despeckling algorithm for SAR images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 4321–4329 (2021). https://doi.org/10.1109/JSTARS.2021.3071864

    Article  Google Scholar 

  9. Deledalle, C.A., Denis, L., Tabti, S., Tupin, F.: MuLoG, or how to apply gaussian denoisers to multi-channel SAR speckle reduction? IEEE Trans. Image Process. 26(9), 4389–4403 (2017). https://doi.org/10.1109/TIP.2017.2713946

    Article  MathSciNet  Google Scholar 

  10. Deledalle, C.A., Denis, L., Tupin, F.: Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Trans. Image Process. 18(12), 2661–2672 (2009). https://doi.org/10.1109/TIP.2009.2029593

    Article  MathSciNet  Google Scholar 

  11. Di Martino, G., Poderico, M., Poggi, G., Riccio, D., Verdoliva, L.: Benchmarking framework for SAR despeckling. IEEE Trans. Geosci. Remote Sens. 52(3), 1596–1615 (2014). https://doi.org/10.1109/TGRS.2013.2252907

    Article  Google Scholar 

  12. Dong, Y., Zeng, T.: A convex variational model for restoring blurred images with multiplicative noise. SIAM J. Imag. Sci. 6(3), 1598–1625 (2013)

    Article  MathSciNet  Google Scholar 

  13. Evans, L.C.: Weak convergence methods for nonlinear partial differential equations, vol. 74. American Mathematical Soc. (1990)

  14. Evans, L.C.: Partial differential equations, vol. 19. American Mathematical Society (2022)

  15. Foucart, S., Rauhut, H.: An invitation to compressive sensing. In: A mathematical Introduction to Compressive Sensing, pp. 1–39. Springer (2013)

  16. Gan, Y., Zhang, J., Chang, H.: New splitting algorithms for multiplicative noise removal based on Aubert-Aujol model. Numer. Math. Theory Methods Appl. (2022). https://doi.org/10.4208/nmtma.OA-2021-0134

    Article  MathSciNet  Google Scholar 

  17. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. ar**v preprint ar**v:1412.6572 (2014)

  18. Goodman, J.W.: Some fundamental properties of speckle. JOSA 66(11), 1145–1150 (1976)

    Article  Google Scholar 

  19. **, Z., Yang, X.: Analysis of a new variational model for multiplicative noise removal. J. Math. Anal. Appl. 362(2), 415–426 (2010)

    Article  MathSciNet  Google Scholar 

  20. Ko, J., Lee, S.: SAR image despeckling using continuous attention module. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 15, 3–19 (2022). https://doi.org/10.1109/JSTARS.2021.3132027

    Article  Google Scholar 

  21. Kuan, D.T., Sawchuk, A.A., Strand, T.C., Chavel, P.: Adaptive noise smoothing filter for images with signal-dependent noise. IEEE Trans. Pattern Anal. Mach. Intell. 2, 165–177 (1985). https://doi.org/10.1109/TPAMI.1985.4767641

    Article  Google Scholar 

  22. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: Artificial Intelligence Safety and Security, pp. 99–112. Chapman and Hall/CRC (2018)

  23. Ladyzhenskaia, O.A., Solonnikov, V.A., Ural’tseva, N.N.: Linear and quasi-linear equations of parabolic type, vol. 23. American Mathematical Soc. (1988)

  24. Lee, J.S.: Digital image enhancement and noise filtering by use of local statistics. IEEE Trans. Pattern Anal. Mach. Intell. 2, 165–168 (1980). https://doi.org/10.1109/TPAMI.1980.4766994

    Article  Google Scholar 

  25. Lions, J.L.: Optimal Control of Systems Governed by Partial Differential Equations, vol. 170, pp. 100–108, Springer, Berlin (1971).

  26. Lopes, A., Nezry, E., Touzi, R., Laur, H.: Maximum a posteriori speckle filtering and first order texture models in sar images. In: 10th Annual International Symposium on Geoscience and Remote Sensing, pp. 2409–2412 (1990). https://doi.org/10.1109/IGARSS.1990.689026

  27. Lv, Y.: Total generalized variation denoising of speckled images using a primal-dual algorithm. J. Appl. Math. Comput. 62(1–2), 489–509 (2020)

    Article  MathSciNet  Google Scholar 

  28. Ma, X., Wang, C., Yin, Z., Wu, P.: Sar image despeckling by noisy reference-based deep learning method. IEEE Trans. Geosci. Remote Sens. 58(12), 8807–8818 (2020). https://doi.org/10.1109/TGRS.2020.2990978

    Article  Google Scholar 

  29. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. ar**v preprint ar**v:1706.06083 (2017)

  30. Majee, S., Ray, R.K., Majee, A.K.: A new non-linear hyperbolic-parabolic coupled PDE model for image despeckling. IEEE Trans. Image Process. 31, 1963–1977 (2022). https://doi.org/10.1109/TIP.2022.3149230

    Article  Google Scholar 

  31. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  Google Scholar 

  32. Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574–2582 (2016)

  33. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990). https://doi.org/10.1109/34.56205

    Article  Google Scholar 

  34. Ramos-Llordén, G., Vegas-Sánchez-Ferrero, G., Martin-Fernandez, M., Alberola-López, C., Aja-Fernández, S.: Anisotropic diffusion filter with memory based on speckle statistics for ultrasound images. IEEE Trans. Image Process. 24(1), 345–358 (2014)

    Article  MathSciNet  Google Scholar 

  35. Rudin, L., Lions, P.L., Osher, S.: Multiplicative Denoising and Deblurring: Theory and Algorithms, pp. 103–119. Springer, New York (2003). https://doi.org/10.1007/0-387-21810-6_6

    Book  Google Scholar 

  36. Shan, X., Sun, J., Guo, Z.: Multiplicative noise removal based on the smooth diffusion equation. J. Math. Imaging Vis. 61, 763–779 (2019)

    Article  MathSciNet  Google Scholar 

  37. Shan, X., Sun, J., Guo, Z., Yao, W., Zhou, Z.: Fractional-order diffusion model for multiplicative noise removal in texture-rich images and its fast explicit diffusion solving. BIT Numer. Math. (2022). https://doi.org/10.1007/s10543-022-00913-3

    Article  MathSciNet  Google Scholar 

  38. Shen, H., Zhou, C., Li, J., Yuan, Q.: Sar image despeckling employing a recursive deep CNN prior. IEEE Trans. Geosci. Remote Sens. 59(1), 273–286 (2021). https://doi.org/10.1109/TGRS.2020.2993319

    Article  Google Scholar 

  39. Shi, J., Osher, S.: A nonlinear inverse scale space method for a convex multiplicative noise model. SIAM J. Imaging Sci. 1(3), 294–321 (2008). https://doi.org/10.1137/070689954

    Article  MathSciNet  Google Scholar 

  40. Simon, J.: Compact sets in the space l p (o, t; b). Ann. Mat. 146, 65–96 (1986)

    Article  Google Scholar 

  41. Su, D., Zhang, H., Chen, H., Yi, J., Chen, P.Y., Gao, Y.: Is robustness the cost of accuracy?–a comprehensive study on the robustness of 18 deep image classification models. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 631–648 (2018)

  42. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations, ICLR 2014 (2014)

  43. Tur, M., Chin, K.C., Goodman, J.W.: When is speckle noise multiplicative? Appl. Opt. 21(7), 1157–1159 (1982)

    Article  Google Scholar 

  44. Vitale, S., Ferraioli, G., Pascazio, V.: Multi-objective CNN-based algorithm for SAR despeckling. IEEE Trans. Geosci. Remote Sens. 59(11), 9336–9349 (2021). https://doi.org/10.1109/TGRS.2020.3034852

    Article  Google Scholar 

  45. Weickert, J., Romeny, B., Viergever, M.: Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Trans. Image Process. 7(3), 398–410 (1998). https://doi.org/10.1109/83.661190

    Article  Google Scholar 

  46. Xu, B., Cui, Y., Li, Z., Zuo, B., Yang, J., Song, J.: Patch ordering-based SAR image despeckling via transform-domain filtering. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 8(4), 1682–1695 (2015). https://doi.org/10.1109/JSTARS.2014.2375359

    Article  Google Scholar 

  47. Yan, H., Zhang, J., Feng, J., Sugiyama, M., Tan, V.Y.: Towards adversarially robust deep image denoising. ar**v preprint ar**v:2201.04397 (2022)

  48. Yao, W., Guo, Z., Sun, J., Wu, B., Gao, H.: Multiplicative noise removal for texture images based on adaptive anisotropic fractional diffusion equations. SIAM J. Imaging Sci. 12(2), 839–873 (2019)

  49. Yu, Y., Acton, S.T.: Speckle reducing anisotropic diffusion. IEEE Trans. Image Process. 11(11), 1260–1270 (2002)

  50. Zhang, Q., Yuan, Q., Li, J., Yang, Z., Ma, X.: Learning a dilated residual network for SAR image despeckling. Remote Sens. 10(2), 55 (2018). https://doi.org/10.3390/rs10020196

    Article  Google Scholar 

  51. Zhou, Z., Guo, Z., Dong, G., Sun, J., Zhang, D., Wu, B.: A doubly degenerate diffusion model based on the gray level indicator for multiplicative noise removal. IEEE Trans. Image Process. 24(1), 249–260 (2014)

    Article  MathSciNet  Google Scholar 

  52. Zhou, W., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Harbin Institute of Technology, Harbin, 150001, China

    Li Cheng, Yuming **ng, Yao Li & Zhichang Guo

Authors
  1. Li Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuming **ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhichang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

L.C. performed theorem derivations, experiments, and manuscript writing. Y.X. performed the review of the manuscript. Y.L guided neural network adversarial attacks. Z.G. involved in method illustration and manuscript review.

Corresponding author

Correspondence to Yao Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, L., **ng, Y., Li, Y. et al. A Diffusion Equation for Improving the Robustness of Deep Learning Speckle Removal Model. J Math Imaging Vis (2024). https://doi.org/10.1007/s10851-024-01199-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10851-024-01199-6

Keywords

Search

Navigation