SpringerLink
Log in

Diffusion–Shock Inpainting

  • Conference paper
  • First Online:
Scale Space and Variational Methods in Computer Vision (SSVM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14009))

  • 1078 Accesses

Abstract

We propose diffusion–shock (DS) inpainting as a hitherto unexplored integrodifferential equation for filling in missing structures in images. It combines two carefully chosen components that have proven their usefulness in different applications: homogeneous diffusion inpainting and coherence-enhancing shock filtering. DS inpainting enjoys the complementary synergy of its building blocks: It offers a high degree of anisotropy along an eigendirection of the structure tensor. This enables it to connect interrupted structures over large distances. Moreover, it benefits from the sharp edge structure generated by the shock filter, and it exploits the efficient filling-in effect of homogeneous diffusion. The second order equation that underlies DS inpainting inherits a continuous maximum–minimum principle from its constituents. In contrast to other attractive second order inpainting equations such as edge-enhancing anisotropic diffusion, we can guarantee this property also for the proposed discrete algorithm. Our experiments show a performance that is comparable to or better than many linear or nonlinear, isotropic or anisotropic processes of second or fourth order. They include homogeneous diffusion, biharmonic interpolation, TV inpainting, edge-enhancing anisotropic diffusion, the methods of Tschumperlé and of Bornemann and März, Cahn–Hilliard inpainting, and Euler’s elastica.

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 741215, ERC Advanced Grant INCOVID).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Thailand)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 93.08
Price includes VAT (Thailand)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 109.99
Price excludes VAT (Thailand)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alvarez, L., Mazorra, L.: Signal and image restoration using shock filters and anisotropic diffusion. SIAM J. Numer. Anal. 31, 590–605 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bertozzi, A.L., Esedoglu, S., Gillette, A.: Inpainting of binary images using the Cahn-Hilliard equation. IEEE Trans. Image Process. 16(1), 285–291 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. Math. Imaging Vis. 28(3), 259–278 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  4. Brockett, R.W., Maragos, P.: Evolution equations for continuous-scale morphology. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, San Francisco, CA, vol. 3, pp. 125–128 (1992)

    Google Scholar 

  5. Burger, M., He, L., Schönlieb, C.: Inpainting of binary images using the Cahn-Hilliard equation. SIAM J. Imag. Sci. 2, 1129–11671 (2009)

    Article  MATH  Google Scholar 

  6. Carlsson, S.: Sketch based coding of grey level images. Signal Process. 15, 57–83 (1988)

    Article  Google Scholar 

  7. Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6(2), 298–311 (1997)

    Article  Google Scholar 

  8. Duchon, J.: Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces. RAIRO Analyse Numérique 10, 5–12 (1976)

    Article  MathSciNet  Google Scholar 

  9. Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, New York (1973)

    MATH  Google Scholar 

  10. Efros, A.A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proceedings of Seventh International Conference on Computer Vision, Kerkyra, Greece, vol. 2, pp. 1033–1038. IEEE Computer Society Press (1999)

    Google Scholar 

  11. Förstner, W., Gülch, E.: A fast operator for detection and precise location of distinct points, corners and centres of circular features. In: Proceedings ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data, Interlaken, Switzerland, pp. 281–305 (1987)

    Google Scholar 

  12. Iijima, T.: Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bull. Electrotech. Lab. 26, 368–388 (1962). (in Japanese)

    Google Scholar 

  13. Kämper, N., Weickert, J.: Domain decomposition algorithms for real-time homogeneous diffusion inpainting in 4K. In: Proceedings of 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, Singapore, pp. 1680–1684 (2022)

    Google Scholar 

  14. Kang, S., Tai, X.C., Zhu, W.: Survey of fast algorithms for Euler’s elastica-based image segmentation. In: Kimmel, R., Tai, X.C. (eds.) Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, Handbook of Numerical Analysis, vol. 20, pp. 533–552. Elsevier (2019)

    Google Scholar 

  15. Kornprobst, P., Deriche, R., Aubert, G.: Image coupling, restoration and enhancement via PDEs. In: Proceedings of 1997 IEEE International Conference on Image Processing, Washington, DC, vol. 4, pp. 458–461 (1997)

    Google Scholar 

  16. Kramer, H.P., Bruckner, J.B.: Iterations of a non-linear transformation for enhancement of digital images. Pattern Recogn. 7, 53–58 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  17. Mainberger, M., et al.: Optimising spatial and tonal data for homogeneous diffusion inpainting. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 26–37. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24785-9_3

    Chapter  Google Scholar 

  18. Masnou, S., Morel, J.M.: Level lines based disocclusion. In: Proceedings of 1998 IEEE International Conference on Image Processing, Chicago, IL, vol. 3, pp. 259–263 (1998)

    Google Scholar 

  19. Mumford, D.: Elastica and computer vision. In: Bajaj, C.L. (ed.) Algebraic Geometry and its Applications, vol. 5681, pp. 491–506. Springer, New York (1994). https://doi.org/10.1007/978-1-4612-2628-4_31

    Chapter  MATH  Google Scholar 

  20. Novak, A., Reinić, N.: Shock filter as the classifier for image inpainting problem using the Cahn–Hilliard equation. Comput. Math. Appl. 123, 105–114 (2022)

    MathSciNet  MATH  Google Scholar 

  21. Osher, S., Rudin, L.I.: Feature-oriented image enhancement using shock filters. SIAM J. Numer. Anal. 27, 919–940 (1990)

    Article  MATH  Google Scholar 

  22. Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)

    Article  Google Scholar 

  23. Rouy, E., Tourin, A.: A viscosity solutions approach to shape-from-shading. SIAM J. Numer. Anal. 29(3), 867–884 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  24. Schmaltz, C., Peter, P., Mainberger, M., Ebel, F., Weickert, J., Bruhn, A.: Understanding, optimising, and extending data compression with anisotropic diffusion. Int. J. Comput. Vision 108(3), 222–240 (2014)

    Article  MathSciNet  Google Scholar 

  25. Schrader, K., Alt, T., Weickert, J., Ertel, M.: CNN-based Euler’s elastica inpainting with deep energy and deep image prior. In: 10th European Workshop on Visual Information Processing (EUVIP), Lisbon (2022)

    Google Scholar 

  26. Shen, J., Chan, T.F.: Mathematical models for local non-texture inpaintings. SIAM J. Numer. Anal. 62(3), 1019–1043 (2002)

    MATH  Google Scholar 

  27. Soille, P.: Morphological Image Analysis, 2nd edn. Springer, Berlin (2004). https://doi.org/10.1007/978-3-662-05088-0

    Book  MATH  Google Scholar 

  28. Tschumperlé, D.: Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE’s. Int. J. Comput. Vision 68(1), 65–82 (2006)

    Article  MATH  Google Scholar 

  29. van den Boomgaard, R.: Decomposition of the Kuwahara-Nagao operator in terms of linear smoothing and morphological sharpening. In: Talbot, H., Beare, R. (eds.) Mathematical Morphology: Proceedings of Sixth International Symposium, Sydney, Australia, pp. 283–292. CSIRO Publishing (2002)

    Google Scholar 

  30. Weickert, J.: Coherence-enhancing shock filters. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 1–8. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45243-0_1

    Chapter  Google Scholar 

  31. Weickert, J.: Mathematische Bildverarbeitung mit Ideen aus der Natur. Mitteilungen der DMV 20, 80–92 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  32. Weickert, J., Welk, M.: Tensor field interpolation with PDEs. In: Weickert, J., Hagen, H. (eds.) Visualization and Processing of Tensor Fields, pp. 315–325. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-31272-2_19

    Chapter  Google Scholar 

  33. Welk, M., Weickert, J.: PDE evolutions for M-smoothers in one, two, and three dimensions. J. Math. Imaging Vis. 63, 157–185 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  34. Welk, M., Weickert, J., Galić, I.: Theoretical foundations for spatially discrete 1-D shock filtering. Image Vis. Comput. 25(4), 455–463 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Karl Schrader for providing us with the images and results from his publication [25].

Author information

Authors and Affiliations

  1. Mathematical Image Analysis Group, Department of Mathematics and Computer Science, E1.7, Saarland University, 66123, Saarbrücken, Germany

    Kristina Schaefer & Joachim Weickert

Authors
  1. Kristina Schaefer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristina Schaefer .

Editor information

Editors and Affiliations

  1. CNRS, Université Côte d'Azur, Sophia-Antipolis, France

    Luca Calatroni

  2. University of Insubria, Como, Italy

    Marco Donatelli

  3. University of Bologna, Bologna, Italy

    Serena Morigi

  4. University of Modena and Reggio Emilia, Modena, Italy

    Marco Prato

  5. University of Genova, Genova, Italy

    Matteo Santacesaria

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schaefer, K., Weickert, J. (2023). Diffusion–Shock Inpainting. In: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2023. Lecture Notes in Computer Science, vol 14009. Springer, Cham. https://doi.org/10.1007/978-3-031-31975-4_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-31975-4_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-31974-7

  • Online ISBN: 978-3-031-31975-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation