SpringerLink
Log in

Simultaneous denoising and super resolution of document images

  • Published:
Sādhanā Aims and scope Submit manuscript

Abstract

In this paper, we propose a unified approach for denoising and super-resolution of document images. The approach is a one shot unpaired technique where a single unpaired example is used as reference for training a SinGAN (Shaham et al., in: Proceedings of the IEEE/CVF international conference on computer vision, 2019) model. The training is carried out in 2 steps. First we use a clean reference image to train a SinGAN to learn the characteristics of the clean image. Then we perform super resolution and denoising of given test image using another SinGAN. Our unique formulation of the loss function helps in this task by prompting the generated images to have characteristics similar to the reference clean image. We conduct experiments on publicly available datasets (Kaggle Dirty Documents Images and DIBCO) and obtain promising results. We also evaluate the performance of our model for OCR and obtain a higher recognition rate compared to competing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

References

  1. Likforman-Sulem L, Darbon J and Smith E H B 2011 Enhancement of historical printed document images by combining total variation regularization and non-local means filtering. Image and vision computing 29(5): 351–363

    Article  Google Scholar 

  2. Lu S, Chen B M and Ko C C 2006 A partition approach for the restoration of camera images of planar and curled document. Image and Vision Computing 24(8): 837–848

    Article  Google Scholar 

  3. Mitianoudis N and Papamarkos N 2015 Document image binarization using local features and gaussian mixture modeling. Image and Vision Computing 38: 33–51

    Article  Google Scholar 

  4. Shocher A, Cohen N and Irani M 2018 “Zero-shot” super-resolution using deep internal learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3118–3126

  5. Shaham T R, Dekel T and Michaeli T 2019 Singan: Learning a generative model from a single natural image. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4570–4580

  6. Walha R, Drira F, Lebourgeois F, Alimi A M and Garcia C 2016 Resolution enhancement of textual images: a survey of single image-based methods. IET Image Processing 10(4): 325–337

    Article  Google Scholar 

  7. Buades A, Coll B and Morel J-M 2005 A review of image denoising algorithms, with a new one. Multiscale modeling & simulation 4(2): 490–530

    Article  MathSciNet  Google Scholar 

  8. Awad A 2019 Denoising images corrupted with impulse, gaussian, or a mixture of impulse and gaussian noise. Engineering Science and Technology, an International Journal 22(3): 746–753

    Article  Google Scholar 

  9. Ling BW-K, Ho CY-F, Dai Q and Reiss J D 2014 Reduction of quantization noise via periodic code for oversampled input signals and the corresponding optimal code design. Digital Signal Processing 24: 209–222

    Article  Google Scholar 

  10. Rajagopal A, Hamilton R B and Scalzo F 2016 Noise reduction in intracranial pressure signal using causal shape manifolds. Biomedical signal processing and control 28: 19–26

    Article  Google Scholar 

  11. Ilesanmi A E, Idowu O P, Chaumrattanakul U and Makhanov S S 2021 Multiscale hybrid algorithm for pre-processing of ultrasound images. Biomedical Signal Processing and Control 66: 102396

    Article  Google Scholar 

  12. Huang T S 1972 Stability of two-dimensional recursive filters. IEEE Transactions on Audio and Electroacoustics 20: 158–163

    Article  MathSciNet  Google Scholar 

  13. Zhao H and Zheng Z 2016 Bias-compensated affine-projection-like algorithms with noisy input. Electronics Letters 52(9): 712–714

    Article  Google Scholar 

  14. Ilesanmi A E and Ilesanmi T O 2021 Methods for image denoising using convolutional neural network: A review. Complex & Intelligent Systems 7(5): 2179–2198

    Article  Google Scholar 

  15. Lucas A, Iliadis M, Molina R and Katsaggelos A K 2018 Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEE Signal Processing Magazine 35(1): 20–36

    Article  Google Scholar 

  16. Koesten L, Simperl E, Blount T, Kacprzak E and Tennison J 2020 Everything you always wanted to know about a dataset: Studies in data summarisation. International Journal of Human-Computer Studies 135: 102367

    Article  Google Scholar 

  17. **ao-Jiao Mao, Chunhua Shen and Yang B Y 2016 Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proc. Advances in Neural Information Processing Systems (NIPS)

  18. Zhang K, Zuo W, Chen Y, Meng D and Zhang L 2017 Beyond a Gaussian denoiser: Residual learning of deep cnn for image denoising.IEEE Transactions on Image Processing

  19. Shi W, Jiang F, Zhang S, Wang R, Zhao D and Zhou H 2019 Hierarchical residual learning for image denoising. Signal Processing: Image Communication 76: 243–251

    Google Scholar 

  20. Li X, **ao J, Zhou Y, Yuanzheng Y, Nianzu L, Wang X, Wang S and Gao S 2020 Detail retaining convolutional neural network for image denoising. Journal of Visual Communication and Image Representation, 71

  21. Guo B., Song K., Dong H., Yan Y., Tu Z., and Zhu L. 2020. NERnet: Noise estimation and removal network for image denoising. Journal of Visual Communication and Image Representation, 71

  22. Ronneberger O, Fischer P and Brox T 2015 U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer, LNCS 9351: 234–241

    Google Scholar 

  23. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y 2020 Generative adversarial networks. Communications of the ACM 63(11): 139–144

    Article  MathSciNet  Google Scholar 

  24. Lyu Q, Guo M and Pei Z 2020 DeGAN: Mixed noise removal via generative adversarial networks. Applied Soft Computing 95: 106478

    Article  Google Scholar 

  25. Jiang Q, Chen Y, Wang G and Ji T 2020 A novel deep neural network for noise removal from underwater image. Signal Processing: Image Communication 87: 115921

    Google Scholar 

  26. Kim H-J and Lee D 2020 Image denoising with conditional generative adversarial networks (cgan) in low dose chest images. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 954: 161914

    Article  Google Scholar 

  27. Majumdar A 2018 Blind denoising autoencoder. IEEE transactions on neural networks and learning systems 30(1): 312–317

    Article  Google Scholar 

  28. Langarica S and Núñez F 2023 Contrastive blind denoising autoencoder for real time denoising of industrial iot sensor data. Engineering Applications of Artificial Intelligence 120: 105838

    Article  Google Scholar 

  29. Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M and Aila T 2018 Noise2Noise: Learning image restoration without clean data. In: Proc. International Conference on Machine Learning

  30. Pang T, Zheng H, Quan Y and Ji H 2021 Recorrupted-to-recorrupted: unsupervised deep learning for image denoising. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 2043–2052

  31. Krull A, Buchholz T-O and Jug F 2019 Noise2Void-learning denoising from single noisy images. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 2129–2137

  32. Krull A, Vičar T, Prakash M, Lalit M and Jug F 2020 Probabilistic noise2void: Unsupervised content-aware denoising. Frontiers in Computer Science 2: 5

    Article  Google Scholar 

  33. Sharma M, Ray A, Chaudhury S and Lall B 2017 A noise-resilient super-resolution framework to boost ocr performance. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 1: 466–471. IEEE

  34. Xu X, Sun D, Pan J, Zhang Y, Pfister H and Yang M-H 2017 Learning to super-resolve blurry face and text images. In: Proceedings of the IEEE International Conference on Computer Vision, pages 251–260

  35. Salismans T, Goodfellow I, Zaremba W, Cheung V, Radford A and Chen X 2016 Improved techniques for training GANs. In: Advances in Neural Information Processing Systems (NIPS)

  36. Ray A, Sharma M, Upadhyay A, Makwana M, Chaudhury S, Trivedi A, Singh A and Saini A 2019 An end-to-end trainable framework for joint optimization of document enhancement and recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 59–64. IEEE

  37. Mao X, Shen C and Yang Y-B 2016 Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in Neural Information Processing Systems (NIPS)

  38. Haris M, Shakhnarovich G and Ukita N 2018 Deep back-projection networks for super-resolution. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1664–1673

  39. Peng X and Wang C 2020 Building super-resolution image generator for ocr accuracy improvement. In: International Workshop on Document Analysis Systems, pages 145–160. Springer

  40. Wang Z, Simoncelli E and Bovik A 2003 Multiscale structural similarity for image quality assessment. In: Asilomar Conference on Signals, Systems and Computers, pages 1298 – 1402

  41. Mao X, Li Q, **e H, Lau R, Wang Z and Smolley S P 2017 Least squares generative adversarial networks. In: International Conference on Computer Vision

  42. Jolicoeur-Martineau A 2019 The relativistic discriminator: a key element missing from standard GAN. In: International Conference on Learning Represenations (ICLR)

  43. Yamanaka J, Kuwashima S and Kurita T 2017 Fast and accurate image super resolution by deep cnn with skip connection and network in network. In: International Conference on Neural Information Processing (NIPS), pages 217–225

  44. Quan Y, Yang J, Chen Y, Xu Y and Ji H 2020 Collaborative deep learning for super-resolving blurry text images. IEEE Transactions on Computational Imaging 6: 778–790

    Article  MathSciNet  Google Scholar 

  45. Johnson J, Alahi A and Fei-Fei L 2016 Perceptual losses for real-time style transfer and super-resolution. In: European conference on Computer Vision (ECCV), pages 694–711. Springer

  46. Arjovsky M, Chintala S and Bottou L 2017 Wassertein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML)

  47. Adler J and Lunz S 2018 Banach wasserstein GAN.In: Proc. Advances in Neural Information Processing Systems (NeurIPS)

  48. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V and Courville A. 2017. Improved Training of Wassertein GANs. In Proceedings Neural Information Processing Systems (NeurIPS)

  49. Hradiš M, Kotera J, Zemcık P and Šroubek F 2015 Convolutional neural networks for direct text deblurring. In: Proceedings of British Machine Vision Conference (BMVC), volume 10

  50. Frank A 2010 UCI machine learning repository. Irvine, CA: University of california, school of information and computer science. http://archive.ics.uci.edu/ml

  51. Dua D and Graff C 2017 UCI machine learning repository

  52. Do T-H, Ramos Terrades O and Tabbone S 2019 DSD: document sparse-based denoising algorithm. Pattern Analysis and Applications 22(1): 177–186

    Article  MathSciNet  Google Scholar 

  53. Habibunnisha N, Sivamani K, Seetharaman R and Nedumaran D 2019 Reduction of noises from degraded document images using image enhancement techniques.In: 2019 Third International Conference on Inventive Systems and Control (ICISC), pages 522–525. IEEE

  54. Dumpala V, Kurupathi S R, Bukhari S S Dengel A 2019 Removal of historical document degradations using conditional gans. In: Proceedings of International Conference on Pattern Recognition Applications and Methods (ICPRAM), pages 145–154

  55. Souibgui M A and Kessentini Y 2022 DE-GAN: a conditional generative adversarial network for document enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence. 44(3): 1180–1191

    Article  Google Scholar 

Download references

Acknowledgements

Authors would like to thank Dr. Yashaswi Verma for insightful comments on the paper.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, IIT Jodhpur, Jodhpur, India

    Divya Srivastava & Gaurav Harit

Authors
  1. Divya Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaurav Harit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaurav Harit.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srivastava, D., Harit, G. Simultaneous denoising and super resolution of document images. Sādhanā 49, 35 (2024). https://doi.org/10.1007/s12046-023-02326-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12046-023-02326-6

Keywords

Search

Navigation