Log in

Robust Image Restoration with an Adaptive Huber Function Based Fidelity

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Numerous image restoration algorithms have been proposed in the last several decades. These algorithms usually optimize an objective function consisting of an \({\ell _2}\) norm based fidelity and a regularization term, whose optimality could be justified from the view of maximum a posteriori estimation with an assumption that the noise is Gaussian. However, it is known that the \({\ell _2}\) norm based fidelity is very sensitive to gross errors that may appear in the observation. Since real-world image restoration tasks are usually hindered by abnormal pixels, impulsive noise, and other heavy-tailed noise, the utility of these traditional algorithms is limited. Although some robust algorithms have been proposed by replacing the \({\ell _2}\) norm based fidelity with a robust one, they are designed for specific restoration tasks (e.g., multi-frame super-resolution) with a fixed image prior (e.g., the total-variation) and have not provided a principled way to justify the choice of a robust fidelity term. Currently designing a robust algorithm for general image restoration tasks is still an open problem. This paper studies the problem of robust image restoration in both theoretical and algorithmic manners. In the theoretical part, we point out that Huber function based fidelity could be justified from the pespective of minimax estimation, which facilities the choice of the robust fidelity term. In the algorithmic part, we first propose an adaptive approach to set the threshold of the Huber function, and then we derive an efficient and flexible method to solve the proposed robust formulation of the image restoration problem, which enables the proposed algorithm to incorporate various image priors. Experiments have demonstrated the robustness of the proposed algorithm and its utility in real-world image restoration tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Data Availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  • Abdelhamed, A., Brubaker, M. A., & Brown, M. S. (2019). Noise flow: Noise modeling with conditional normalizing flows. In IEEE international conference on computer vision (pp. 3165–3173).

  • Abdelhamed, A., Lin, S., & Brown, M. S. (2018). A high-quality denoising dataset for smartphone cameras. In IEEE conference on computer vision and pattern recognition (pp. 1692–1700).

  • Abuolaim, A., & Brown, M. S. (2020). Defocus deblurring using dual-pixel data. In European conference on computer vision (pp. 111–126).

  • Aubert, G., & Aujol, J. F. (2008). A variational approach to removing multiplicative noise. SIAM Journal on Applied Mathematics, 68(4), 925–946.

    Article  MathSciNet  Google Scholar 

  • Bertsekas, D. P. (2014). Constrained optimization and Lagrange multiplier methods. Academic Press.

    Google Scholar 

  • Buades, A., Coll, B., & Morel, J. M. (2005). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490–530.

    Article  MathSciNet  Google Scholar 

  • Carrillo, R. E., & Barner, K. E. (2013). Lorentzian iterative hard thresholding: Robust compressed sensing with prior information. IEEE Transactions on Signal Processing, 61(19), 4822–4833.

    Article  MathSciNet  Google Scholar 

  • Carrillo, R. E., Barner, K. E., & Aysal, T. C. (2010). Robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. IEEE Journal of Selected Topics in Signal Processing, 4(2), 392–408.

    Article  Google Scholar 

  • Chan, S. H., Wang, X., & Elgendy, O. A. (2016). Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 3(1), 84–98.

    Article  MathSciNet  Google Scholar 

  • Chang, S. G., Yu, B., & Vetterli, M. (2000). Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9), 1532–1546.

    Article  MathSciNet  Google Scholar 

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L., & Johnstone, J. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81(3), 425–455.

    Article  MathSciNet  Google Scholar 

  • Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004). Fast and robust multiframe super-resolution. IEEE Transactions on Image Processing, 13(10), 1327–1344.

    Article  Google Scholar 

  • Granas, A., & Dugundji, J. (2003). Fixed point theory (Vol. 14). Springer.

    Book  Google Scholar 

  • Huang, J. B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In IEEE conference on computer vision and pattern recognition (pp. 5197–5206).

  • Huber, P. J. (1992). Robust estimation of a location parameter. In S. Kotz & N. L. Johnson (Eds.), Breakthroughs in statistics (pp. 492–518). Springer.

    Chapter  Google Scholar 

  • Huber, P. J. (2011). Robust statistics (pp. 1248–1251). Springer.

    Google Scholar 

  • Jang, G., Lee, W., Son, S., & Lee K. M. (2021). C2N: Practical generative noise modeling for real-world denoising. In IEEE international conference on computer vision (pp. 2350–2359).

  • Le, T., Chartrand, R., & Asaki, T. J. (2007). A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging and Vision, 27(3), 257–263.

    Article  MathSciNet  Google Scholar 

  • Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., & Ma, Y. (2012). Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 171–184.

    Article  Google Scholar 

  • Liu, X., Chen, L., Wang, W., & Zhao, J. (2018). Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Transactions on Image Processing, 27(10), 4971–4986.

    Article  MathSciNet  Google Scholar 

  • Liu, X., & Zhao, J. (2017). Robust multi-frame super-resolution with adaptive norm choice and difference curvature based BTV regularization. In IEEE global conference on signal and information processing (pp. 388–392).

  • Mairal, J., Elad, M., & Sapiro, G. (2007). Sparse representation for color image restoration. IEEE Transactions on Image Processing, 17(1), 53–69.

    Article  MathSciNet  Google Scholar 

  • Maronna, R. A., Martin, R. D., Yohai, V. J., & Salibin-Barrera, M. (2019). Robust statistics: Theory and methods. Hoboken: Wiley.

    Google Scholar 

  • Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE international conference on computer vision (pp. 416–423).

  • Meng, D., & De La Torre, F. (2013). Robust matrix factorization with unknown noise. In IEEE international conference on computer vision (pp. 1337–1344).

  • Patanavijit, V., & Jitapunkul, S. (2007). A Lorentzian stochastic estimation for a robust iterative multiframe super-resolution reconstruction with Lorentzian–Tikhonov regularization. EURASIP Journal on Advances in Signal Processing, 2007, 1–21.

    Article  Google Scholar 

  • Pham, D. S., & Venkatesh, S. (2013). Efficient algorithms for robust recovery of images from compressed data. IEEE Transactions on Image Processing, 22(12), 4724–4737.

    Article  MathSciNet  Google Scholar 

  • Razavi, S. A., Ollila, E., & Koivunen, V. (2012). Robust greedy algorithms for compressed sensing. In European signal processing conference (pp. 969–973).

  • Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1–4), 259–268.

    Article  MathSciNet  Google Scholar 

  • Steidl, G., & Teuber, T. (2010). Removing multiplicative noise by Douglas–Rachford splitting methods. Journal of Mathematical Imaging and Vision, 36(2), 168–184.

    Article  MathSciNet  Google Scholar 

  • Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B. (2013). Plug-and-play priors for model based reconstruction. In IEEE global conference on signal and information processing (pp. 945–948).

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wei, K., Fu, Y., Zheng, Y., & Yang, J. (2021). Physics-based noise modeling for extreme low-light photography. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 8520–8537.

    Google Scholar 

  • Wen, F., Liu, P., Liu, Y., Qiu, R. C., & Yu, W. (2016). Robust sparse recovery in impulsive noise via \({\ell _p}\)-\({\ell _1}\) optimization. IEEE Transactions on Signal Processing, 65(1), 105–118.

    Article  Google Scholar 

  • Woo, H., & Yun, S. (2013). Proximal linearized alternating direction method for multiplicative denoising. SIAM Journal on Scientific Computing, 35(2), B336–B358.

    Article  MathSciNet  Google Scholar 

  • Yan, M. (2013). Restoration of images corrupted by impulse noise and mixed gaussian impulse noise using blind inpainting. SIAM Journal on Imaging Sciences, 6(3), 1227–1245.

  • Yue, Z., Zhao, Q., Zhang, L., & Meng, D. (2020). Dual adversarial network: Toward real-world noise removal and noise generation. In European conference on computer vision (pp. 41–58).

  • Zeng, X., & Yang, L. (2013). A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization. Digital Signal Processing, 23(1), 98–109.

  • Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 6360–6376.

    Article  Google Scholar 

  • Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017). Learning deep CNN denoiser prior for image restoration. In IEEE conference on computer vision and pattern recognition (pp. 3929–3938).

  • Zhang, K., Zuo, W., & Zhang, L. (2018). FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9), 4608–4622.

    Article  MathSciNet  Google Scholar 

  • Zhang, Y., Li, D., Law, K. L., Wang, X., Qin, H., & Li, H. (2022). IDR: Self-supervised image denoising via iterative data refinement. In IEEE conference on computer vision and pattern recognition (pp. 2098–2107).

  • Zhang, Y., Li, D., Shi, X., He, D., Song, K., Wang, X., Qin, H., & Li, H. (2023). KBNet: Kernel basis network for image restoration. ar**v preprint ar**v:2303.02881

  • Zhao, Q., Meng, D., Xu, Z., Zuo, W., & Zhang, L. (2014). Robust principal component analysis with complex noise. In International conference on machine learning (pp. 55–63).

  • Zou, Y., & Fu, Y. (2022). Estimating fine-grained noise model via contrastive learning. In IEEE conference on computer vision and pattern recognition (pp. 12682–12691).

  • Zuo, W., & Lin, Z. (2011). A generalized accelerated proximal gradient approach for total-variation-based image restoration. IEEE Transactions on Image Processing, 20(10), 2748–2759.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is supported in part by National Natural Science Foundation of China under Grant 62131003.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hua Huang.

Additional information

Communicated by Zhouchen Lin.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, L., Huang, H. Robust Image Restoration with an Adaptive Huber Function Based Fidelity. Int J Comput Vis (2024). https://doi.org/10.1007/s11263-024-02163-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11263-024-02163-y

Keywords

Navigation