Abstract
Numerous image restoration algorithms have been proposed in the last several decades. These algorithms usually optimize an objective function consisting of an \({\ell _2}\) norm based fidelity and a regularization term, whose optimality could be justified from the view of maximum a posteriori estimation with an assumption that the noise is Gaussian. However, it is known that the \({\ell _2}\) norm based fidelity is very sensitive to gross errors that may appear in the observation. Since real-world image restoration tasks are usually hindered by abnormal pixels, impulsive noise, and other heavy-tailed noise, the utility of these traditional algorithms is limited. Although some robust algorithms have been proposed by replacing the \({\ell _2}\) norm based fidelity with a robust one, they are designed for specific restoration tasks (e.g., multi-frame super-resolution) with a fixed image prior (e.g., the total-variation) and have not provided a principled way to justify the choice of a robust fidelity term. Currently designing a robust algorithm for general image restoration tasks is still an open problem. This paper studies the problem of robust image restoration in both theoretical and algorithmic manners. In the theoretical part, we point out that Huber function based fidelity could be justified from the pespective of minimax estimation, which facilities the choice of the robust fidelity term. In the algorithmic part, we first propose an adaptive approach to set the threshold of the Huber function, and then we derive an efficient and flexible method to solve the proposed robust formulation of the image restoration problem, which enables the proposed algorithm to incorporate various image priors. Experiments have demonstrated the robustness of the proposed algorithm and its utility in real-world image restoration tasks.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Figa_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11263-024-02163-y/MediaObjects/11263_2024_2163_Fig7_HTML.png)
Data Availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Abdelhamed, A., Brubaker, M. A., & Brown, M. S. (2019). Noise flow: Noise modeling with conditional normalizing flows. In IEEE international conference on computer vision (pp. 3165–3173).
Abdelhamed, A., Lin, S., & Brown, M. S. (2018). A high-quality denoising dataset for smartphone cameras. In IEEE conference on computer vision and pattern recognition (pp. 1692–1700).
Abuolaim, A., & Brown, M. S. (2020). Defocus deblurring using dual-pixel data. In European conference on computer vision (pp. 111–126).
Aubert, G., & Aujol, J. F. (2008). A variational approach to removing multiplicative noise. SIAM Journal on Applied Mathematics, 68(4), 925–946.
Bertsekas, D. P. (2014). Constrained optimization and Lagrange multiplier methods. Academic Press.
Buades, A., Coll, B., & Morel, J. M. (2005). A review of image denoising algorithms, with a new one. Multiscale Modeling and Simulation, 4(2), 490–530.
Carrillo, R. E., & Barner, K. E. (2013). Lorentzian iterative hard thresholding: Robust compressed sensing with prior information. IEEE Transactions on Signal Processing, 61(19), 4822–4833.
Carrillo, R. E., Barner, K. E., & Aysal, T. C. (2010). Robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. IEEE Journal of Selected Topics in Signal Processing, 4(2), 392–408.
Chan, S. H., Wang, X., & Elgendy, O. A. (2016). Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 3(1), 84–98.
Chang, S. G., Yu, B., & Vetterli, M. (2000). Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9), 1532–1546.
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.
Donoho, D. L., & Johnstone, J. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81(3), 425–455.
Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004). Fast and robust multiframe super-resolution. IEEE Transactions on Image Processing, 13(10), 1327–1344.
Granas, A., & Dugundji, J. (2003). Fixed point theory (Vol. 14). Springer.
Huang, J. B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In IEEE conference on computer vision and pattern recognition (pp. 5197–5206).
Huber, P. J. (1992). Robust estimation of a location parameter. In S. Kotz & N. L. Johnson (Eds.), Breakthroughs in statistics (pp. 492–518). Springer.
Huber, P. J. (2011). Robust statistics (pp. 1248–1251). Springer.
Jang, G., Lee, W., Son, S., & Lee K. M. (2021). C2N: Practical generative noise modeling for real-world denoising. In IEEE international conference on computer vision (pp. 2350–2359).
Le, T., Chartrand, R., & Asaki, T. J. (2007). A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging and Vision, 27(3), 257–263.
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., & Ma, Y. (2012). Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 171–184.
Liu, X., Chen, L., Wang, W., & Zhao, J. (2018). Robust multi-frame super-resolution based on spatially weighted half-quadratic estimation and adaptive BTV regularization. IEEE Transactions on Image Processing, 27(10), 4971–4986.
Liu, X., & Zhao, J. (2017). Robust multi-frame super-resolution with adaptive norm choice and difference curvature based BTV regularization. In IEEE global conference on signal and information processing (pp. 388–392).
Mairal, J., Elad, M., & Sapiro, G. (2007). Sparse representation for color image restoration. IEEE Transactions on Image Processing, 17(1), 53–69.
Maronna, R. A., Martin, R. D., Yohai, V. J., & Salibin-Barrera, M. (2019). Robust statistics: Theory and methods. Hoboken: Wiley.
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE international conference on computer vision (pp. 416–423).
Meng, D., & De La Torre, F. (2013). Robust matrix factorization with unknown noise. In IEEE international conference on computer vision (pp. 1337–1344).
Patanavijit, V., & Jitapunkul, S. (2007). A Lorentzian stochastic estimation for a robust iterative multiframe super-resolution reconstruction with Lorentzian–Tikhonov regularization. EURASIP Journal on Advances in Signal Processing, 2007, 1–21.
Pham, D. S., & Venkatesh, S. (2013). Efficient algorithms for robust recovery of images from compressed data. IEEE Transactions on Image Processing, 22(12), 4724–4737.
Razavi, S. A., Ollila, E., & Koivunen, V. (2012). Robust greedy algorithms for compressed sensing. In European signal processing conference (pp. 969–973).
Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1–4), 259–268.
Steidl, G., & Teuber, T. (2010). Removing multiplicative noise by Douglas–Rachford splitting methods. Journal of Mathematical Imaging and Vision, 36(2), 168–184.
Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B. (2013). Plug-and-play priors for model based reconstruction. In IEEE global conference on signal and information processing (pp. 945–948).
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
Wei, K., Fu, Y., Zheng, Y., & Yang, J. (2021). Physics-based noise modeling for extreme low-light photography. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11), 8520–8537.
Wen, F., Liu, P., Liu, Y., Qiu, R. C., & Yu, W. (2016). Robust sparse recovery in impulsive noise via \({\ell _p}\)-\({\ell _1}\) optimization. IEEE Transactions on Signal Processing, 65(1), 105–118.
Woo, H., & Yun, S. (2013). Proximal linearized alternating direction method for multiplicative denoising. SIAM Journal on Scientific Computing, 35(2), B336–B358.
Yan, M. (2013). Restoration of images corrupted by impulse noise and mixed gaussian impulse noise using blind inpainting. SIAM Journal on Imaging Sciences, 6(3), 1227–1245.
Yue, Z., Zhao, Q., Zhang, L., & Meng, D. (2020). Dual adversarial network: Toward real-world noise removal and noise generation. In European conference on computer vision (pp. 41–58).
Zeng, X., & Yang, L. (2013). A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization. Digital Signal Processing, 23(1), 98–109.
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 6360–6376.
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017). Learning deep CNN denoiser prior for image restoration. In IEEE conference on computer vision and pattern recognition (pp. 3929–3938).
Zhang, K., Zuo, W., & Zhang, L. (2018). FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9), 4608–4622.
Zhang, Y., Li, D., Law, K. L., Wang, X., Qin, H., & Li, H. (2022). IDR: Self-supervised image denoising via iterative data refinement. In IEEE conference on computer vision and pattern recognition (pp. 2098–2107).
Zhang, Y., Li, D., Shi, X., He, D., Song, K., Wang, X., Qin, H., & Li, H. (2023). KBNet: Kernel basis network for image restoration. ar**v preprint ar**v:2303.02881
Zhao, Q., Meng, D., Xu, Z., Zuo, W., & Zhang, L. (2014). Robust principal component analysis with complex noise. In International conference on machine learning (pp. 55–63).
Zou, Y., & Fu, Y. (2022). Estimating fine-grained noise model via contrastive learning. In IEEE conference on computer vision and pattern recognition (pp. 12682–12691).
Zuo, W., & Lin, Z. (2011). A generalized accelerated proximal gradient approach for total-variation-based image restoration. IEEE Transactions on Image Processing, 20(10), 2748–2759.
Acknowledgements
This work is supported in part by National Natural Science Foundation of China under Grant 62131003.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Zhouchen Lin.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Song, L., Huang, H. Robust Image Restoration with an Adaptive Huber Function Based Fidelity. Int J Comput Vis (2024). https://doi.org/10.1007/s11263-024-02163-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11263-024-02163-y