SpringerLink
Log in

Blind Image Deblurring with Unknown Kernel Size and Substantial Noise

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Blind image deblurring (BID) has been extensively studied in computer vision and adjacent fields. Modern methods for BID can be grouped into two categories: single-instance methods that deal with individual instances using statistical inference and numerical optimization, and data-driven methods that train deep-learning models to deblur future instances directly. Data-driven methods can be free from the difficulty in deriving accurate blur models, but are fundamentally limited by the diversity and quality of the training data—collecting sufficiently expressive and realistic training data is a standing challenge. In this paper, we focus on single-instance methods that remain competitive and indispensable. However, most such methods do not prescribe how to deal with unknown kernel size and substantial noise, precluding practical deployment. Indeed, we show that several state-of-the-art (SOTA) single-instance methods are unstable when the kernel size is overspecified, and/or the noise level is high. On the positive side, we propose a practical BID method that is stable against both, the first of its kind. Our method builds on the recent ideas of solving inverse problems by integrating physical models and structured deep neural networks, without extra training data. We introduce several crucial modifications to achieve the desired stability. Extensive empirical tests on standard synthetic datasets, as well as real-world NTIRE2020 and RealBlur datasets, show the superior effectiveness and practicality of our BID method compared to SOTA single-instance as well as data-driven methods. The code of our method is available at https://github.com/sun-umn/Blind-Image-Deblurring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Algorithm 1
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35

Similar content being viewed by others

Data Availability

Part of the code and datasets used during the current study, necessary to interpret, replicate and build upon the findings reported in the article, are available in the Github repository https://github.com/sun-umn/Blind-Image-Deblurring

Notes

  1. Indeed, by Young’s convolution inequality and the fact \({\Vert {\varvec{k}} \Vert _1 = 1}\), .

  2. See also similar ideas for the inverse filtering approach in Cabrelli (1985) and Sun and Donoho (2021).

  3. In particular, if they form a measure-zero set.

  4. The result in Eq. (11) assumes a circular convolution model: \(\varvec{y} = \varvec{a} \circledast \varvec{z}\), but it is well known that the linear convolution can be written as circular convolution by appropriate zero-padding to the two convolving components.

  5. We note in passing that the reason we do not use FBC directly is that it may be misleading: the correspondence ratio as they define it can be larger than 1, so in principle the average approaching 1 does not imply that recovery is good. When checking their code (https://github.com/shizenglin/Measure-and-Control-Spectral-Bias), we find that they actually truncate values greater than 1, which potentially make the metric more misleading.

  6. The existing synthetic BID datasets are too small to support training data-driven methods.

  7. Available at https://webee.technion.ac.il/people/anat.levin/papers/LevinEtalCVPR09Data.rar

  8. Available at http://vllab.ucmerced.edu/wlai24/cvpr16_deblur_study/

  9. LAI16 has 4 trajectories to synthesize non-uniform motion blur also, which we do not consider in this paper. Moreover, it also includes 100 real-world blurry images without groundtruth kernels.

  10. Available at (registration needed to download the dataset): https://competitions.codalab.org/competitions/22233#learn_the_details. We suspect that this is a superset of the REDS (REalistic and Dynamic Scenes) dataset (available at https://seungjunnah.github.io/Datasets/reds.html), at least with the same generation procedure as that of REDS.

  11. Available at: http://cg.postech.ac.kr/research/realblur/

  12. NTIRE2020 is developed for data-driven approaches that require an extensive training set.

  13. Code available at: http://cs.brown.edu/~lbsun/deblur2013/deblur2013iccp.html

  14. Code available at: https://jspan.github.io/projects/dark-channel-deblur/index.html

  15. Code available at: https://www.dropbox.com/s/qmxkkwgnmuwrfoj/code_iccv2017_outlier.zip?dl=0

  16. Code available at: https://github.com/lisiyaoATbnu/low_rank_kernel

  17. Code available at: https://github.com/csdwren/SelfDeblur

  18. In DONG17 the loss consists in applying \(h(z) = z^2/2 - \log {(a+e^{bz^2})}/(2b)\) element-wise to \(\varvec{y} - {\varvec{k}} *{\varvec{x}}\), where \(a, b > 0\) and so that \(h(z) \le 0\). Note that \(h(z) \sim O(z^2)\) as \(z \rightarrow 0\), and h(z) approaches the constant 0 when z is large.

  19. SRN is available at: https://github.com/jiangsutx/SRN-Deblur; DeblurGAN-v2 is available at: https://github.com/VITA-Group/DeblurGANv2; ZHANG20 is available at: https://github.com/HDCVLab/Deblurring-by-Realistic-Blurring.

  20. Available at: https://drive.google.com/file/d/1fHkPiZOvLQSc4HhT8-wA6dh0M4skpTMi/view

References

  • Ahmed, A., Recht, B., & Romberg, J. (2014). Blind deconvolution using convex programming. IEEE Transactions on Information Theory, 60(3), 1711–1732. https://doi.org/10.1109/tit.2013.2294644

    Article  MathSciNet  Google Scholar 

  • Aljadaany, R., Pal, D. K., & Savvides, M. (2019). Douglas-rachford networks: Learning both the image prior and data fidelity terms for blind image deconvolution. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2019.01048

  • Asim, M., Shamshad, F., & Ahmed, A. (2020). Blind image deconvolution using deep generative priors. IEEE Transactions on Computational Imaging, 6, 1493–1506. https://doi.org/10.1109/tci.2020.3032671

    Article  MathSciNet  Google Scholar 

  • Benichoux, A., Vincent, E., & Gribonval, R. (2013). A fundamental pitfall in blind deconvolution with sparse and shift-invariant priors. In: IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE . https://doi.org/10.1109/icassp.2013.6638838

  • Bostan, E., Heckel, R., Chen, M., Kellman, M., & Waller, L. (2020). Deep phase decoder: Self-calibrating phase microscopy with an untrained deep neural network. Optica, 7(6), 559–562.

    Article  Google Scholar 

  • Cabrelli, C. A. (1985). Minimum entropy deconvolution and simplicity: A noniterative algorithm. Geophysics, 50(3), 394–413. https://doi.org/10.1190/1.1441919

    Article  Google Scholar 

  • Chan, T., & Wong, C. K. (1998). Total variation blind deconvolution. IEEE Transactions on Image Processing, 7(3), 370–375. https://doi.org/10.1109/83.661187

    Article  Google Scholar 

  • Chen, L., Fang, F., Wang, T., Zhang, G.(2019). Blind image deblurring with local maximum gradient prior. In: IEEE Conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2019.00184

  • Chen, L., Fang, F., Zhang, J., Liu, J., & Zhang, G. (2020). OID: Outlier identifying and discarding in blind image deblurring. In: European Conference on Computer Vision (ECCV), pp. 598–613. Springer International Publishing. https://doi.org/10.1007/978-3-030-58595-2_36

  • Chen, L., Zhang, J., Lin, S., Fang, F., & Ren, J. S. (2021). Blind deblurring for saturated images. In: IEEE Conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr46437.2021.00624

  • Cheung, S. C., Shin, J. Y., Lau, Y., Chen, Z., Sun, J., Zhang, Y., Müller, M. A., Eremin, I. M., Wright, J. N., & Pasupathy, A. N.(2020). Dictionary learning in fourier-transform scanning tunneling spectroscopy. Nature Communications 11(1):1081. https://doi.org/10.1038/s41467-020-14633-1

  • Chi, Y. (2016). Guaranteed blind sparse spikes deconvolution via lifting and convex optimization. IEEE Journal of Selected Topics in Signal Processing, 10(4), 782–794. https://doi.org/10.1109/jstsp.2016.2543462

    Article  Google Scholar 

  • Cho, S., & Lee, S. (2009). Fast motion deblurring. In: ACM Trans. Graph. ACM Press . https://doi.org/10.1145/1661412.1618491

  • Cho, S., Lee, S.(2017). Convergence analysis of MAP based blur kernel estimation. In: IEEE International Conference on Computer Vision (ICCV). IEEE https://doi.org/10.1109/iccv.2017.515

  • Choudhary, S., & Mitra, U. (2014). Sparse blind deconvolution: What cannot be done. In IEEE international symposium on information theory. IEEE . https://doi.org/10.1109/isit.2014.6875385

  • Choudhary, S., & Mitra, U. (2018). On the properties of the rank-two null space of nonsparse and canonical-sparse blind deconvolution. IEEE Transactions on Signal Processing, 66(14), 3696–3709. https://doi.org/10.1109/tsp.2018.2815014

    Article  MathSciNet  Google Scholar 

  • Darestani, M. Z., & Heckel, R. (2021). Accelerated MRI with un-trained neural networks. IEEE Transactions on Computational Imaging, 7, 724–733. https://doi.org/10.1109/tci.2021.3097596

    Article  Google Scholar 

  • Ding, Z., & Luo, Z. Q. (2000). A fast linear programming algorithm for blind equalization. IEEE Transactions on Communications, 48(9), 1432–1436. https://doi.org/10.1109/26.870004

    Article  Google Scholar 

  • Dong, J., Pan, J., Su, Z., & Yang, M. H.(2017). Blind image deblurring with outlier handling. In IEEE international conference on computer vision (ICCV). IEEE . https://doi.org/10.1109/iccv.2017.271

  • Donoho, D. (1981). ON minimum entropy deconvolution. In Applied time series analysis II, pp. 565–608. Elsevier . https://doi.org/10.1016/b978-0-12-256420-8.50024-1

  • Ekanadham, C., Tranchina, D., Simoncelli, E.(2011). A blind sparse deconvolution method for neural spike identification. In Advances in neural information processing systems

  • Fang, L., Liu, H., Wu, F., Sun, X., & Li, H. (2014). Separable kernel for image deblurring. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2014.369

  • Gandelsman, Y., Shocher, A., & Irani, M. (2019). “double-DIP”: Unsupervised image decomposition via coupled deep-image-priors. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE (2019). https://doi.org/10.1109/cvpr.2019.01128

  • Gong, D., Tan, M., Zhang, Y., van den Hengel, A., & Shi, Q. (2017). Self-paced kernel estimation for robust blind image deblurring. In IEEE International conference on computer vision (ICCV). IEEE . https://doi.org/10.1109/iccv.2017.184

  • Gong, D., Tan, M., Zhang, Y., Hengel, A. V. D., & Shi, Q. (2016). Blind image deconvolution by automatic gradient activation. In IEEE Conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2016.202

  • Heckel, R., Hand, P. (2019). Deep decoder: Concise image representations from untrained non-convolutional networks. In International conference on learning representations

  • Heckel, R., & Soltanolkotabi, M. (2019). Denoising and regularization via exploiting the structural bias of convolutional generators. ar**v preprint ar**v:1910.14634

  • Heckel, R., & Soltanolkotabi, M. (2020). Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation. ar**v:2005.03991

  • Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International conference on learning representations. https://openreview.net/forum?id=HJz6tiCqYm

  • Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101. https://doi.org/10.1214/aoms/1177703732

    Article  MathSciNet  Google Scholar 

  • Hurley, N., & Rickard, S. (2009). Comparing measures of sparsity. IEEE Transactions on Information Theory, 55(10), 4723–4741. https://doi.org/10.1109/tit.2009.2027527

    Article  MathSciNet  Google Scholar 

  • **, M., Roth, S., & Favaro, P. (2018). Normalized blind deconvolution. In European conference on computer vision (ECCV), pp. 694–711. Springer International Publishing . https://doi.org/10.1007/978-3-030-01234-2_41

  • Joshi, N., Szeliski, R., Kriegman, D.J.(2008). PSF estimation using sharp edge prediction. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2008.4587834

  • Joshi, N., Zitnick, C.L., Szeliski, R., & Kriegman, D. J. (2009). Image deblurring and denoising using color priors. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2009.5206802

  • Kech, M., & Krahmer, F. (2017). Optimal injectivity conditions for bilinear inverse problems with applications to identifiability of deconvolution problems. SIAM Journal on Applied Algebra and Geometry, 1(1), 20–37. https://doi.org/10.1137/16m1067469

    Article  MathSciNet  Google Scholar 

  • Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., & Harmeling, S.(2012). Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. In European conference on computer vision (ECCV), pp. 27–40. Springer Berlin Heidelberg . https://doi.org/10.1007/978-3-642-33786-4_3

  • Koh, J., Lee, J., & Yoon, S. (2021). Single-image deblurring with neural networks: A comparative survey. Computer Vision and Image Understanding, 203, 103134. https://doi.org/10.1016/j.cviu.2020.103134

    Article  Google Scholar 

  • Komodakis, N., & Paragios, N. (2013). MRF-based blind image deconvolution. In Asian conference on computer vision (ACCV), pp. 361–374. Springer Berlin Heidelberg . https://doi.org/10.1007/978-3-642-37431-9_28

  • Krishnan, D., & Fergus, R.(2009) Fast image deconvolution using hyper-laplacian priors. In Advances in Neural Information Processing Systems. https://proceedings.neurips.cc/paper/2009/file/3dd48ab31d016ffcbf3314df2b3cb9ce-Paper.pdf

  • Krishnan, D., Tay, T., & Fergus, R. (2011). Blind deconvolution using a normalized sparsity measure. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2011.5995521

  • Kundur, D., & Hatzinakos, D. (1996). Blind image deconvolution. IEEE Signal Processing Magazine, 13(3), 43–64. https://doi.org/10.1109/79.489268

    Article  Google Scholar 

  • Kuo, H. W., Zhang, Y., Lau, Y., & Wright, J. (2020). Geometry and symmetry in short-and-sparse deconvolution. SIAM Journal on Mathematics of Data Science, 2(1), 216–245. https://doi.org/10.1137/19m1237569

    Article  MathSciNet  Google Scholar 

  • Kupyn, O., Martyniuk, T., Wu, J., & Wang, Z. (2019). Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8878–8887

  • Lai, W.S., Huang, J.B., Hu, Z., Ahuja, N., Yang, M.H.(2016). A comparative study for single image blind deblurring. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2016.188

  • Lawrence, H., Bramherzig, D., Li, H., Eickenberg, M., & Gabrié, M. (2020). Phase retrieval with holography and untrained priors: Tackling the challenges of low-photon nanoscale imaging. ar**v preprint ar**v:2012.07386

  • Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Understanding blind deconvolution algorithms. IEEE Transactionson Pattern Analysis and Machine Intelligence, 33(12), 2354–2367. https://doi.org/10.1109/tpami.2011.148

    Article  Google Scholar 

  • Lewicki, M. S. (1998). A review of methods for spike sorting: The detection and classification of neural action potentials. Network: Computation in Neural Systems, 9(4), R53–R78. https://doi.org/10.1088/0954-898x_9_4_001

    Article  Google Scholar 

  • Li, L., Pan, J., Lai, W. S., Gao, C., Sang, N., Yang, M. H. (2018). Learning a discriminative prior for blind image deblurring. In 2018 IEEE/CVF conference on computer vision and pattern recognition. IEEE . https://doi.org/10.1109/cvpr.2018.00692

  • Li, T., Wang, H., Zhuang, Z., & Sun, J. (2023). Deep random projector: Accelerated deep image prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 18176–18185.

  • Li, T., Zhuang, Z., Liang, H., Peng, L., Wang, H., & Sun, J. (2021). Self-validation: Early stop** for single-instance deep generative priors. In British Machine Vision Conference (BMVC).

  • Li, X., Ling, S., Strohmer, T., & Wei, K. (2019). Rapid, robust, and reliable blind deconvolution via nonconvex optimization. Applied and Computational Harmonic Analysis, 47(3), 893–934. https://doi.org/10.1016/j.acha.2018.01.001

    Article  MathSciNet  Google Scholar 

  • Li, Y., Lee, K., & Bresler, Y.(2015). A unified framework for identifiability analysis in bilinear inverse problems with applications to subspace and sparsity models. ar**v:1501.06120

  • Li, Y., Lee, K., & Bresler, Y. (2017). Identifiability and stability in blind deconvolution under minimal assumptions. IEEE Transactions on Information Theory, 63(7), 4619–4633. https://doi.org/10.1109/tit.2017.2689779

    Article  MathSciNet  Google Scholar 

  • Li, Y., Tofighi, M., Geng, J., Monga, V., & Eldar, Y. C. (2019). Deep algorithm unrolling for blind image deblurring. ar**v:1902.03493

  • Liu, Y., Dong, W., Gong, D., Zhang, L., & Shi, Q. (2018). Deblurring natural image using super-gaussian fields. In European conference on computer vision (ECCV), pp. 467–484. Springer International Publishing . https://doi.org/10.1007/978-3-030-01246-5_28

  • Ma, X., Hill, P., & Achim, A. (2021). Unsupervised image fusion using deep image priors. ar**v:2110.09490

  • Michaeli, T., & Irani, M. (2014). Blind deblurring using internal patch recurrence. In European conference on computer vision (ECCV), pp. 783–798. Springer International Publishing . https://doi.org/10.1007/978-3-319-10578-9_51

  • Michelashvili, M., & Wolf, L. (2019). Speech denoising by accumulating per-frequency modeling fluctuations. ar**v:1904.07612

  • Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., & Lee, K. M. (2019). NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study. In 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE . https://doi.org/10.1109/cvprw.2019.00251

  • Nah, S., Kim, T. H., & Lee, K. M.(2017). Deep multi-scale convolutional neural network for dynamic scene deblurring. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2017.35

  • Nah, S., Son, S., Lee, S., Timofte, R., & Lee, K. M. (2021). Ntire 2021 challenge on image deblurring. ar**v:2104.14854

  • Nah, S., Son, S., Timofte, R., & Lee, K. M. (2020). NTIRE 2020 challenge on image and video deblurring. ar**v:2005.01244

  • Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020). Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1(1), 39–56. https://doi.org/10.1109/jsait.2020.2991563

    Article  Google Scholar 

  • Pan, J., Dong, J., Liu, Y., Zhang, J., Ren, J., Tang, J., Tai, Y. W., & Yang, M. H. (2021). Physics-based generative adversarial models for image restoration and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(7), 2449–2462. https://doi.org/10.1109/tpami.2020.2969348

    Article  Google Scholar 

  • Pan, J., Hu, Z., Su, Z., Yang, M. H. (2014). Deblurring text images via l0-regularized intensity and gradient prior. In IEEE Conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2014.371

  • Pan, J., Lin, Z., Su, Z., & Yang, M. H. (2016). Robust kernel estimation with outliers handling for image deblurring. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2016.306

  • Pan, J., Sun, D., Pfister, H., & Yang, M. H. (2016). Blind image deblurring using dark channel prior. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2016.180

  • Perrone, D., Favaro, P.(2014). Total variation blind deconvolution: The devil is in the details. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2014.372

  • Qayyum, A., Ilahi, I., Shamshad, F., Boussaid, F., Bennamoun, M., & Qadir, J. (2021). Untrained neural network priors for inverse imaging problems: A survey. TechRxiv . https://doi.org/10.36227/techrxiv.14208215

  • Ravula, S., & Dimakis, A. G. (2019). One-dimensional deep image prior for time series inverse problems. ar**v:1904.08594

  • Ren, D., Zhang, K., Wang, Q., Hu, Q., & Zuo, W. (2020). Neural blind deconvolution using deep priors. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr42600.2020.00340

  • Rim, J., Lee, H., Won, J., & Cho, S. (2020). Real-world blur dataset for learning and benchmarking deblurring algorithms. In European conference on computer vision (ECCV), pp. 184–201. Springer International Publishing. https://doi.org/10.1007/978-3-030-58595-2_12

  • Schuler, C. J., Hirsch, M., Harmeling, S., & Scholkopf, B. (2016). Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7), 1439–1451. https://doi.org/10.1109/tpami.2015.2481418

    Article  Google Scholar 

  • Sheikh, H., & Bovik, A. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430–444. https://doi.org/10.1109/tip.2005.859378

    Article  Google Scholar 

  • Shi, Z., Mettes, P., Maji, S., & Snoek, C. G. M. (2022). On measuring and controlling the spectral bias of the deep image prior. International Journal of Computer Vision, 130(4), 885–908. https://doi.org/10.1007/s11263-021-01572-7

    Article  Google Scholar 

  • Si-Yao, L., Ren, D., & Yin, Q. (2019). Understanding kernel size in blind deconvolution. In IEEE winter conference on applications of computer vision (WACV). IEEE. https://doi.org/10.1109/wacv.2019.00224

  • Sitzmann, V., Martel, J., Bergman, A., Lindell, D., & Wetzstein, G. (2020). Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33, 7462–7473.

    Google Scholar 

  • Sun, L., Cho, S., Wang, J., & Hays, J. (2013). Edge-based blur kernel estimation using patch priors. In IEEE international conference on computational photography (ICCP). IEEE. https://doi.org/10.1109/iccphot.2013.6528301

  • Sun, Q., & Donoho, D. (2021). Convex sparse blind deconvolution. ar**v:2106.07053

  • Szeliski, R. (2021). Computer vision: Algorithms and applications (2nd ed.). London: Springer.

    Google Scholar 

  • Tai, Y.W., & Lin, S. (2012). Motion-aware noise filtering for deblurring of noisy and blurry images. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2012.6247653

  • Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., & Ng, R.(2020). Fourier features let networks learn high frequency functions in low dimensional domains. In Advances in neural information processing systems.

  • Tao, X., Gao, H., Shen, X., Wang, J., & Jia, J. (2018). Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8174–8182.

  • Tayal, K., Manekar, R., Zhuang, Z., Yang, D., Kumar, V., Hofmann, F., & Sun, J. (2021). Phase retrieval using single-instance deep generative prior. In OSA optical sensors and sensing congress 2021 (AIS, FTS, HISE, SENSORS, ES). OSA . https://doi.org/10.1364/ais.2021.jw2a.37

  • Tran, P., Tran, A., Phung, Q., & Hoai, M. (2021). Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR46437.2021.01178

  • Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2020). Deep image prior. International Journal of Computer Vision, 128(7), 1867–1888. https://doi.org/10.1007/s11263-020-01303-4

    Article  Google Scholar 

  • Vasu, S.(2021). Image and video deblurring: A curated list of resources for image and video deblurring. https://github.com/subeeshvasu/Awesome-Deblurring. Accessed on Dec 12 2021

  • Vembu, S., Verdu, S., Kennedy, R., & Sethares, W. (1994). Convex cost functions in blind equalization. IEEE Transactions on Signal Processing, 42(8), 1952–1960. https://doi.org/10.1109/78.301833

    Article  Google Scholar 

  • Wang, H., Li, T., Zhuang, Z., Chen, T., Liang, H., & Sun, J. (2021). Early stop** for deep image prior. ar**v:2112.06074

  • Wang, Z., Wang, Z., Li, Q., & Bilen, H. (2019). Image deconvolution with deep image and kernel priors. In 2019 IEEE/CVF international conference on computer vision workshop (ICCVW). IEEE . https://doi.org/10.1109/iccvw.2019.00127

  • Wiggins, R. A. (1978). Minimum entropy deconvolution. Geoexploration, 16(1–2), 21–35. https://doi.org/10.1016/0016-7142(78)90005-4

    Article  Google Scholar 

  • Williams, F., Schneider, T., Silva, C., Zorin, D., Bruna, J., & Panozzo, D. (2019). Deep geometric prior for surface reconstruction. ar**v:1811.10943

  • Wipf, D., & Zhang, H. (2014). Revisiting bayesian blind deconvolution. Journal of Machine Learning Research, 15(111), 3775–3814.

    MathSciNet  Google Scholar 

  • Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In European Conference on Computer Vision, pp. 157–170. Springer Berlin Heidelberg . https://doi.org/10.1007/978-3-642-15549-9_12

  • Xu, L., Zheng, S., & Jia, J.(2013). Unnatural l0 sparse representation for natural image deblurring. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2013.147

  • Yan, Y., Ren, W., Guo, Y., Wang, R., & Cao, X. (2017). Image deblurring via extreme channels prior. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2017.738

  • Yang, D., Zhuang, Z., Phillips, N.W., KaySong, Zdora, M.C., Harder, R., Cha, W., Liu, W., Barmherzig, D.A., Sun, J., & Hofmann, F. (2022). Application of single-instance deep generative priors for reconstruction of highly strained gold microcrystals in bragg coherent x-ray diffraction. In preparation 25

  • Yang, L., & Ji, H. (2019). A variational EM framework with adaptive edge selection for blind motion deblurring. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2019.01041

  • Zhang, K., Luo, W., Zhong, Y., Ma, L., Stenger, B., Liu, W., & Li, H. (2020). Deblurring by realistic blurring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2737–2746.

  • Zhang, K., Ren, W., Luo, W., Lai, W. S., Stenger, B., Yang, M. H., & Li, H. (2022). Deep image deblurring: A survey. International Journal of Computer Vision, 130(9), 2103–2130. https://doi.org/10.1007/s11263-022-01633-5

  • Zhang, K., Zuo, W., & Zhang, L. (2019). Deep plug-and-play super-resolution for arbitrary blur kernels. In 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2019.00177

  • Zhang, R., Isola, P., Efros, A.A., Shechtman, E., & Wang, O.(2018). The unreasonable effectiveness of deep features as a perceptual metric 16.

  • Zhang, Y., Kuo, H. W., & Wright, J. (2020). Structured local optima in sparse blind deconvolution. IEEE Transactions on Information Theory, 66(1), 419–452. https://doi.org/10.1109/tit.2019.2940657

    Article  MathSciNet  Google Scholar 

  • Zhang, Y., Lau, Y., Kuo, H.W., Cheung, S., Pasupathy, A., & Wright, J. (2017). On the global geometry of sphere-constrained sparse blind deconvolution. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2017.466

  • Zhong, L., Cho, S., Metaxas, D., Paris, S., & Wang, J. (2013). Handling noise in single image deblurring using directional filters. In IEEE conference on computer vision and pattern recognition (CVPR). IEEE . https://doi.org/10.1109/cvpr.2013.85

  • Zhou, K. C., & Horstmeyer, R. (2020). Diffraction tomography with a deep image prior. Optics Express, 28(9), 12872. https://doi.org/10.1364/oe.379200

    Article  Google Scholar 

  • Zhuang, Z., Yang, D., Hofmann, F., Barmherzig, D., & Sun, J. (2022). Practical phase retrieval using double deep image priors. ar**v preprint ar**v:2211.00799

Download references

Acknowledgements

Zhong Zhuang, Hengkang Wang, and Ju Sun are partially supported by NSF CMMI 2038403. We thank the anonymous reviewers and the associate editor for their insightful comments that have substantially helped us improve the presentation of this paper. We thank Le Peng and Wenjie Zhang for allowing us to use the e-scooter image of Fig. 1 that they captured. The authors acknowledge the Minnesota Supercomputing Institute (MSI) at the University of Minnesota for providing resources that contributed to the research results reported within this paper.

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering, University of Minnesota, Minneapolis, USA

    Zhong Zhuang

  2. Computer Science and Engineering, University of Minnesota, Minneapolis, USA

    Taihui Li, Hengkang Wang & Ju Sun

Authors
  1. Zhong Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Taihui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hengkang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ju Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ju Sun.

Additional information

Communicated by Jiaya Jia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 List of common acronyms

Table 4 List of acronyms (in alphabetic order)
Full size table

See Table 4

Fig. 36
figure 36

Contrast-enhanced version of Fig. 30 after histogram equalization

Full size image
Fig. 37
figure 37

Contrast-enhanced version of Fig. 32 after histogram equalization

Full size image

1.2 Contrast-Enhanced Version of Figs. 30 and 32

To reveal more details for images in Figs. 30 and 32 that are about extremely dark scenes, we perform histogram equalization to enhance the contrast and display the results as follows (Figs. 36, 37).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhuang, Z., Li, T., Wang, H. et al. Blind Image Deblurring with Unknown Kernel Size and Substantial Noise. Int J Comput Vis 132, 319–348 (2024). https://doi.org/10.1007/s11263-023-01883-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-023-01883-x

Keywords

Search

Navigation