SpringerLink
Log in

Iterative dual regression network for blind image super-resolution

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Previous single-image super-resolution methods assume that the blur kernel is known (e.g., bicubic) when degrading from high-resolution (HR) images to low-resolution (LR) images. They use a single degradation to train a model to restore HR images. However, the actual degradation in the real world is often unknown. It is difficult to deal with LR images caused by different degradations. To cope with the above situation, previous methods attempt to restore SR images using a blur kernel estimation structure that combines with a non-blind SR network. There are two problems that should be earnestly considered: (1) For accurate blur kernel estimation, insufficient correlation of consecutive kernels lead to an unsatisfied reconstruction result. (2) For ill-posed issue of image reconstruction, a more efficient constraint condition is worth trying. To solve the two problems, we propose an iterative dual regression network for an adaptive and precision blur kernel estimation, which improves the speed of kernel estimation by learning a dual map**. Specifically, we design a Predictor-Generator structure: the Predictor, through several iterations, searching for accurate kernels through intermediate kernels and generated SR images; the Generator, generating final SR images with the help of the predicted kernels. More importantly, the elaborately designed dual learning strategy can not only provide additional constraints for accurate kernel estimation but also reduce the domain gap between SR images and HR images. Experiments on synthetic degraded images and real-world images show that our network is competitive in performance and superior in visual results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

Data supporting this study are openly available from DF2K at https://github.com/LimBee/NTIRE2017

References

  1. Li, Z., Yang, J., Liu, Z., Yang, X., Jeon, G., Wu, W.: Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3867–3876 (2019)

  2. Chen, L., Tang, R., Anisetti, M., Yang, X.: A lightweight iterative error reconstruction network for infrared image super-resolution in smart grid. Sustain. Cities Soc. 66, 102520 (2021)

    Article  Google Scholar 

  3. Chen, L., Yang, X., Jeon, G., Anisetti, M., Liu, K.: A trusted medical image super-resolution method based on feedback adaptive weighted dense network. Artif. Intell. Med. 106, 101857 (2020)

    Article  Google Scholar 

  4. Dong, C., Loy, C. C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, pp. 184–199. Springer (2014)

  5. Dong, C., Loy, C. C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, pp. 391–407. Springer (2016)

  6. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  7. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,pp. 136–144 (2017)

  8. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta,A., Aitken, A., Tejani, A., Totz, J., Wang , Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)

  9. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4799–4807 (2017)

  10. Guo, Y., Chen, J., Wang, J., Chen, Q., Cao, J., Deng, Z., Xu, Y., Tan, M.: Closed-loop matters: dual regression networks for single image super-resolution In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5407–5416 (2020)

  11. He, Z., Liu, K., Liu, Z., Dou, Q., Yang, X.: A lightweight multi-scale feature integration network for real-time single image super-resolution. J. Real-Time Image Process. 18(4), 1221–1234 (2021)

    Article  Google Scholar 

  12. Zou, Y., Yang, X., Albertini, M.K., Hussain, F.: Lmsn: a lightweight multi-scale network for single image super-resolution. Multimed. Syst. 27(4), 845–856 (2021)

    Article  Google Scholar 

  13. Meng, B., Wang, L., He, Z., Jeon, G., Dou, Q., Yang, X.: Gradient information distillation network for real-time single-image super-resolution. J. Real-Time Image Proc. 18(2), 333–344 (2021)

    Article  Google Scholar 

  14. Liu, A., Liu, Y., Gu, J., Qiao, Y., Dong, C.: Blind image super-resolution: a survey and beyond. ar**v preprint ar**v:2107.03055 (2021)

  15. Gu, J., Lu, H., Zuo, W., Dong, C.: Blind super-resolution with iterative kernel correction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1604–1613 (2019)

  16. Luo, Z., Huang, Y., Li, S., Wang, L., Tan, T.: Unfolding the alternating optimization for blind super resolution. ar**v preprint ar**v:2010.02631 (2020)

  17. Shocher, A., Cohen, N., Irani, M.: “Zero-shot” super-resolution using deep internal learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3118–3126 (2018)

  18. Cheng, X., Fu, Z., Yang, J.: Zero-shot image super-resolution with depth guided internal degradation learning. In: European Conference on Computer Vision, pp. 265–280. Springer (2020)

  19. Bell-Kligler, S., Shocher, A., Irani, M.: Blind super-resolution kernel estimation using an internal-gan. ar**v preprint ar**v:1909.06581 (2019)

  20. Hui, Z., Li, J., Wang, X., Gao, X.: Learning the non-differentiable optimization for blind super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2093–2102 (2021)

  21. Yamac, M., Ataman, B., Nawaz, A.: Kernelnet: A blind super-resolution kernel estimation network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 453–462 (2021)

  22. Wang, L., Wang, Y., Dong, X., Xu, Q., Yang, J., An, W., Guo, Y.: Unsupervised degradation representation learning for blind super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10581–10590 (2021)

  23. Jo, Y., Oh, S. W., Vajda, P., Kim, S. J.: Tackling the ill-posedness of super-resolution through adaptive target generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16236–16245 (2021)

  24. He, D., **a, Y., Qin, T., Wang, L., Yu, N., Liu, T.-Y., Ma, W.-Y.: Dual learning for machine translation. Adv. Neural. Inf. Process. Syst. 29, 820–828 (2016)

    Google Scholar 

  25. Yi, Z., Zhang, H., Tan, P., Gong, M.: Dualgan: Unsupervised dual learning for image-to-image translation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2849–2857 (2017)

  26. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  27. Yin, G., Wang, W., Yuan, Z., Yu, D., Sun, S., Wang, C.: Conditional meta-network for blind super-resolution with multiple degradations. ar**v preprint ar**v:2104.03926 (2021)

  28. Zhang, K., Gool, L. V., Timofte, R.: Deep unfolding network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3217–3226 (2020)

  29. Agustsson, E., Timofte, R.: Ntire: challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 126–135 (2017)

  30. Bevilacqua, M., Roumy, A., Guillemot, C., Morel, A.: Low-complexity single image super-resolution based on nonnegative neighbor embedding. bmvc (2012)

  31. Zeyde, R., Elad, M., Protter,M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730. Springer (2010)

  32. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE International Conference on Computer Vision (2002)

  33. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE (2015)

  34. Kingma, D. P., Ba, J.: Adam: A method for stochastic optimization. ar**v preprint ar**v:1412.6980 (2014)

  35. Zhang, K., Liang, J., Van Gool, L., Timofte, R.: Designing a practical degradation model for deep blind image super-resolution. In: IEEE International Conference on Computer Vision, pp. 4791–4800 (2021)

  36. Wang, X., **e, L., Dong, C., Shan, Y.: Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1905–1914 (2021)

  37. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1833–1844 (2021)

Download references

Funding

This work was supported in part by the Science Foundation of Sichuan Science and Technology Department under Grant 2021YFH0119 and in part by Sichuan University under Grant 2020SCUNG205.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610065, China

    Chunting Lei, Sihan Yang, **aomin Yang & Binyu Yan

  2. Department of Embedded Systems Engineering, Incheon National University, Incheon, 22012, Korea

    Gwanggil Jeon

Authors
  1. Chunting Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sihan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. **aomin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Binyu Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gwanggil Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chunting Lei and Sihan Yan wrote the main manuscript text. **aomin Yang, Binyu Yan and Gwanggil Jeon examined the contents of the paper carefully.

Corresponding author

Correspondence to Binyu Yan.

Ethics declarations

Competing interests

No, I declare that the authors have no competing interests as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Ethical Approval

This declaration is not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 494 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, C., Yang, S., Yang, X. et al. Iterative dual regression network for blind image super-resolution. SIViP 17, 2437–2446 (2023). https://doi.org/10.1007/s11760-022-02460-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02460-4

Keywords

Search

Navigation