SpringerLink
Log in

EGARNet: adjacent residual lightweight super-resolution network based on extended group-enhanced convolution

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Convolutional neural networks can solve single-image super-resolution (SR) problems owing to their powerful learning capabilities. At present, most lightweight networks are realized by stacking lightweight modules to deepen the network, which leads to the loss of flow characteristics during transmission. This paper proposes a lightweight SR network (EGARNet) based on extended group-enhanced convolution. Residual learning is introduced outside the network of cascaded lightweight modules, and adjacent residual convolution is proposed. Consequently, the shallow network features are fused by residual blocks and deep high-frequency features, which is conducive to the reconstruction of image edge structure. Our model not only reconstructs a clear edge structure but also balances the relationship between the model complexity and reconstructed image quality. The model was confirmed to be effective and lightweight using Set5, Set14, Urban100, and BSD100 test sets.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data that support the findings of this study are openly available in DIV2K at https://data.vision.ee.ethz.ch/cvl/DIV2K/. The other data are included in the paper.

References

  1. Georgescu, M. I., Ionescu, R. T., Miron, A. I., Savencu, O., Ristea, N. C., Verga, N., & Khan, F. S.: Multimodal multi-head convolutional attention with various kernel sizes for medical image super-resolution. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 2195–2205 (2023)

  2. Li, G., Lyu, J., Wang, C., Dou, Q., Qin, J.: WavTrans: Synergizing wavelet and cross-attention transformer for multi-contrast MRI super-resolution. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 463–473. Springer, Cham (2022)

    Google Scholar 

  3. Lyu, Q., Shan, H., Steber, C., Helis, C., Whitlow, C., Chan, M., Wang, G.: Multi-contrast super-resolution MRI through a progressive network. IEEE Trans. Med. Imaging 39(9), 2738–2749 (2020)

    Article  Google Scholar 

  4. Liu, Y., Sun, W.: Blind super-resolution for single remote sensing image via sparse representation and transformed self-similarity. J. Phys. Conf. Ser. 1575(1), 012115 (2020)

    Article  Google Scholar 

  5. Sheikholeslami, M.M., Nadi, S., Naeini, A.A., Ghamisi, P.: An efficient deep unsupervised super-resolution model for remote sensing images. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 13, 1937–1945 (2020)

    Article  Google Scholar 

  6. Jia, S., Wang, Z., Li, Q., Jia, X., & Xu, M.: Multi-Attention Generative Adversarial Network for Remote Sensing Image Super Resolution. IEEE Transactions on Geoscience and Remote Sensing (2022)

  7. McDonnell, L.A., Heeren, R.M.: Imaging mass spectrometry. Mass Spectrom. Rev. 26(4), 606–643 (2007)

    Article  Google Scholar 

  8. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Article  Google Scholar 

  9. Muqeet, A., et al.: Ultra lightweight image super-resolution with multi-attention layers. Preprint at https://arxiv.org/abs/quant-ph/2008.12912 2.5 (2020).

  10. Gao, G., et al.: Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer. ar**v e-prints (2022).

  11. Gao, G., Li, W., Li, J., Wu, F., Lu, H., Yu, Y.: Feature distillation interaction weighting network for lightweight image super-resolution. Proc. AAAI Conf. Artif. Intell. 36(1), 661–669 (2022)

    Google Scholar 

  12. Muqeet, A., et al.: Multi-attention based ultra lightweight image super-resolution. In: Bartoli, A., Fusiello, A. (eds.) Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer International Publishing, Cham (2020)

    Google Scholar 

  13. Park, K., Soh, J. W., & Cho, N. I.: Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution. IEEE Transactions on Multimedia (2021)

  14. Mehri, A., Ardakani, P. B., & Sappa, A. D.: MPRNet: Multi-path residual network for lightweight image super resolution. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 2704–2713 (2021)

  15. Liu, J., Tang, J., Wu, G.: Residual feature distillation network for lightweight image super-resolution. In: Bartoli, A., Fusiello, A. (eds.) European Conference on Computer Vision, pp. 41–55. Springer, Cham (2020)

    Google Scholar 

  16. Kim, J., Lee, J. K., & Lee, K. M.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1637–1645 (2016)

  17. Choi, Jun-Ho, et al.: Lightweight and efficient image super-resolution with block state-based recursive network. Preprint at https://arxiv.org/abs/quant-ph/1811.12546 (2018)

  18. Ayazoğlu, M.: IMDeception: Grouped Information Distilling Super-Resolution Network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 756–765 (2022)

  19. Wang, Y.: Edge-enhanced Feature Distillation Network for Efficient Super-Resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 777–785 (2022)

  20. Park, S.U., and Nojun K.,: Local-Selective Feature Distillation for Single Image Super-Resolution. Preprint at https://arxiv.org/abs/quant-ph/2111.10988 (2021)

  21. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Leibe, B. (ed.) European conference on computer vision, pp. 391–407. Springer, Cham (2016)

    Google Scholar 

  22. Tai, Y., Yang, J., & Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3147–3155 (2017)

  23. Hui, Z., Wang, X., & Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 723–731 (2018)

  24. Lai, W. S., Huang, J. B., Ahuja, N., & Yang, M. H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp. 624–632 (2017)

  25. Wang, L., et al.: Exploring sparsity in image super-resolution for efficient inference. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2021)

  26. Zhang, K., Zuo, W., & Zhang, L.,: Learning a single convolutional super-resolution network for multiple degradations. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3262–3271 (2018)

  27. Kim, J., Lee, J. K., & Lee, K. M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1646–1654 (2016)

  28. Ahn, N., Kang, B., & Sohn, K. A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV). pp. 252–268 (2018)

  29. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., & Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1874–1883 (2016)

  30. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., & Shi, W. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4681–4690 (2017)

  31. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2472–2481 (2018)

  32. He, K., et al.: Deep Residual Learning for Image Recognition. In: IEEE Conference on Computer Vision and Pattern Recognition IEEE (2016).

  33. Hsu, C.C., Lin, C.H.: Dual reconstruction with densely connected residual network for single image super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3643–3650. IEEE, New York (2019)

    Chapter  Google Scholar 

  34. Song, D., Xu, C., Jia, X., Chen, Y., Xu, C., Wang, Y.: Efficient residual dense block search for image super-resolution. Proc. AAAI Conf. Artif. Intell. 34(07), 12007–12014 (2020)

    Google Scholar 

  35. Li, S., He, F., Du, B., Zhang, L., Xu, Y., & Tao, D.: Fast spatio-temporal residual network for video super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10522–10531 (2019)

  36. Hui, Z., Gao, X., Yang, Y., & Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th acm international conference on multimedia. pp. 2024–2032 (2019)

  37. Tian, C., et al.: Image super-resolution with an enhanced group convolutional neural network. Neural Netw. 153, 373–385 (2022)

    Article  Google Scholar 

  38. Tian, C., Zhang, Y., Zuo, W., Lin, C. W., Zhang, D., & Yuan, Y.: A heterogeneous group CNN for image super-resolution. In: IEEE Transactions on Neural Networks and Learning Systems (2022)

  39. Lin, G., Wu, Q., Qiu, L., Huang, X.: Image super-resolution using a dilated convolutional neural network. Neurocomputing 275, 1219–1230 (2018)

    Article  Google Scholar 

  40. Yu, F., and Koltun. V.: Multi-Scale Context Aggregation by Dilated Convolutions. ICLR (2016)

  41. Zheng, L., et al.: Efficient Mixed Transformer for Single Image Super-Resolution. Preprint at https://arxiv.org/abs/quant-ph/2305.11403 (2023)

  42. Wang, H., et al.: Omni Aggregation Networks for Lightweight Image Super-Resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

  43. Zhu, Y., et al.: Denoising Diffusion Models for Plug-and-Play Image Restoration. Preprint at https://arxiv.org/abs/quant-ph/2305.08995 (2023).

  44. Xu, M., Jie M., Yuanyuan Z.: Dual-Diffusion: Dual Conditional Denoising Diffusion Probabilistic Models for Blind Super-Resolution Reconstruction in RSIs. Preprint at https://arxiv.org/abs/quant-ph/2305.12170 (2023)

  45. Zhao, H., et al.: Efficient image super-resolution using pixel attention. In: Bartoli, A., Fusiello, A. (eds.) Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer International Publishing, Cham (2020)

    Google Scholar 

  46. Liu, J., Tang, J., Wu, G.: Residual Feature Distillation Network for Lightweight Image Super-Resolution. Springer, Cham (2021)

    Google Scholar 

  47. Tian, C., et al.: Lightweight image super-resolution with enhanced CNN. Knowl.-Based Syst. 205, 106235 (2020)

    Article  Google Scholar 

  48. Sun, Long, et al. "Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution." ar**v preprint Preprint at https://arxiv.org/abs/quant-ph/2302.13800 (2023).

  49. Fu, J., et al.: KXNet: A model-driven deep neural network for blind super-resolution. In: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIX. Springer Nature Switzerland, Cham (2022)

    Google Scholar 

  50. Tian, C., et al.: Asymmetric CNN for Image Super resolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems PP.99 (2021):1–13.

  51. Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, M. L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding (2012)

  52. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423. IEEE, New York (2001)

    Chapter  Google Scholar 

  53. Huang, J. B., Singh, A., & Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5197–5206 (2015).

Download references

Acknowledgements

This work was partly supported by Anhui Provincial Universities Outstanding Young Backbone Talents Domestic Visiting Study and Research Project (Grant No. gxgnfx2019006), the University Synergy Innovation Program of Anhui Province, China (Grant Nos: GXXT-2021-002, GXXT-2022-033), and the projects of Natural Science Foundation of Anhui Provincial Department of Education (Grant No. KJ2019A0603).

Author information

Authors and Affiliations

  1. Anhui Engineering Research Center for Intelligent Computing and Application On Cognitive Behavior (ICACB), School of Computer Science and Technology, Huaibei Normal University, Huaibei, Anhui, China

    Longfeng Shen & Fenglan Qin

  2. Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China

    Longfeng Shen, Hongying Zhu, Dengdi Sun & Hai Min

  3. Anhui Big-Data Research Center On University Management, Huaibei, Anhui, China

    Longfeng Shen & Fenglan Qin

  4. School of Life Sciences, University of Science and Technology of China, Hefei, Anhui, China

    Hongying Zhu

  5. School of Artificial Intelligence, Anhui University, Hefei, Anhui, China

    Dengdi Sun

  6. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, Anhui, China

    Hai Min

Authors
  1. Longfeng Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fenglan Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongying Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dengdi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hai Min
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LS and FQ wrote the main namuscript text, HZ, DS and HM processed the data, FQ prepared the figures. All authors reviewed the manuscript.

Corresponding author

Correspondence to Longfeng Shen.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Communicated by F. Wu.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, L., Qin, F., Zhu, H. et al. EGARNet: adjacent residual lightweight super-resolution network based on extended group-enhanced convolution. Multimedia Systems 29, 2651–2668 (2023). https://doi.org/10.1007/s00530-023-01137-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01137-3

Keywords

Search

Navigation