SpringerLink
Log in

A channel-gained single-model network with variable rate for multispectral image compression in UAV air-to-ground remote sensing

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicle (UAV) air-to-ground remote sensing technology, has the advantages of long flight duration, real-time image transmission, wide applicability, low cost, and so on. To better preserve the integrity of image features during transmission and storage, and improve efficiency in the meanwhile, image compression is a very important link. Nowadays the image compressor based on deep learning framework has been updating as the technological development. However, in order to obtain enough bit rates to fit the performance curve, there is always a severe computational burden, especially for multispectral image compression. This problem arises not only because the complexity of the algorithm is deepening, but also repeated training with rate-distortion optimization. In this paper, a channel-gained single-model network with variable rate for multispectral image compression is proposed. First, a channel gained module is introduced to map the channel content of the image to vector domain as amplitude factors, which leads to representation scaling, as well as obtaining the image representation of different bit rates in a single model. Second, after extracting spatial-spectral features, a plug-and-play dynamic response attention mechanism module is applied to take good care of distinguishing the content correlation of features and weighting the important area dynamically without adding extra parameters. Besides, a hyperprior autoencoder is used to make full use of edge information for entropy estimation, which contributes to a more accurate entropy model. The experiments prove that the proposed method greatly reduces the computational cost, while maintaining good compression performance and surpasses JPEG2000 and some other algorithms based on deep learning in PSNR, MSSSIM and MSA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Data availability

No datasets were generated or analysed during the current study.

References

  1. Roy, S.K., Manna, S., Song, T., Bruzzone, L.: Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59(9), 7831–7843 (2021). https://doi.org/10.1109/TGRS.2020.3043267

    Article  Google Scholar 

  2. Wallace, G.K.: The jpeg still picture compression standard. IEEE Trans. Consum. Electron. (1992). https://doi.org/10.1109/30.125072

    Article  Google Scholar 

  3. Taubman, D.S., Marcellin, M.W.: Jpeg2000—image compression fundamentals, standards and practice. In: The Kluwer International Series in Engineering and Computer Science (2013). https://api.semanticscholar.org/CorpusID:62197160

  4. Bellard, F.: Bpg image format (2014)

  5. Cui, Z., Wang, J., Gao, S., Guo, T., Feng, Y., Bai, B.: Asymmetric gained deep image compression with continuous rate adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10532–10541 (2021)

  6. Egmont-Petersen, M., Ridder, D., Handels, H.: Image processing with neural networks—a review. Pattern Recogn. 35(10), 2279–2301 (2002)

    Article  Google Scholar 

  7. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 25, 1097–1105 (2012)

    Google Scholar 

  8. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp. 818–833. Springer (2014)

  9. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ar**v preprint ar**v:1409.1556 (2014)

  10. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  12. Jiang, F., Tao, W., Liu, S., Ren, J., Guo, X., Zhao, D.: An end-to-end compression framework based on convolutional neural networks. IEEE Trans. Circuits Syst. Video Technol. 28(10), 3007–3018 (2017)

    Article  Google Scholar 

  13. Roy, S.K., Krishna, G., Dubey, S.R., Chaudhuri, B.B.: HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 17(2), 277–281 (2019)

    Article  Google Scholar 

  14. Kong, F., Zhou, Y., Shen, Q., Wen, K.: End-to-end multispectral image compression using convolutional neural network. Chin. J. Lasers 46(10), 1009001–1 (2019)

    Google Scholar 

  15. Kong, F., Zhao, S., Li, Y., Li, D.: End-to-end multispectral image compression framework based on adaptive multiscale feature extraction. J. Electron. Imaging 30(1), 013010–013010 (2021)

    Article  Google Scholar 

  16. Kong, F., Hu, K., Li, Y., Li, D., Zhao, S.: Spectral-spatial feature partitioned extraction based on CNN for multispectral image compression. Remote Sens. 13(1), 9 (2020)

    Article  Google Scholar 

  17. Kong, F., Zhao, S., Li, Y., Li, D., Zhou, Y.: A residual network framework based on weighted feature channels for multispectral image compression. Ad Hoc Netw. 107, 102272 (2020)

    Article  Google Scholar 

  18. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. ar**v preprint ar**v:1802.01436 (2018)

  19. Webb, B.S., Dhruv, N.T., Solomon, S.G., Tailby, C., Lennie, P.: Early and late mechanisms of surround suppression in striate cortex of macaque. J. Neurosci. 25(50), 11666–11675 (2005)

    Article  Google Scholar 

  20. Yang, L., Zhang, R.-Y., Li, L., **e, X.: SimAM: a simple, parameter-free attention module for convolutional neural networks. In: International Conference on Machine Learning, pp. 11863–11874. PMLR (2021)

  21. Toderici, G., O’Malley, S.M., Hwang, S.J., Vincent, D., Minnen, D., Baluja, S., Covell, M., Sukthankar, R.: Variable rate image compression with recurrent neural networks. ar**v preprint ar**v:1511.06085 (2015)

  22. Toderici, G., Vincent, D., Johnston, N., ** Hwang, S., Minnen, D., Shor, J., Covell, M.: Full resolution image compression with recurrent neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5306–5314 (2017)

  23. Johnston, N., Vincent, D., Minnen, D., Covell, M., Singh, S., Chinen, T., Hwang, S.J., Shor, J., Toderici, G.: Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4385–4393 (2018)

  24. Choi, Y., El-Khamy, M., Lee, J.: Variable rate deep image compression with a conditional autoencoder. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3146–3154 (2019)

  25. Yang, F., Herranz, L., Van De Weijer, J., Guitián, J.A.I., López, A.M., Mozerov, M.G.: Variable rate deep image compression with modulated autoencoder. IEEE Signal Process. Lett. 27, 331–335 (2020)

    Article  Google Scholar 

  26. Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. ar**v preprint ar**v:1703.00395 (2017)

  27. Tong, K., Wu, Y., Li, Y., Zhang, K., Zhang, L., **, X.: QVRF: a quantization-error-aware variable rate framework for learned image compression. ar**v preprint ar**v:2303.05744 (2023)

  28. Yin, S., Li, C., Bao, Y., Liang, Y., Meng, F., Liu, W.: Universal efficient variable-rate neural image compression. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2025–2029. IEEE (2022)

  29. Jia, C., Ge, Z., Wang, S., Ma, S., Gao, W.: Rate distortion characteristic modeling for neural image compression. In: 2022 Data Compression Conference (DCC), pp. 202–211. IEEE (2022)

  30. Mnih, V., Heess, N., Graves, A., et al.: Recurrent models of visual attention. Adv Neural Inf Process Syst. 2, 2204–2212 (2014)

    Google Scholar 

  31. Li, M., Zuo, W., Gu, S., You, J., Zhang, D.: Learning content-weighted deep image compression. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3446–3461 (2020)

    Article  Google Scholar 

  32. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  33. Kong, F., Cao, T., Li, Y., Li, D., Hu, K.: Multi-scale spatial-spectral attention network for multispectral image compression based on variational autoencoder. Signal Process. 198, 108589 (2022)

    Article  Google Scholar 

  34. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. ar**v preprint ar**v:1802.01436 (2018)

  35. Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyperprior. ar**v preprint ar**v:1802.01436 (2018)

  36. Zhou, L., Cai, C., Gao, Y., Su, S., Wu, J.: Variational autoencoder for low bit-rate image compression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2617–2620 (2018)

  37. Yang, L., Zhang, R.-Y., Li, L., **e, X.: SimAM: a simple, parameter-free attention module for convolutional neural networks. In: International Conference on Machine Learning, pp. 11863–11874. PMLR (2021)

  38. Hariharan, B., Malik, J., Ramanan, D.: Discriminative decorrelation for clustering and classification. In: European Conference on Computer Vision, pp. 459–472. Springer (2012)

  39. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. ar**v preprint ar**v:1412.6980 (2014)

  40. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402. IEEE (2003)

  41. Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. ar**v preprint ar**v:1611.01704 (2016)

  42. Minnen, D., Singh, S.: Channel-wise autoregressive entropy models for learned image compression. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 3339–3343. IEEE (2020)

Download references

Author information

Author notes

  1. Daiyin Zhu and Kedi Hu contributed equally to this work.

Authors and Affiliations

  1. Informationization Department (InformationTechnology Center), Nan**g University of Aeronautics and Astronautics, Nan**g, 211106, Jiangsu, China

    Wei Wang

  2. College of Electronic and Information Engineering, Nan**g University of Aeronautics and Astronautics, Nan**g, 211106, Jiangsu, China

    Daiyin Zhu

  3. College of Astronautics, Nan**g University of Aeronautics and Astronautics, Nan**g, 211106, Jiangsu, China

    Kedi Hu

Authors
  1. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daiyin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kedi Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Wei Wang conceptualized and designed the algorithm, implemented the initial codebase, contributed to algorithm improvements and code optimization, and prepared the original manuscript draft. Daiyin Zhu provided essential theoretical insights, and critically revised the manuscript for important intellectual content, and conducted a thorough review and final approval of the manuscript prior to submission. Kedi Hu contributed to the development and fine-tuning of the algorithm, performed substantial debugging, designed and executed the performance tests, analyzed the computational results, and assisted with manuscript writing and revision. All authors discussed the results and contributed to the final manuscript.

Corresponding author

Correspondence to Wei Wang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Communicated by Qiu Shen.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Zhu, D. & Hu, K. A channel-gained single-model network with variable rate for multispectral image compression in UAV air-to-ground remote sensing. Multimedia Systems 30, 193 (2024). https://doi.org/10.1007/s00530-024-01398-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01398-6

Keywords

Search

Navigation