Abstract
Hyperspectral images have multi-dimensional information and play an important role in many fields. Recently, based on the compressed sensing (CS), spectral snapshot compressive imaging (SCI) can balance spatial and spectral resolution compared with traditional methods, so it has attached more and more attention. The Plug-and-Play (PnP) framework based on spectral SCI can effectively reconstruct high-quality hyperspectral images, but there exists a serious problem of parameter dependence. In this paper, we propose a PnP hyperspectral reconstruction method based on reinforcement learning (RL), where a suitable policy network through deep reinforcement learning can adaptively tune the parameters in the PnP method to adjust the denoising strength, penalty factor of the deep denoising network, and the terminal time of iterative optimization. Compared with other model-based and learning-based methods and methods with different parameters tuning policies, the reconstruction results obtained by the proposed method have advantages in quantitative indicators and visual effects.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arad, B., Ben-Shahar, O.: Sparse recovery of hyperspectral signal from natural RGB images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 19–34. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_2
Basedow, R.W., Carmer, D.C., Anderson, M.E.: Hydice system: implementation and performance. In: Proceedings of Imaging Spectrometry, vol. 2480, pp. 258–267. SPIE (1995)
Bioucas-Dias, J.M., Figueiredo, M.A.: A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16(12), 2992–3004 (2007)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
Chan, S.H., Wang, X., Elgendy, O.A.: Plug-and-play ADMM for image restoration: fixed-point convergence and applications. IEEE Trans. Comput. Imaging 3(1), 84–98 (2016)
Choi, I., Kim, M., Gutierrez, D., Jeon, D., Nam, G.: High-quality hyperspectral reconstruction using a spectral prior. Technical report (2017)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Feng, Y.Z., Sun, D.W.: Application of hyperspectral imaging in food safety inspection and control: a review. Crit. Rev. Food Sci. Nutr. 52(11), 1039–1058 (2012)
Fu, Y., Lam, A., Sato, I., Sato, Y.: Adaptive spatial-spectral dictionary learning for hyperspectral image restoration. Int. J. Comput. Vision 122(2), 228–245 (2017)
Fu, Y., Zheng, Y., Sato, I., Sato, Y.: Exploiting spectral-spatial correlation for coded hyperspectral image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3727–3736 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, H., Nie, G., Zheng, Y., Fu, Y.: Image restoration from patch-based compressed sensing measurement. Neurocomputing 340, 145–157 (2019)
Lai, Z., Wei, K., Fu, Y.: Deep plug-and-play prior for hyperspectral image restoration. Neurocomputing 481, 281–293 (2022)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. ar**v preprint ar**v:1509.02971 (2015)
Hohmann, M., et al.: In-vivo multispectral video endoscopy towards in-vivo hyperspectral video endoscopy. J. Biophotonics 10(4), 553–564 (2016)
Ono, S.: Primal-dual plug-and-play image restoration. IEEE Signal Process. Lett. 24(8), 1108–1112 (2017)
Plaza, A., et al.: Recent advances in techniques for hyperspectral image processing. Remote Sens. Environ. 113, S110–S122 (2009)
Rick Chang, J., Li, C.L., Poczos, B., Vijaya Kumar, B., Sankaranarayanan, A.C.: One network to solve them all-solving linear inverse problems using deep projection models. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5888–5897 (2017)
Tirer, T., Giryes, R.: Image restoration by iterative denoising and backward projections. IEEE Trans. Image Process. 28(3), 1220–1234 (2018)
Wagadarikar, A., John, R., Willett, R., Brady, D.: Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47(10), B44-51 (2008)
Wald, L.: Quality of high resolution synthesised images: is there a simple criterion? In: Proceedings of Third Conference Fusion of Earth Data: Merging Point Measurements, Raster Maps and Remotely Sensed Images, pp. 99–103. SEE/URISCA (2000)
Wang, L., **ong, Z., Shi, G., Wu, F., Zeng, W.: Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE Trans. Pattern Anal. Mach. Intell. 39(10), 2104–2111 (2017)
Wang, L., **ong, Z., Gao, D., Shi, G., Wu, F.: Dual-camera design for coded aperture snapshot spectral imaging. Appl. Opt. 54(4), 848–858 (2015)
Wang, L., **ong, Z., Gao, D., Shi, G., Zeng, W., Wu, F.: High-speed hyperspectral video acquisition with a dual-camera architecture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4942–4950 (2015)
Wang, L., Zhang, T., Fu, Y., Huang, H.: Hyperreconnet: joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging. IEEE Trans. Image Process. 28(5), 2257–2270 (2018)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wei, K., Fu, Y.: Low-rank Bayesian tensor factorization for hyperspectral image denoising. Neurocomputing 331, 412–423 (2019)
Wei, K., Fu, Y., Huang, H.: 3-D quasi-recurrent neural network for hyperspectral image denoising. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 363–375 (2020)
**ong, Z., Shi, Z., Li, H., Wang, L., Liu, D., Wu, F.: HSCNN: CNN-based hyperspectral image recovery from spectrally undersampled projections. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 518–525 (2017)
Yamaguchi, M., et al.: High-fidelity video and still-image communication based on spectral information: natural vision system and its applications. In: Proceedings of Spectral Imaging: Eighth International Symposium on Multispectral Color Science, vol. 6062, pp. 129–140. SPIE (2006)
Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grants No. 62171038, No. 61827901, and No. 62088101.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fu, Y., Zhang, Y. (2022). Reinforcement Learning Based Plug-and-Play Method for Hyperspectral Image Reconstruction. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13604. Springer, Cham. https://doi.org/10.1007/978-3-031-20497-5_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-20497-5_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20496-8
Online ISBN: 978-3-031-20497-5
eBook Packages: Computer ScienceComputer Science (R0)