Abstract
Block compressed sensing (BCS), which is widely used in compressed image sensing (CIS), brings the advantages of lower complexity for sampling and reconstruction. But it will result in redundant information sampling in the smooth block and insufficient information sampling in the texture block, which makes the reconstruction quality of image details poor. To address this problem, we propose a novel Hybrid Sampling and Gradient Attention Network for CIS, dubbed HSGANet. In HSGANet, a new linear sampling strategy, called hybrid BCS (HBCS) sampling is proposed to realize BCS sampling with balanced information entropy for sampling blocks. Specifically, HBCS is composed of the proposed block permutation sampling (BPS) and BCS sampling, where the BPS is used to increase the proportion of texture block information in the image measurement, which is realized by BCS sampling following image block permutation. Furthermore, a selection algorithm is developed to achieve optimal image information balanced permutation. To match HBCS, we design the initial reconstruction fusion sub-network and the deep reconstruction sub-network which is constructed by cascading GA-Blocks with gradient attention sub-network. In each phase, the gradient attention sub-network can achieve pixel-level adaptive fusion of the gradient map obtained by minimizing measurement errors of BPS and BCS. Extensive experimental results show that our HSGANet has a great improvement in reconstruction accuracy than the state-of-the-art methods with a comparable running speed and model complexity.
Similar content being viewed by others
References
Candes, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005). https://doi.org/10.1109/TIT.2005.858979
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). https://doi.org/10.1109/TIT.2006.871582
Duarte, M.F., Davenport, M.A., Takhar, D., Laska, J.N., Sun, T., Kelly, K.F., Baraniuk, R.G.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008). https://doi.org/10.1109/MSP.2007.914730
Liu, Y., Yuan, X., Suo, J., Brady, D.J., Dai, Q.: Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2990–3006 (2019). https://doi.org/10.1109/TPAMI.2018.2873587
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 58(6), 1182–1195 (2007)
Gan, L.: Block compressed sensing of natural images. In: 2007 15th International Conference on Digital Signal Processing, pp. 403–406 (2007). https://doi.org/10.1109/ICDSP.2007.4288604
Chen, C., Tramel, E.W., Fowler, J.E.: Compressed-sensing recovery of images and video using multihypothesis predictions. In: 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), pp. 1193–1198 (2011). https://doi.org/10.1109/ACSSC.2011.6190204
Zhang, J., Zhao, D., Gao, W.: Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23(8), 3336–3351 (2014). https://doi.org/10.1109/TIP.2014.2323127
Zha, Z., Liu, X., Zhang, X., Chen, Y., Tang, L., Bai, Y., Wang, Q., Shang, Z.: Compressed sensing image reconstruction via adaptive sparse nonlocal regularization. Vis. Comput. 34(1), 117–137 (2018)
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: Reconnet: Non-iterative reconstruction of images from compressively sensed measurements. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 449–458 (2016). https://doi.org/10.1109/CVPR.2016.55
Yao, H., Dai, F., Zhang, S., Zhang, Y., Tian, Q., Xu, C.: Dr2-net: Deep residual reconstruction network for image compressive sensing. Neurocomputing 359, 483–493 (2019)
Mousavi, A., Baraniuk, R.G.: Learning to invert: signal recovery via deep convolutional networks. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2272–2276 (2017). https://doi.org/10.1109/ICASSP.2017.7952561
Lohit, S., Kulkarni, K., Kerviche, R., Turaga, P., Ashok, A.: Convolutional neural networks for noniterative reconstruction of compressively sensed images. IEEE Trans. Comput. Imaging 4(3), 326–340 (2018). https://doi.org/10.1109/TCI.2018.2846413
Shi, W., Jiang, F., Zhang, S., Zhao, D.: Deep networks for compressed image sensing. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 877–882 (2017). https://doi.org/10.1109/ICME.2017.8019428
Shi, W., Jiang, F., Liu, S., Zhao, D.: Scalable convolutional neural network for image compressed sensing. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12282–12291 (2019). https://doi.org/10.1109/CVPR.2019.01257
Canh, T.N., Jeon, B.: Difference of convolution for deep compressive sensing. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2105–2109 (2019). https://doi.org/10.1109/ICIP.2019.8803165
Shi, W., Jiang, F., Liu, S., Zhao, D.: Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29, 375–388 (2020). https://doi.org/10.1109/TIP.2019.2928136
Canh, T.N., Jeon, B.: Multi-scale deep compressive imaging. IEEE Trans. Comput. Imaging 7, 86–97 (2021). https://doi.org/10.1109/TCI.2020.3034433
Tian, J., Yuan, W., Tu, Y.: Image compressed sensing using multi-scale residual generative adversarial network. Vis. Comput. 1–10 (2021)
Zhang, J., Ghanem, B.: Ista-net: Interpretable optimization-inspired deep network for image compressive sensing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1828–1837 (2018). https://doi.org/10.1109/CVPR.2018.00196
Zhang, J., Zhao, C., Gao, W.: Optimization-inspired compact deep compressive sensing. IEEE J. Sel. Top. Signal Process. 14(4), 765–774 (2020). https://doi.org/10.1109/JSTSP.2020.2977507
Pei, H., Yang, C., Cao, Y.: Deep smoothed projected landweber network for block-based image compressive sensing. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 2870–2874 (2020). https://doi.org/10.1109/ICIP40778.2020.9191043
Zhang, Z., Liu, Y., Liu, J., Wen, F., Zhu, C.: Amp-net: Denoising-based deep unfolding for compressive image sensing. IEEE Trans. Image Process. 30, 1487–1500 (2021). https://doi.org/10.1109/TIP.2020.3044472
Su, Y., Lian, Q.: ipiano-net: Nonconvex optimization inspired multi-scale reconstruction network for compressed sensing. Signal Processing: Image Communication 89, 115989 (2020)
You, D., **e, J., Zhang, J.: Ista-net++: Flexible deep unfolding network for compressive sensing. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2021). https://doi.org/10.1109/ICME51207.2021.9428249
You, D., Zhang, J., **e, J., Chen, B., Ma, S.: Coast: Controllable arbitrary-sampling network for compressive sensing. IEEE Trans. Image Process. 30, 6066–6080 (2021). https://doi.org/10.1109/TIP.2021.3091834
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594
Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)
Metzler, C.A., Maleki, A., Baraniuk, R.G.: From denoising to compressed sensing. IEEE Trans. Inf. Theory 62(9), 5117–5144 (2016). https://doi.org/10.1109/TIT.2016.2556683
Li, C., Yin, W., Jiang, H., Zhang, Y.: An efficient augmented Lagrangian method with applications to total variation minimization. Comput. Optim. Appl. 56(3), 507–530 (2013)
Du, J., **e, X., Wang, C., Shi, G., Xu, X., Wang, Y.: Fully convolutional measurement network for compressive sensing image reconstruction. Neurocomputing 328, 105–112 (2019)
Lin, M., Chen, Q., Yan, S.: Network in network. ar**v preprint ar**v:1312.4400 (2013)
Mardani, M., Sun, Q., Vasawanala, S., Papyan, V., Monajemi, H., Pauly, J., Donoho, D.: Neural proximal gradient descent for compressive imaging. ar**v preprint ar**v:1806.03963 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011). https://doi.org/10.1109/TPAMI.2010.161
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. ar**v preprint ar**v:1412.6980 (2014)
Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206 (2015). https://doi.org/10.1109/CVPR.2015.7299156
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–4232 (2001). https://doi.org/10.1109/ICCV.2001.937655
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, pp. 711–730 (2010). Springer
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Acknowledgements
This work is supported by the Key Program of Natural Science Foundation of Guangdong Province (2017A030311028) and the Natural Science Foundation of Guangdong Province (2019A1515011949)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, X., Yang, C. & Chen, W. A hybrid sampling and gradient attention network for compressed image sensing. Vis Comput 39, 4213–4226 (2023). https://doi.org/10.1007/s00371-022-02585-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02585-0