Abstract
To improve the rate-distortion performance of block-based image compressive sensing and reduce the influence of image edges, this paper proposes a measurement-domain spiral predictive coding method, which can make full use of the intrinsic spatial relationship of natural images. For the measurements of each compressive-sensing block, the optimal measurement prediction is selected from a set of measurement prediction candidates that are generated by eight possible directional prediction modes. Then, the resulting residual is processed by scalar quantization. The block prediction starts from the center of an image and spreads around in spiral scanning, where each block can select the optimal measurement prediction from the measurement prediction candidates as many as possible. The experimental results show that the proposed method can achieve the best rate-distortion performance as compared with the existing methods, while the complexity is basically similar.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
In recent years, there has been significant interest in the paradigm of compressive sensing (CS) [1, 2], which is an emerging technique for sparse or compressible signals. A small amount of measurement data can be obtained by using linear random projecting, and then the signal can be reconstructed with high probability by the non-linear optimization. To achieve the perfect reconstruction, some block-based compressive sensing (BCS) methods have been proposed [3,4,5,6,7], which usually utilize the BCS smoothed projected Landweber (BCS-SPL) algorithm to reconstruct an original image [8].
Many literatures assume that the measurement process is effectuated within the sensing device, wherein lower-dimensional measurements are obtained with respect to high-dimensional data. Accordingly, the measurement process can be regarded as conducting data acquisition and data compression simultaneously. However, the measurement process does not bring a real compression in the strict information theoretic sense, because it cannot produce a bitstream from the sensing device directly, which only can be seen as a technology of dimensionality reduction in essence [9]. The real compressive sensing is the process that produces an ultimate compressed bitstream from the input data, which needs to be cooperated with quantization and entropy coding modules. Without effective quantization process, the image compressive sensing is found inefficient in rate-distortion performance [10]. As a result, much attention has been devoted to the improvement of rate-distortion performance of quantized compressive sensing [11,12,13], which appears very complicated and is not applicable to the BCS framework. Therefore, one straightforward solution is to simply apply the scalar quantization (SQ) to the measurements acquired by the sensing device.
Inspired by the success of the block-based hybrid video coding, the intra prediction coding technology can be also used in the BCS measurement process. Mun and Fowler [14] proposed the block-based quantized compressive sensing of natural images with differential pulse-code modulation (DPCM), coupled with uniform scalar quantization, where the previous reconstructed measurement is taken as the candidate of the current measurement and subtracted from the current measurement. Instead of applying quantization directly to each block of BCS measurements, the resulting residual is then scalar-quantized. The simple DPCM-plus-SQ method provides surprisingly competitive rate-distortion performance. However, it is not efficient enough to leverage previous measurement as the prediction of the current measurement, because the non-stationarity of natural images is ignored. Zhang et al. [15] extended the DPCM-based BCS measurement coding and proposed the spatially directional predictive coding (SDPC), where the intrinsic spatial correlation between neighboring measurements of natural images are further explored. For each compressive sensing block, the optimal measurement prediction is selected from a set of multiple measurement prediction candidates, which are generated by four directional prediction modes. Unlike SDPC, Li et al. [16] introduced a median filter predictive quantization (MFPQ) method and took the median of several reconstructed adjacent blocks around the current measurement as the optimal measurement prediction of the current measurement. Although the R-D performance is slightly lower than that of SDPC, the computational complexity of MFPQ is also lower than that of SDPC, and better error resilience is achieved. Compared with DPCM, the SDPC method obtains accurate measurement prediction with better rate-distortion performance, but it also increases computational complexity. Another problem is that Refs. [15, 16] are affected by image edge. For the measurements of image edge, only the measurements from their vertical direction, that is, the previous measurement is effectively utilized. What’s more, the existing methods only consider the measurement-domain prediction of maximum four directional modes, not all possible directional modes. In theory, there are eight directional modes in the current measurement. Zhou et al. [17] further proposed an angular measurement intra prediction with compressive sensing, and designed more structural rows in random measurement matrix for embedding more accurate adjacent boundary information, but it requires far large computation due to its pixel-based operation.
To overcome the shortcomings of the aforementioned methods, this paper proposes a measurement-domain spiral predictive coding (SPC) method. The proposed SPC-plus-SQ method starts from the central block of an image and continues to predict the next block in spiral scanning order. For each current measurement, the optimal measurement prediction is selected from a set of multiple measurement prediction candidates generated by at most eight direction prediction modes, which is more accurate than the aforementioned methods. The proposed SPC-plus-SQ method is not affected by image edge simultaneously. This process is shown in Fig. 1, and the experimental results will verify that the proposed method can achieve the best rate-distortion performance as compared with the SQ alone, DPCM-plus-SQ, SDPC-plus-SQ and MFPQ-plus-SQ methods. In addition, its computational complexity is also almost similar with the SDPC-plus-SQ method.
This paper is organized as follows. Section 2 briefly reviews the BCS framework. The details of proposed method are given in Sect. 3. Experimental results are shown in Sect. 4, and conclusions are drawn in Sect. 5.
2 The BCS Framework
For real signal \( y \in \Re^{N} \), if it is \( k \)-sparse in orthogonal transformation domain \( \Psi \), then
where \( \uptheta \) is a sparse coefficient vector, and real signal \( {\text{y}} \) is sparse under the domain Ψ. The measurement matrix \( \Phi = \left[ {\varphi_{1}^{T} ,\varphi_{2}^{T} , \ldots \varphi_{i}^{T} , \ldots ,\varphi_{M}^{T} } \right] \) is used to measure the signal \( {\text{y}} \) and the measurement vector \( {\text{x}} \in {\Re }^{M} \) is obtained, where \( \varphi_{i}^{T} \in \Re^{N} \) is the row vector, \( k < M \ll N \). Then, the measurement equation can be expressed as
where the signal \( {\text{x}} \) can be reconstructed accurately by the non-linear optimization calculation.
In order to avoid the large storage cost of measurement matrix and the rapid increase of sampling/recovery complexity as the size of sensing signal increases in compressive imaging, a BCS framework of 2D image was proposed, wherein the sampling of an image is driven by random matrices applied on a block-by-block basis. That is, an image \( {\text{y}} \) is firstly divided into \( n \) non-overlapped blocks of size \( B \times B \) with each block denoted by \( {\text{y}}^{\left( i \right)} \in {\Re }^{{B^{2} }} ,i = 1,2, \ldots ,n \) in vector representation along the vertical or horizontal scan order. Then, its corresponding measurement \( {\text{x}}^{\left( i \right)} \) is obtained by
where \( x^{\left( i \right)} \in \Re^{{M_{B} }} \) and \( \Phi_{B} \) is an \( M_{B} \times B^{2} \) measurement matrix such that the subrate for the image as a whole remains \( S = M_{B} /B^{2} . \) It is straightforward to conclude that \( \Phi_{B} \) applied to an image at block level is equivalent to a whole image measurement matrix \( \Phi \) with a constrained structure, namely, \( \Phi \) can be written as a block diagonal with \( \Phi_{B} \) along the diagonal.
3 Spiral Predictive Coding
In an image, each block is not independent with each other, and they have certain spatial correlation with the surrounding blocks. Therefore, it can be inferred that each BCS measurement has some spatial correlation with the measurements around it. Further, Because of the non-stationarity of natural image, correlation coefficients of a measurement are different with its adjacent measurements in different directions. The optimal measurement prediction should be selected from the adjacent reconstructed measurements set for the current measurement. Actually, if the current measurement has more measurement prediction candidates to be chosen, the optimal measurement prediction is closer to the current measurement. This is basic idea of spiral prediction coding for BCS. The details are as given below.
As shown in Fig. 1, an input natural image \( {\text{y}} \) is first divided into n non-overlap** blocks, and the block size is \( B \times B \), and each block is denoted by \( y^{\left( i \right)} \in \Re^{{B^{2} }} ,i = 1,2, \ldots ,n \) in vector. The proposed method is that the first block begins in the center of the image, which is denoted by \( y^{{\left( {n/2} \right)}} \) and is sampled by the BCS measurement matrix in spiral scanning order. Then, all BCS measurements are acquired by Eq. (3). Next, for measurements which have just been acquired by BCS measurement matrix, each measurement is denoted by \( {\text{x}}^{\left( i \right)} \in {\Re }^{{M_{B} }} ,i = 1,2, \ldots ,n \) in vector. The proposed method fully considers all eight directional prediction modes from its neighboring BCS measurement which have already been reconstructed, namely, up, up-right, up-left, right, left, down, down-right, and down-right. More specially, let \( \tilde{x}_{A}^{\left( i \right)} \), \( \tilde{x}_{B}^{\left( i \right)} \), \( \tilde{x}_{C}^{\left( i \right)} \tilde{x}_{D}^{\left( i \right)} \), \( \tilde{x}_{E}^{\left( i \right)} \), \( \tilde{x}_{F}^{\left( i \right)} \), \( \tilde{x}_{G}^{\left( i \right)} \), and \( \tilde{x}_{H}^{\left( i \right)} \) denote the up, up-right, up-left, right, left, down, down-right, and down-right blocks of measurements with regard to \( {\text{x}}^{\left( i \right)} \) respectively, as illustrated in Fig. 2.
It is worth noting that the deficiency of the SDPC-plus-SQ method is that there are fewer spatial direction measurement predictions for the blocks of image edge, usually only the previous measurement prediction. The proposed method starts from the center block and proceeds in the spiral scanning order. Even in the edge blocks of the image, there are more spatial direction measurement predictions to be chosen. Furthermore, the proposed method takes into all-direction measurement predictions around current measurement as the measurement prediction candidates set. Therefore, the proposed method may perform better than the existing methods.
Here defines a set of measurement prediction candidates as follow:
where \( R \) is a collection of above eight directional prediction modes. The measurement prediction candidates in the sets are reconstructed measurements of previously non-overlapped blocks. For the current measurement, the optimal measurement prediction denoted by \( \hat{x}_{P}^{\left( i \right)} \), is determined by minimizing the residual between the current measurement \( {\text{x}}^{\left( i \right)} \) and the measurement prediction candidates in the sets.
Here, \( \left\| * \right\|_{{\ell_{1} }} \) is \( \ell_{1} \) norm which adds all the absolute values of the entries in a vector.
After obtaining the optimal measurement prediction \( \hat{x}_{P}^{\left( i \right)} \), the residual can be calculated by \( d^{\left( i \right)} = {\text{x}}^{\left( i \right)} - \hat{x}_{P}^{\left( i \right)} \). Following, the residual \( d^{\left( i \right)} \) is quantized by a uniform scalar quantization with quantization step q to obtain the quantization index \( s^{\left( i \right)} \).
Then, the quantization index \( s^{\left( i \right)} \) is encoded by the entropy encoder to further reduce its statistical redundancy and transmitted to the decoder at the receiver. Continue to the encoder, the quantized residual \( d^{\left( i \right)} \) is obtained by inverse quantization of the quantization index \( s^{\left( i \right)} \), then residual \( d^{\left( i \right)} \) is added to the optimal measurement prediction \( \hat{x}_{P}^{\left( i \right)} \) just obtained, and the sum is used as the measurement prediction candidate of the adjacent direction mode for the next measurement.
The bitstream to be transmitted in Fig. 1 consists of two parts: the standard bits of the best prediction direction mode which is used to inform the decoder of the selected measurements prediction and the bitstream data obtained by encoding the quantization index \( s^{\left( i \right)} \) with the entropy encoder. This process is applied for all BCS measurements to achieve the final bitstream.
At the decoder side, contrary to the above encoding process, by the de-quantization on the quantization index from the bitstream, the quantized residuals \( \tilde{d}^{\left( i \right)} \) can be obtained, which is then added by the prediction \( \hat{x}_{P}^{\left( i \right)} \), producing the CS measurements \( \tilde{x}^{\left( i \right)} = \hat{x}_{P}^{\left( i \right)} + \tilde{d}^{\left( i \right)} \). At last, all groups of reconstructed measurements \( \tilde{x} \) are obtained sequentially, which are then utilized for the natural image reconstruction by using the BCS-SPL reconstruction algorithm.
In order to demonstrate the superiority of the proposed method, a correlation coefficient \( \rho \) is used for quantitative comparison with DPCM, SDPC, and MFPQ.
The correlation coefficients can measure and compare the measurement vectors of current measurements and their optimal measurement predictions. For each current measurement, the MFPQ method computes the median value of components in measurement vectors of neighboring measurements as the optimal measurement prediction, the DPCM method uses directly previous measurement as the optimal measurement prediction, and the SDPC method selects the optimal measurement prediction from four directional modes by using the least square error criterion. The proposed method is similar to the SDPC method, selecting the optimal measurement prediction from eight directional modes, but the proposed method is not affected by image edges because of the spiral scanning order. In theory, the proposed method can get the most accurate measurement prediction in these methods due to more measurement prediction candidates to be chosen. It can be verified by Table 1 which presents the average correlation coefficients of DPCM, SDPC, MFPQ and SPC. The correlation coefficients are calculated by Eq. (7) over all 16 × 16 blocks for three 512 × 512 test images with different sampling rates. There is no doubt that the maximum average correlation coefficients are obtained by the proposed method in all methods.
It is worth noting that, as shown in the Table 2, although the proposed method achieves the maximum correlation coefficient among these prediction methods, the computational complexity is even slightly lower than that of SDPC. The computational complexity of MFPQ and DPCM is lower than the proposed method, but their average correlation coefficient is much lower, which will affect the quality of image reconstruction.
4 Experimental Results
The experimental results are provided to verify the performance of the proposed method. The rate-distortion efficiency of the proposed SPC-plus-SQ method is examined by comparing it to SDPC-plus-SQ, DPCM-plus-SQ, MFPQ-plus-SQ and SQ alone applied to BCS measurements. The BCS-SPL algorithm is always exploited to reconstruct the decoded measurements by the above five prediction methods.
In the implementations, two grayscale test images are 512 × 512 pixels, the block size for BCS is typically set to be 16 × 16 and the measurement matrix \( \Phi_{B} \) is an orthogonal random Gaussian matrix. Image quality is evaluated by means of the rate-distortion performance in terms of peak signal-to-noise ratio (PSNR) in dB at different bitrates in bits per pixel (bpp). The actual bitrate is estimated by the entropy of the quantizer indices, which would be actually produced by a real entropy coder. The setup of the combination of step-size and subrate is the same as that in Refs. [14,15,16].
Tables 3 and 4 respectively give the PSNR results of two images (namely Lenna and Peppers) at various bitrates (from 0.6 bpp to 1.6 bpp) under the BCS-SPL algorithm, coupled with five prediction & coding techniques, namely, SQ alone, DPCM-plus-SQ, SDPC-plus-SQ, MFPQ-plus-SQ and SPC-plus-SQ. One easily observes that, the proposed method obtains the best PSNR performance among five methods, and it improves 0.18 dB gain in the first test image and 0.12 dB gain in the second test image on average PSNR as compared to SDPC-plus-SQ relatively, which can demonstrate the superiority of the proposed SPC.
5 Conclusion
A new measurement-domain spiral predictive coding is proposed, which can make full use of the intrinsic spatial relationship of natural images and is not affected by image edges. The direction prediction block in the proposed method starts from the center of an image and spreads around in spiral scanning, where each measurement-domain block can select the optimal measurement prediction from the candidates set as many as possible. The experimental results show that the proposed SPC-plus-SQ method can achieve the best rate-distortion performance as compared with the existing methods.
References
Trevisi, M., Akbari, A., Trocan, M., Rodrmíguez-Vázquez, Á., Carmona-Galán, R.: Compressive imaging using RIP-compliant CMOS imager architecture and Landweber reconstruction. IEEE Trans. Circuits Syst. Video Technol. (2019). https://doi.org/10.1109/TCSVT.2019.2892178
Candes, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: ReconNet: non-iterative reconstruction of images from compressively sensed random measurements. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, pp. 449–458 (2016)
Amit, S.U., Deepthi, P.: Rate-distortion analysis of structured sensing matrices for block compressive sensing of images. Sig. Process. Image Commun. 65, 115–127 (2018)
Liu, Y., Yuan, X., Suo, J.L., Brady, D., Dai, Q.H.: Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell. (2018). https://doi.org/10.1109/TPAMI.2018.2873587
**e, X.M., Wang, C., Du, J., Shi, G.M.: Full image recover for block-based compressive sensing. In: International Conference on Multimedia & Expo, San Diego, pp. 1–6 (2018)
Liu, X.M., Zhai, D.M., Zhou, J.T., Zhang, X.F., Zhao, D.B., Gao, W.: Compressive sampling-based image coding for resource-deficient visual communication. IEEE Trans. Image Process. 25(6), 2844–2855 (2016)
Fowler, J.E., Mun, S., Tramel, E.W.: Block-based compressed sensing of images and video. Found. Trends Sig. Process. 4(4), 297–416 (2012)
Chen, Z., et al.: Compressive sensing multi-layer residual coefficients for image coding. IEEE Trans. Circ. Syst. Video Technol. (2019). https://doi.org/10.1109/TCSVT.2019.2898908
Jacques, L., Hammond, D.K., Fadili, J.M.: Dequantizing compressed sensing: when oversampling and non-Gaussian constraints combine. IEEE Trans. Inf. Theory 57(1), 559–571 (2011)
Sun, J.Z., Goyal, V.K.: Optimal quantization of random measurements in compressed sensing. In: IEEE International Symposium on Information Theory, Seoul, pp. 6–10 (2009)
Wang, L.J., Wu, X.L., Shi, G.M.: Binned progressive quantization for compressive sensing. IEEE Trans. Image Process. 21(6), 2980–2990 (2012)
Ahn, J.H., Jiang, H.: Architecture and noise analysis for block-based compressive imaging. In: International Conference on Image Processing, Athens, pp. 31–35 (2018)
Mun, S., Fowler, J.E.: DPCM for quantized block-based compressed sensing of images. In: 20th European Signal Processing Conference, Bucharest, pp. 1424–1428 (2012)
Zhang, J., Zhao, D.B., Jiang, F.: Spatially directional predictive coding for block-based compressive sensing of natural images. In: IEEE International Conference on Image Processing, Melbourne, pp. 1021–1025 (2014)
Li, R., Liu, H.B., He, W.: Space-time quantization and motion-aligned reconstruction for block-based compressive video sensing. KSII Trans. Internet Inf. Syst. 10(1), 321–340 (2017)
Zhou, J.B., Zhou, J.J., Guo, L.: Angular intra prediction based measurement coding algorithm for compressively sensed image. In: IEEE International Conference on Multimedia & Expo, San Diego, pp. 1–6 (2018)
Acknowledgement
This work was supported by the Natural Science Foundation of Shanghai under Grant 18ZR1400300.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tian, W., Liu, H. (2019). Measurement-Domain Spiral Predictive Coding for Block-Based Image Compressive Sensing. In: Zhao, Y., Barnes, N., Chen, B., Westermann, R., Kong, X., Lin, C. (eds) Image and Graphics. ICIG 2019. Lecture Notes in Computer Science(), vol 11903. Springer, Cham. https://doi.org/10.1007/978-3-030-34113-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-34113-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34112-1
Online ISBN: 978-3-030-34113-8
eBook Packages: Computer ScienceComputer Science (R0)