Abstract
String Prediction (SP) is a very efficient screen content coding (SCC) tool. In SP, the self-referencing string plays an important role to improve coding efficiency. But general self-referencing string has the problem of very low pixel copying throughput and is prohibited in the non-self-referencing based SP which has been adopted in the third-generation Audio Video Standard (AVS3). To overcome the problem and bring back the coding gain of self-referencing string, a line-based self-referencing string (LSRS) enabled SP technique is proposed. Moreover, to keep the pixel copying throughput and coding complexity of LSRS enabled SP the same as non-self-referencing based SP, an unbroken-line decomposition algorithm is presented to decompose an LSRS into multiple non-self-referencing strings. In this way, LSRS can be treated in the same way as a non-self-referencing string with the best trade-off between coding efficiency and complexity. Compared with non-self-referencing based SP, using AVS3 reference software HPM, for twelve SCC common test condition YUV test sequences in text and graphics with motion category and mixed content category, the proposed LSRS technique achieves the average Y BD-rate reduction of 0.81% and 0.59% as well as the maximum Y BD-rate reduction of 2.04% and 1.31% for All Intra and Low Delay configurations, respectively, with almost no additional encoding and decoding complexity. The proposed LSRS enabled SP technique has been adopted in AVS3.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from ftp://47.93.196.121/Public/Seqs/Video/ but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the Audio Video Standard Workgroup of China (http://www.avs.org.cn/english/).
References
Bjøntegaard G (2001) Calculation of average PSNR differences between RD-Curves, ITU-T SG16 Q.6 Document, VCEG-M33, Austin, USA
Chen C, Peng W (2017) Intra Line Copy for HEVC Screen Content Intra-Picture Prediction. IEEE Trans Circuits Syst Video Technol 27(7):1568–1579
Feng S, Hu K, Fan E, Zhao L, Wu C (2021) Kalman filter for spatial-temporal regularized correlation filters. IEEE Trans Image Process 30:3263–3278
Hu K, Ye J, Fan E, Shen S, Huang L, Pi J (2017) A novel object tracking algorithm by fusing color and depth information based on single valued neutrosophic cross-entropy. J Intell Fuzzy Syst 32(3):1775–1786
Li B, Xu J, Wu F (2014). 1-D dictionary mode for screen content coding. In: IEEE Visual Communications and Image Processing Conference, 189–192.
Lin T, Zhang P, Wang S, Zhou K, Chen X (2013) Mixed Chroma Sampling-Rate High Efficiency Video Coding for Full-Chroma Screen Content. IEEE Trans Circuits Syst Video Technol 23(1):173–185
Lin W, He X, Dai W, See J, Shinde T, **ong H, Duan L (2020) Key-point sequence lossless compression for intelligent video analysis. IEEE MultiMedia 27(3):12–22
Lin W, Shinde T, Dai W, Liu M, He X, Tiwari A, **ong H (2020) Adaptive lossless compression of skeleton sequences. Signal Process Image Commun 80:115679 1–14
Liu Y, Fang C, Sun J, Huang X (2019) Fast Palette Mode Decision Methods for Coding Game Videos With HEVC-SCC. IEEE Trans Circuits Syst Video Technol 29(10):3061–3067
Ma S, Zhang X, Jia C, Zhao Z, Wang S, Wang S (2020) Image and Video Compression with Neural Networks: A Review. IEEE Trans Circuits Syst Video Technol 30(6):1683–1698
Ma Z, Wang W, Xu M et al Advanced Screen Content Coding Using Color Table and Index Map. IEEE Trans Image Process 23(10):4399–4412
Nguyen T, Xu X, Henry F et al (2021) Overview of the screen content support in VVC: applications, coding tools, and performance. IEEE Trans Circuits Syst Video Technol 31:3801–3817. https://doi.org/10.1109/TCSVT.2021.3074312
Peng W, Walls F, Cohen R et al (2016) Overview of screen content video coding: technologies, standards, and beyond. IEEE J Emerg Sel Top Circuits Syst 6(4):393–408
Wang S, Lin T (2014) United coding method for compound image compression. Multimed Tools Appl 71(3):1263–1282
Wang Y, Zhou Q, Zhao L et al (2020) SCC: Unit Basis Vector String for String Prediction, AVS M5994, China
Wang Y, Xu X, Liu S (2021) Low complexity implementation of intra string copy in AVS3. In: IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 1-4
**ao W, Shi G, Li B, Xu J, Wu F (2018) Fast Hash-Based Inter-Block Matching for Screen Content Coding. IEEE Trans Circuits Syst Video Technol 28(5):1169–1182
**ao W, Bin L, Xu J, Shi G, Wu F (2018) Weighted Rate-Distortion Optimization for Screen Content Coding. IEEE Trans Circuits Syst Video Technol 28(2):499–512
Xu X (2020) AVS3-P2 Common Test Conditions for screen content coding, AVS Document, AVS M5928, China
Xu X, Liu S Overview of screen content coding in recently developed video coding standards. IEEE trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2021.3064210
Xu X, Liu S, Chuang T et al (2016) Intra block copy in HEVC screen content coding extensions. IEEE J Emerg Sel Top Circuits Syst 6(4):409–418
Yang Y, Zhou K, Zhao L, Lin T (2021) An ultralow complexity string matching approach to screen content coding in AVS3[J]. IEEE Trans Circuits Syst Video Technol 31(9):3714–3718
Yang Y, Lin T, Zhao L, Zhou K, Wang S (2021) A string matching based ultra-low complexity lossless screen content coding technique. Multimed Tools Appl 81:2043–2063. https://doi.org/10.1007/s11042-021-11418-6
Zhao L, Lin T, Zhou K, Wang S, Chen X (2016) Pseudo 2D String Matching Technique for High Efficiency Screen Content Coding. IEEE Trans Multimedia 18(3):339–350
Zhao L, Lin T, Zhou K (2017) An efficient ISC offset parameter coding algorithm in screen content coding. Chin J Comput 40(5):1–11
Zhao L, Zhou K, Guo J, Wang S, Lin T (2018) A Universal String Matching Approach to Screen Content Coding. IEEE Trans Multimedia 20(4):796–809
Zhao L, Zhou K, Lin T (2018) Pixel string matching for full-chroma screen and mixed content coding in AVS2. Chin J Comput 41(11):2482–2495
Zhao L, Zhou K, Lin T et al (2019) A universal string prediction approach and its application in AVS2 mixed content coding. Chin J Comput 42(41):1–15
Zhao L, Lin T, Guo J et al (2019) Universal string prediction-based inter coding algorithm optimization in AVS2 mixed content coding. Chin J Comput 42(28):1–13
Zhao L, Lin T, Zhang D, Zhou K, Wang S (2020) An Ultra-Low Complexity and High Efficiency Approach for Lossless Alpha Channel Coding. IEEE Trans Multimedia 22(3):786–794
Zhou K, Zhao L, Lin T (2016) A hardware decoder architecture for general string matching technique. IEEE J Emerg Sel Top Circuits Syst 6(4):560–572
Zhou K, Zhao L, Lin T (2018) A flexible and uniform string matching technique for general screen content coding. Multimed Tools Appl 77:23751–23775
Zhou Q, Zhao L, Wang H et al (2020) CE_SCC_related: Encoding and Decoding Optimization for String Vector based on the Inherent Characteristics of Several Above Strings. AVS M5950, China
Zhou Q, Zhao L, Zhou K, Lin T, Wang H, Wang S, Jiao M (2021) String prediction for 4:2:0 format screen content coding and its implementation in AVS3. IEEE Trans Multimedia 23:3867–3876
Zou F, Chen Y, Karczewicz M, Seregin V (2015) Hash based intra string copy for HEVC based screen content coding. In: ICMEW, pp. 1–4.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. 62271321, No. 61871289), the Science and Technology Plan Project in Basic Public Welfare class of Shaoxing city (No. 2022A11002), the Social Sciences and Humanities Youth Foundation of Ministry of Education (No. 21YJCZH039), Zhejiang Provincial Postdoctoral Science Foundation No. ZJ2022066, Zhejiang Provincial Natural Science Foundation of China (No. LTY22F020003), the Ministry of Education Industry University Cooperation Collaborative Education Project (No. 202101011025), and the Humanities and Social Sciences Project of Shaoxing University (No. 2021LJ001).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there are no conflicts of interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhao, L., Zhou, Q., Hu, K. et al. Line-based self-referencing string prediction technique for screen content coding in AVS3. Multimed Tools Appl 82, 23693–23708 (2023). https://doi.org/10.1007/s11042-023-14673-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-14673-x