SpringerLink
Log in

Tile-size aware bitrate allocation for adaptive 360\(^{\circ }\) video streaming

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

360\(^{\circ }\) videos have become increasingly popular recently, but consume much more bandwidth than non-360\(^{\circ }\) videos. Usually, 360\(^{\circ }\) video streaming partitions the video surface into multiple tiles and encodes the tiles independently to effectively and flexibly use limited link bandwidth. However, current bitrate adaptive algorithms generally aim to maximize the bitrate, rather than perceptual quality, resulting in degradation of user experience. More importantly, we reveal that the distribution of tile size is very skewed, that is, a small number of large tiles consumes more bandwidth than a large number of small tiles, further hurting the overall viewing quality. Therefore, in this paper, we propose a tile-size aware bitrate allocation scheme TSA for adaptive 360\(^{\circ }\) video streaming to improve the viewing experience of users. Specifically, TSA cautiously decreases the quality of a few large tiles to allocate more bandwidth to large number of small tiles, thus improving the perceptual quality of overall video, without sacrificing large tiles excessively. Experiments over real-world datasets show that TSA effectively improves V-VMAF by up to 39% compared with several state-of-the-art adaptive bitrate algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Qian F, Ji L, Han B, Gopalakrishnan V (2016) Optimizing 360 video delivery over cellular networks. In: Proceedings of the Workshop on All Things Cellular: Operations, Applications and Challenges, pp 1–6

  2. Petrangeli S, Swaminathan V, Hosseini M, De Turck F (2017) An http/2-based adaptive streaming framework for 360 virtual reality videos. In: Proceedings of the ACM international conference on Multimedia, pp 306–314

  3. **e L, Xu Z, Ban Y, Zhang X, Guo Z (2017) 360probdash: Improving qoe of 360 video streaming using tile-based http adaptive streaming. In: Proceedings of the ACM international conference on Multimedia, pp 315–323

  4. **e L, Zhang X, Guo Z (2018) Cls: A cross-user learning based system for improving qoe in 360-degree video adaptive streaming. In: Proceedings of the ACM international conference on Multimedia, pp 564–572

  5. Graf M, Timmerer C, Mueller C (2017) Towards bandwidth efficient adaptive streaming of omnidirectional video over http: Design, implementation, and evaluation. In: Proceedings of the ACM on Multimedia Systems Conference, pp 261–271

  6. Dasari M, Bhattacharya A, Vargas S, Sahu P, Balasubramanian A, Das SR (2020) Streaming 360-degree videos using super-resolution. In: Proceedings of the IEEE International Conference on Computer Communications, pp 1977–1986

  7. Qian F, Han B, **ao Q, Gopalakrishnan V (2018) Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices. In: Proceedings of the Annual International Conference on Mobile Computing and Networking, pp 99–114

  8. Chopra L, Chakraborty S, Mondal A, Chakraborty S (2021) Parima: Viewport adaptive 360-degree video streaming. In: Proceedings of the Web Conference, pp 2379–2391

  9. Yadav PK, Ooi WT (2020) Tile rate allocation for 360-degree tiled adaptive video streaming. In: Proceedings of the ACM International Conference on Multimedia, pp 3724–3733

  10. Sodagar I (2011) The mpeg-dash standard for multimedia streaming over the internet. IEEE Multimedia 18(4):62–67

    Article  Google Scholar 

  11. Duanmu Z, Rehman A, Wang Z (2018) A quality-of-experience database for adaptive video streaming. IEEE Trans Broadcast 64(2):474–487

    Article  Google Scholar 

  12. Fan CL, Lee J, Lo WC, Huang CY, Chen KT, Hsu CH (2017) Fixation prediction for 360 video streaming in head-mounted virtual reality. In: Proceedings of the Workshop on Network and Operating Systems Support for Digital Audio and Video, pp 67–72

  13. Yin X, **dal A, Sekar V, Sinopoli B (2015) A control-theoretic approach for dynamic adaptive video streaming over HTTP. In: Proceedings of the ACM conference on SIGCOMM, pp 325–338

  14. Wu C, Tan Z, Wang Z, Yang S (2017) A dataset for exploring user behaviors in VR spherical video streaming. In: Proceedings of the ACM on Multimedia Systems Conference, pp 193–198

  15. FFmpeg. http://ffmpeg.org

  16. 4G/LTE Bandwidth Logs. http://users.ugent.be/~jvdrhoof/dataset-4g/

  17. FCC. https://www.fcc.gov/

  18. Mahimahi. http://mahimahi.mit.edu

  19. Snyder JP (1997) Flattening the earth: two thousand years of map projections. University of Chicago Press

    Google Scholar 

  20. Akhtar Z, Nam YS, Govindan R, Rao S, Chen J, Katz-Bassett E, Zhang H (2018) Oboe: Auto-tuning video ABR algorithms to network conditions. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp 44–58

  21. Niamut OA, Thomas E, D’Acunto L, Concolato C, Denoual F, Lim SY (2016) MPEG DASH SRD: spatial relationship description. In: Proceedings of the International Conference on Multimedia Systems, pp 1–8

  22. **ao M, Zhou C, Liu Y, Chen S (2017) Optile: Toward optimal tiling in 360-degree video streaming. In: Proceedings of the ACM international conference on Multimedia, pp 708–716

  23. Guan Y, Zheng C, Zhang X, Guo Z, Jiang J (2019) Pano: Optimizing 360 video streaming with a better understanding of quality perception. In: Proceedings of the ACM conference on SIGCOMM, pp 394–407

  24. He J, Qureshi MA, Qiu L, Li J, Li F, Han L (2018) Rubiks: Practical 360-degree streaming for smartphones. In: Proceedings of the Annual International Conference on Mobile Systems, Applications, and Services, pp 482–494

  25. Huang T, Zhou C, Zhang RX, Wu C, Yao X, Sun L (2019) Comyco: Quality-aware adaptive video streaming via imitation learning. In: Proceedings of the ACM International Conference on Multimedia, pp 429–437

  26. Jiang J, Sekar V, Zhang H (2012) Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. In: Proceedings of the international conference on Emerging networking experiments and technologies, pp 97–108

  27. Huang TY, Johari R, McKeown N, Trunnell M, Watson M (2014) A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In: Proceedings of the ACM conference on SIGCOMM, pp 187–198

  28. Zhang Y, Zhao P, Bian K, Liu Y, Song L, Li X (2019) DRL360: 360-degree video streaming with deep reinforcement learning. In: Proceedings of IEEE Conference on Computer Communications, pp 1252–1260

  29. Ban Y, Zhang Y, Zhang H, Zhang X, Guo Z (2020) MA360: Multi-agent deep reinforcement learning based live 360-degree video streaming on edge. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp 1–6

  30. Li Z, Zhong P, Huang J, Gao F, Wang J (2023) Achieving QoE fairness in Bitrate Allocation of 360\(^{\circ }\) Video Streaming. IEEE Trans Multimedia 26:1169–1178

    Article  Google Scholar 

  31. Shi J, Pu L, Yuan X, Gong Q, Xu J (2022) Sophon: Super-resolution enhanced 360 video streaming with visual saliency-aware prefetch. In: Proceedings of ACM International Conference on Multimedia, pp 3124–3133

  32. Yaqoob A, Muntean GM (2023) Advanced Predictive Tile Selection Using Dynamic Tiling for Prioritized 360 Video VR Streaming. ACM Trans Multimed Comput Commun Appl 20(1):1–28

    Article  Google Scholar 

  33. Guimard Q, Sassatelli L (2023) SMART360: Simulating Motion prediction and Adaptive bitRate sTrategies for 360\(^{\circ }\) video streaming. In: Proceedings of the Conference on ACM Multimedia Systems, pp 384–390

  34. Li J, Han L, Zhang C, Li Q, Liu Z (2023) Spherical convolution empowered viewport prediction in 360 video multicast with limited FoV feedback. ACM Trans Multimed Comput Commun Appl 19(1):1–23

    Article  Google Scholar 

  35. Li J, Zhang C, Liu Z, Hong R, Hu H (2023) Optimal volumetric video streaming with hybrid saliency based tiling. IEEE Trans Multimedia 25:2939–2953

    Article  Google Scholar 

  36. Chen J, Song Y, Li D, Lin X, Zhou S, Xu W (2024) Specular Removal of Industrial Metal Objects Without Changing Lighting Configuration. IEEE Trans Industr Inf 20(3):3144–3153

    Article  Google Scholar 

  37. Xu H, Li Q, Chen J (2022) Highlight removal from a single grayscale image using attentive GAN. Appl Artif Intell 36(1)

  38. Di Y, Li R, Tian H, Guo J, Shi B, Wang Z, Yan K, Liu Y (2023) A maneuvering target tracking based on fastIMM-extended Viterbi algorithm. Neural Comput & Applic

  39. Cheng D, Chen L, Lv C, Guo L, Kou Q (2022) Light-guided and cross-fusion U-Net for anti-illumination image super-resolution. IEEE Trans Circuits Syst Video Technol 32(12):8436–8449

    Article  Google Scholar 

  40. Zhang H, Liu H, Kim C (2024) Semantic and Instance Segmentation in Coastal Urban Spatial Perception: A Multi-Task Learning Framework with an Attention Mechanism. Sustainability 16(2):833

    Article  Google Scholar 

  41. Pan S, Xu GJ, Guo K, Park SH, Ding H (2023) Video-Based Engagement Estimation of Game Streamers: An Interpretable Multimodal Neural Network Approach. Early Access, IEEE Transactions on Games. https://doi.org/10.1109/TG.2023.3348230

    Book  Google Scholar 

  42. Wu J, Zhu J, Zhang J, Dang P, Li W, Guo Y, Fu L, Lai J, You J, **e Y, Liang C (2023) A dynamic holographic modelling method of digital twin scenes for bridge construction. Int J Digit Earth 16(1):2404–2425

    Article  Google Scholar 

  43. Liu H, Yuan H, Liu Q, Hou J, Zeng H, Kwong S (2021) A hybrid compression framework for color attributes of static 3D point clouds. IEEE Trans Circuits Syst Video Technol 32(3):1564–1577

    Article  Google Scholar 

  44. Liu Q, Yuan H, Hamzaoui R, Su H, Hou J, Yang H (2021) Reduced reference perceptual quality model with application to rate control for video-based point cloud compression. IEEE Trans Image Process 30:6623–6636

    Article  Google Scholar 

  45. Fu C, Yuan H, Xu H, Zhang H, Shen L (2023) TMSO-Net: Texture adaptive multi-scale observation for light field image depth estimation. J Vis Commun Image Represent 90:103731

    Article  Google Scholar 

  46. Sheng H, Wang S, Yang D, Cong R, Cui Z, Chen R (2023) Cross-view recurrence-based self-supervised super-resolution of light field. IEEE Trans Circuits Syst Video Technol 33(12):7252–7266

    Article  Google Scholar 

  47. Shi Y, ** J, Hu D, Cai Z, Xu K (2023) RayMVSNet++: learning ray-based 1D implicit fields for accurate multi-view stereo. IEEE Trans Pattern Anal Mach Intell 45(11):13666–13682

    Google Scholar 

  48. Zheng W, Lu S, Yang Y, Yin Z, Yin L (2024) Lightweight transformer image feature extraction network. PeerJ Computer Science 10:e1755

    Article  Google Scholar 

  49. Yang D, Cui Z, Sheng H, Chen R, Cong R, Wang S, **ong Z (2024) An Occlusion and Noise-aware Stereo Framework Based on Light Field Imaging for Robust Disparity Estimation. IEEE Trans Comput 73(3):764–777

    Article  Google Scholar 

  50. Liu S, Huang S, Xu X, Lloret J, Muhammad K (2023) Efficient Visual Tracking Based on Fuzzy Inference for Intelligent Transportation Systems. IEEE Trans Intell Trans Syst 24(12):15795–15806

    Article  Google Scholar 

  51. Liu S, Wang S, Liu X, Gandomi A, Daneshmand M, Muhammad K, De Albuquerque V (2021) Human Memory Update Strategy: A Multi-Layer Template Update Mechanism for Remote Visual Monitoring. IEEE Trans Multimed 23:2188–2198

    Article  Google Scholar 

  52. Liu S, Wang S, Liu X, Lin C, Lv Z (2021) Fuzzy Detection Aided Real-Time and Robust Visual Tracking Under Complex Environments. IEEE Trans Fuzzy Syst 29(1):90–102

    Article  Google Scholar 

Download references

Acknowledgements

We thank the reviewers for their insightful feedback. This work was supported by the National Natural Science Foundation of China(62302524, 62132022), and the Science and Technology Innovation Program of Hunan Province (2024JJ6531). This work was also carried out in part using computing resources at the High Performance Computing Center of Central South University.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Central South University, Changsha, 410083, Hunan, China

    Jiawei Huang, Mingyue Liu, **gling Liu, Feng Gao & Jianxin Wang

  2. School of Informatics, The University of Edinburgh, Edinburgh, UK

    Weihe Li

Authors
  1. Jiawei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyue Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. **gling Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weihe Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to **gling Liu.

Ethics declarations

Conflicts of interest

The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, J., Liu, M., Liu, J. et al. Tile-size aware bitrate allocation for adaptive 360\(^{\circ }\) video streaming. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19486-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19486-0

Keywords

Search

Navigation