SpringerLink
Log in

QoE optimized adaptive streaming method for 360°virtual reality videos over MEC-assisted network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As well-known, virtual reality (VR) video covers the entire 360° scene, while the best visual angle of the human eyes is about 120°, and the Field of View (FoV) supported by VR terminal is only 90 ° ~ 110°. Therefore, the viewpoint-aware adaptive streaming methods have attracted more and more attention in the research of for VR video transmission over the Internet. In this context, to further improve the overall QoE of all users, this paper first design a reasonable, effective and complete system framework for tile-based 360° VR video to achieve the centralized optimization approach on viewpoint-aware adaptive streaming over future Mobile Edge Computing (MEC) assisted 5G heterogeneous networks. And then, based on the video quality, quality fluctuation as well as the error characteristics of the predicted viewpoints, a novel priority-based weighted QoE assessment method is presented to measure the contribution of each tile on the user’s QoE. Besides, the end-to-end delay of each tile is also estimated under the guidance of the optimized MEC-server assignment. On these bases, we finally formulate a novel adaptive optimization problem with the aim of maximizing the QoE of the tiles that delivers to all users within the delay constraint and solve it by develo** a low-complexity Lagrange Relaxation (LR) based heuristic algorithm. The simulation results show that our algorithm outperforms the others in the overall QoE of all users, i.e., increasing video quality by at least 3.888dB and decreasing quality fluctuation by at least 1.7477 dB under the premise of guaranteeing smooth playback.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Huang W, Ding L, Zhai G, Min X, Hwang JN, Xu Y et al (2019) Utility-oriented resource allocation for 360-degree video transmission over heterogeneous networks [J]. Digital Signal Process 84:1–14

    Article  Google Scholar 

  2. Choi K, Vladyslav Z, Choi M et al (2016) On 2d representation format of panoramic video [C], IEEE1857.9-04-M1027, Guiyang, 1-6

  3. Corbillon X, Devlic A, Simon G et al (2017) Viewport-adaptive navigable 360-degree video delivery [C], Proceedings of IEEE International Conference on Communications (ICC), 1-7

  4. Hu Q, Zhou J, Zhang X et al (2021) Viewport-adaptive 360-degree video coding[J]. Multimed Tools Appl 79(17-18):12205–12226

    Article  Google Scholar 

  5. End-to-end optimizations for dynamic streaming [EB/OL], https://code.facebook.com/posts/6375617964280 84

  6. ISO/IEC JTC1/SC29/WG11/M, VR/360 video truncated square pyramid geometry for omaf [S], 2016

  7. Corbillon X, Devlic A, Simon G, Chakareski J (2017) Optimal set of 360-degree videos for viewport-adaptive streaming. Proc. ACM Multimedia Conf., 943-951

  8. Hosseini M, Swaminathan V (2016) Adaptive 360 VR video streaming: divide and conquer [C]. IEEE International Symposium on Multimedia (ISM), USA

  9. Ochi D, Kunita Y, Kameda A, Kojima A, Iwaki S (2015) Live streaming system for omnidirectional video [J], In IEEE Virtual Reality (VR)

  10. Feuvre J, Concolato C (2016) Tiled-based adaptive streaming using MPEG-DASH [C], Proceedings of the 7th ACM International Conference on Multimedia Systems (MMSys ’16), 1-3

  11. Qian F, Ji L, Han B, Gopalakrishnan V (2016) Optimizing 360 video delivery over cellular networks [C], Proceedings of the 5th Workshop on All Things Cellular: Operations, Appl Challenges. ACM, 1-6

  12. Petrangeli S, Swaminathan V (2017) An http/2-based adaptive streaming framework for 360° virtual reality videos [C]”, Proceedings of the 25th ACM International Conference on Multimedia, October 23–27, 1-9

  13. Ghosh A, Aggarwal V, Qian F (2017) A rate adaptation algorithm for tile-based 360-degree video streaming, Apr. [online] Available: http://ar**v:1704.08215

  14. Cheng Q et al (2022) Design and Analysis of MEC- and Proactive Caching-based 360 Mobile VR Video Streaming [J]. IEEE Trans Multimed 24:1529–1544

    Article  Google Scholar 

  15. Liu YW, Liu JX, Argyriou A et al (2019) MEC-Assisted Panoramic VR Video Streaming Over Millimeter Wave Mobile Networks[J]. IEEE Trans Multimed 21(5):1302–1316

    Article  Google Scholar 

  16. Du JB et al (2020) MEC-Assisted Immersive VR Video Streaming Over Terahertz Wireless Networks: A Deep Reinforcement Learning Approach [J]. IEEE Internet Things J 7(10):9517–9529

    Article  Google Scholar 

  17. Zhang X, Hu X, Zhong L, Shirmohammadi S, Zhang L (2020) Cooperative tile-based 360-degree panoramic streaming in heterogeneous networks using scalable video coding [J]. IEEE Trans Circuits Syst Video Technol 30(1):217–231

    Article  Google Scholar 

  18. Yang MY, Liang H, Yang FZ (2021) Real-Time Adaptive Switching Mechanism Towards Viewport-Adaptive Omnidirectional Video Streaming[C], IEEE International Conference on Multimedia & Expo Workshops (ICMEW)

  19. Nguyen TC, Yun JH (2018) Predictive Tile Selection for 360-Degree VR Video Streaming in Bandwidth- Limited Networks [J]. IEEE Commun Lett 22(9):1858–1861

    Article  Google Scholar 

  20. Nguyen DV, Tran HT, Thang TC (2019) Adaptive tiling selection for viewport adaptive streaming of 360-degree video. IEICE Trans Inf Syst 102(1):48–51

    Article  Google Scholar 

  21. Nguyen DV, Tran HTT, Pham AT, Thang TC (2019) An optimal tile-based approach for viewport-adaptive 360-degree video streaming. IEEE J Emerg Sel Topics Circuits Syst 9(1):29–42

    Article  Google Scholar 

  22. Ban YX, **e L, Xu Z et al (2018) CUB360: Exploiting Cross-Users Behaviors for Viewport Prediction in 360 Video Adaptive Streaming [C], Proceedings of IEEE International Conference on Multimedia and Expo (ICME), 1-6

  23. **e L, Xu Z, Ban Y et al (2017) 360ProbDASH: improving QoE of 360 video streaming using tile-based http adaptive streaming [C], Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA, October 23–27

  24. **ao MB, Zhou C, Swaminathan V et al (2018) BAS-360°: Exploring Spatial and Temporal Adaptability in 360-degree Videos over HTTP/2 [C], Proceedings of IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, 953 – 961

  25. da Costa Filho RIT, Luizelli MC, Vega MT, van der Hooft J, Petrangeli S, Wauters T et al. (2018) Predicting the performance of virtual reality video streaming in mobile networks [C], Proc. of the ACM Multimedia Systems Conf, 270-283

  26. Han Y, Ma Y et al (2019) QoE Oriented Adaptive Streaming Method for 360° Virtual Reality Videos [C], 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), 1655-1659

  27. Quan W, Pan Y, **ang B, Zhang L (2020) Reinforcement Learning Driven Adaptive VR Streaming with Optical Flow Based QoE [C], Proceedings of IEEE 18th International Conference on Industrial Informatics (INDIN), 231-236

  28. Kan N, Zou J, Tang K, Li C, Liu N, **ong H (2019) Deep reinforcement learning-based rate adaptation for adaptive 360-degree video streaming, Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), 4030-4034

  29. **ao G, Chen X, Wu M, Zhou Z (2019) Deep reinforcement learning-driven intelligent panoramic video bitrate adaptation, Proc. ACM Turing Celebration Conf. China, 1-5

  30. Zhang Y, Zhao P, Bian K, Liu Y, Song L, Li X (2019) DRL360: 360-degree video streaming with deep reinforcement learning, Proc. IEEE Conf. Comput. Commun. (IEEE INFOCOM), 1252-1260

  31. Wei W, Han J et al (2021) MP-VR: An MPTCP-Based Adaptive Streaming Framework for 360-degree Virtual Reality Videos [C], Proceedings of IEEE International Conference on Communications, 1-6. https://doi.org/10.1109/ICC42927.2021.9500817

  32. Deng R (2022) Optimized resource allocation for multipath cooperative video transmission over MEC-assisted 5G heterogeneous networks[J]. Multimed Tools Appl 81(28):40135–40157

    Article  Google Scholar 

  33. Hu YC, Patel M, Sabella D et al Mobile edge computing a key technology towards 5G[EB/OL], http://www.etsi.org/images/files/ETSIWhitePapers/etsi_wp11_mec_a_keytechnology_towar_ds_5g.pdf

  34. Petrangeli S, Swaminathan V, Hosseini M, De Turck F (2017) An http/2-based adaptive streaming framework for 360 virtual reality videos, Proceedings of the 25th ACM international conference on Multimedia, 306-314

  35. Zink M, Sitaraman R, Nahrstedt K (2019) Scalable 360° video stream delivery: Challenges, solutions, and opportunities. Proc IEEE 107(4):639–650

    Article  Google Scholar 

  36. Standard ISO/IEC 23009-1, “MPEG-DASH (Dynamic Adaptive Streaming Over HTTP),” Accessed on: Aug. 26, 2020. [Online]. Avilable:https://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema _files/

  37. 3GPP TS 38.214, NR; Physical layer procedures for data

  38. 3GPP TS 38.331, NR; Radio Resource Control (RRC) protocol specification

  39. Battle L, Chang R, Stonebraker M (2016) Dynamic prefetching of data tiles for interactive visualization [C], ACM Intl Conf Manage Data, 1363-1375

  40. Xu M, Song YH, Wang JY et al (2018) Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach [J], IEEE Trans. Pattern Analysis and Machine Intelligence

  41. Fan C, Lee J, Lo W et al (2017) Fixation prediction for 360° video streaming in head-mounted virtual reality [C], Proceedings of the 27th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, USA, 67-72

  42. Dodge S, Karam L (2018) Visual saliency prediction using a mixture of deep neural networks. IEEE Trans Image Process 27(8):4080–4090

    Article  MathSciNet  Google Scholar 

  43. Tang ZH, Luo YD, Jiang HB, Qin TF (2017) Motion saliency detection for compressed video[J], Journal of Electronic Imaging, 26(05)

  44. Nam H, Kim KH, Schulzrinne H (2016) QoE matters more than QoS: Why people stop watching cat videos, IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, 1-9

  45. Dimopoulos G, Leontiadis I, Barlet-Ros P, Papagiannaki K (2016) Measuring Video QoE from Encrypted Traffic, ACM Proceedings of the Internet Measurement Conference (IMC ’16)

  46. Paudyal P, Battisti F, Carli M (2016) Impact of video content and transmission impairments on quality of experience, Multimed Tools Appl

  47. Ge X, Pan LH (2017) Multipath cooperative communications networks for augmented and virtual reality transmission [J]. IEEE Trans Multimed 19(10):2345–2358

    Article  Google Scholar 

  48. Wang Q, Liu G, Wang H, Chen Y, Deng R, He L, Liu Z (2014) A Multiuser Simulation System for Video Transmission over HSDPA, Springer Multimed Tools Appl 3105-3142

  49. Corbillon X, Simone F, Simon G (2017) 360-degree video head movement dataset [C], In: Proc. of ACM on Multimedia Syst, 99–204

  50. Deng R (2020) DASH based video caching in MEC-assisted heterogeneous networks[J]. Multimed Tools Appl 79(5):21073–21094

    Article  Google Scholar 

  51. He J, Qureshi M, Qiu L, Li J, Li F, Han L (2018) Rubiks: practical 360-degree streaming for smartphones [J], In: Proceedings of the 16th ACM Annual International Conference Mobile System Application and Service, 482–494

  52. Wang Z, Li F (2021) Convolutional neural network based low complexity HEVC intra encoder[J]. Multimed Tools Appl 80:2441–2460

    Article  Google Scholar 

  53. Zhang H, Li F, Yan Z (2022) A novel transmission approach based on video content for 360-degree streaming[J]. Multimed Tools Appl 81:34067–34085

    Article  Google Scholar 

  54. Zhang G et al (2016) Fundamentals of Heterogeneous Backhaul Design–Analysis and Optimization. IEEE Trans Commun 64(2):876–889

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by National Natural Science Foundation of China (61901250).

Author information

Authors and Affiliations

  1. School of Journalism and Communication, Shaanxi Normal University, **’an, China

    Rui Deng

Authors
  1. Rui Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rui Deng.

Ethics declarations

Competing interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, R. QoE optimized adaptive streaming method for 360°virtual reality videos over MEC-assisted network. Multimed Tools Appl 83, 52737–52761 (2024). https://doi.org/10.1007/s11042-023-17528-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17528-7

Keywords

Search

Navigation