SpringerLink
Log in

Reliving the Dataset: Combining the Visualization of Road Users’ Interactions with Scenario Reconstruction in Virtual Reality

  • Conference paper
  • First Online:
2021 6th International Conference on Intelligent Transportation Engineering (ICITE 2021) (ICITE 2021)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 901))

Included in the following conference series:

Abstract

One core challenge in the development of automated vehicles is their capability to deal with a multitude of complex traffic scenarios with many, hard to predict traffic participants. As part of the iterative development process, it is necessary to detect critical scenarios and generate knowledge from them to improve the highly automated driving (HAD) function. In order to tackle this challenge, numerous datasets have been released in the past years, which act as the basis for the development and testing of such algorithms. Nevertheless, the remaining challenges are to find relevant scenes, such as safety-critical corner cases, in these datasets and to understand them completely.

Therefore, this paper presents a methodology to process and analyze naturalistic motion datasets in two ways: on the one hand, our approach maps scenes of the datasets to a generic semantic scene graph which allows for a high-level and objective analysis. Here, arbitrary criticality measures, e.g. TTC, RSS or SFF, can be set to automatically detect critical scenarios between traffic participants. On the other hand, the scenarios are recreated in a realistic virtual reality (VR) environment, which allows for a subjective close-up analysis from multiple, interactive perspectives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now
Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 319.93
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 406.59
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    We follow the definitions of scene and scenario by Ulbrich et al. [34].

  2. 2.

    We follow the conceptualization of corner-cases by Heidecker et al. [14].

References

  1. Balas, V.E., Balas, M.M.: Driver assisting by inverse time to collision. In: 2006 World Automation Congress, WAC 2006, June 2014, pp. 1–7 (2006). https://doi.org/10.1109/WAC.2006.376059

  2. Bender, P., Ziegler, J., Stiller, C.: Lanelets: efficient map representation for autonomous driving. In: Proceedings of the IEEE Intelligent Vehicles Symposium (Iv), pp. 420–425 (2014). https://doi.org/10.1109/IVS.2014.6856487

  3. Bock, J., et al.: The inD dataset: a drone dataset of naturalistic road user trajectories at German intersections. ar**v preprint ar**v:1911.07602 (2019)

  4. Caesar, H., et al.: Nuscenes: a multimodal dataset for autonomous driving. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 11618–11628, March 2020. https://doi.org/10.1109/CVPR42600.2020.01164

  5. Cipresso, P., Giglioli, I.A.C., Raya, M.A., Riva, G.: The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature. Front. Psychol. 9, 1–20 (2018). https://doi.org/10.3389/fpsyg.2018.02086

  6. Dixon, L., et al.: Scenario description and knowledge-based scenario generation. RAND Corp., Santa Monica, CA, USA, RR-1776-NYCEDC (2018). https://www.pegasusprojekt.de/files/tmpl/Pegasus-Abschlussveranstaltung/05_Scenario_Description_and_Knowledge-Based_Scenario_Generation.pdf. Accessed 19 Feb 2021

  7. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning, pp. 1–16 (2017)

    Google Scholar 

  8. Dupuis, M., Strobl, M., Grezlikowski, H.: Opendrive 2010 and beyond-status and future of the de facto standard for the description of road networks. In: Proceedings of the Driving Simulation Conference Europe, pp. 231–242 (2010)

    Google Scholar 

  9. Epic Games, I.: Unreal engine. https://www.unrealengine.com/en-US/. Accessed 24 Feb 2021

  10. Farooq, B., Cherchi, E., Sobhani, A.: Virtual Immersive reality for stated preference travel behaviour experiments: a case study of autonomous Vehicles on Urban Roads. ar**v (2018)

    Google Scholar 

  11. Fleck, T., et al.: Towards large scale urban traffic reference data: smart infrastructure in the test area autonomous driving Baden-Württemberg. In: IAS (2018)

    Google Scholar 

  12. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012). https://doi.org/10.1109/CVPR.2012.6248074

  13. Hägerstrand, T.: What about people in regional science? In: Papers of the Regional Science Association, vol. 24 (1970)

    Google Scholar 

  14. Heidecker, F., et al.: An application-driven conceptualization of corner cases for perception in highly automated driving (2021)

    Google Scholar 

  15. Huang, Y., Chen, Y.: Autonomous driving with deep learning: a survey of state-of-art technologies (2020)

    Google Scholar 

  16. Intel: C++ Library for Responsibility Sensitive Safety (2021). https://github.com/intel/ad-rss-lib

  17. ISO 22839:2013: Intelligent transport systems — forward vehicle collision mitigation systems — operation, performance, and verification requirements, September 2018. https://www.iso.org/standard/45339.html

  18. Junietz, P.M.: Microscopic and macroscopic risk metrics for the safety validation of automated driving. TU Darmstadt, Darmstadt (2019). http://tuprints.ulb.tu-darmstadt.de/9282/

  19. Koopman, P.: The heavy tail safety ceiling. In: Automated and Connected Vehicle Systems Testing Symposium, pp. 1–2 (2018). http://users.ece.cmu.edu/~koopman

  20. Kraak, M.j.: The space-time cube revisited from a geovisualization perspective. In: 21st International Cartographic Conference (ICC), pp. 10–16, August 2003. ISBN: 0-958-46093-0

    Google Scholar 

  21. Laflamme, C.É.N., Giguère, P., Pomerleau, F.: Driving datasets literature review. ar**v (2019)

    Google Scholar 

  22. Luo, H., Yang, T., Kwon, S., Zuo, M., Li, W., Choi, I.: Using virtual reality to identify and modify risky pedestrian behaviors amongst Chinese children. Traffic Injury Prev. 21(1), 108–113 (2020). https://doi.org/10.1080/15389588.2019.1694667

  23. Meir, A., Oron-Gilad, T., Parmet, Y.: Are child-pedestrians able to identify hazardous traffic situations? Measuring their abilities in a virtual reality environment. Saf. Sci. 80, 33–40 (2015). https://doi.org/10.1016/j.ssci.2015.07.007

  24. Millais, P., Jones, S.L., Kelly, R.: Exploring data in virtual reality: comparisons with 2D data visualizations. In: Proceedings of the 2018 Conference on Human Factors in Computing Systems, pp. 5–10, April 2018. https://doi.org/10.1145/3170427.3188537

  25. Nistér, D., Lee, H.L., Ng, J., Wang, Y.: The safety force field. In: NVIDIA White Paper (2019). https://www.nvidia.com/content/dam/en-zz/Solutions/self-driving-cars/safety-force-field/the-safety-force-field.pdf

  26. Nvidia: DriveWorks SDK Reference - Safety Force Field (2020). https://docs.nvidia.com/drive/driveworks-3.0/safetyforcefield_mainsection.html

  27. Okechukwu, M., Udoka, F.: Understanding virtual reality technology: advances and applications. In: Advances in Computer Science and Engineering, June 2015 (2011). https://doi.org/10.5772/15529

  28. Pek, C., Manzinger, S., Koschi, M., Althoff, M.: Using online verification to prevent autonomous vehicles from causing accidents. Nat. Mach. Intell. 2(9), 518–528 (2020). https://doi.org/10.1038/s42256-020-0225-y

  29. Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)

    Google Scholar 

  30. Reski, N., Alissandrakis, A.: Open data exploration in virtual reality: a comparative study of input technology. Virtual Real. 24(1), 1–22 (2020). https://doi.org/10.1007/s10055-019-00378-w

  31. Scholtes, M., et al.: 6-layer model for a structured description and categorization of urban traffic and environment (2021)

    Google Scholar 

  32. Shalev-Shwartz, S., Shammah, S., Shashua, A.: On a formal model of safe and scalable self-driving cars. ar**v, pp. 1–37 (2017)

    Google Scholar 

  33. Sun, P., et al.: Scalability in perception for autonomous driving: Waymo open dataset. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2443–2451 (2020). https://doi.org/10.1109/CVPR42600.2020.00252

  34. Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F., Maurer, M.: Defining and substantiating the terms scene, situation, and scenario for automated driving. In: Proceedings of the IEEE Conference on Intelligent Transportation Systems, ITSC 2015, pp. 982–988, October 2015. https://doi.org/10.1109/ITSC.2015.164

  35. Van Dam, A., Laidlaw, D.H., Simpson, R.M.: Experiments in immersive virtual reality for scientific visualization. Comput. Graph. (Pergamon) 26(4), 535–555 (2002). https://doi.org/10.1016/S0097-8493(02)00113-9

    Article  Google Scholar 

  36. Wirth, F., Quchl, J., Ota, J., Stiller, C.: PointAtMe: efficient 3D point cloud labeling in virtual reality. In: Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (Iv), pp. 1693–1698, June 2019. https://doi.org/10.1109/IVS.2019.8814115

  37. Xu, W., Yao, W., Zhao, H., Zha, H.: A vehicle model for micro-traffic simulation in dynamic urban scenarios. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2267–2274 (2011). https://doi.org/10.1109/ICRA.2011.5980101

  38. Yu, F., et al.: BDD100K: a diverse driving dataset for heterogeneous multitask learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2633–2642 (2020). https://doi.org/10.1109/CVPR42600.2020.00271

  39. Zhan, W., et al.: Interaction dataset: an international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps. ar**v:1910.03088 [cs, eess], September 2019

  40. Zhao, H., et al.: Safety score: a quantitative approach to guiding safety-aware autonomous vehicle computing system design. In: Proceedings of the IEEE Intelligent Vehicles Symposium (Iv), pp. 1479–1485 (2020). https://doi.org/10.1109/IV47402.2020.9304602

  41. Zheng, L., Ismail, K., Meng, X.: Traffic conflict techniques for road safety analysis: open questions and some insights. Can. J. Civ. Eng. 41(7), 633–641 (2014). https://doi.org/10.1139/cjce-2013-0558

    Article  Google Scholar 

  42. Zipfl, M., Fleck, T., Zofka, M.R., Zöllner, J.M.: From traffic sensor data to semantic traffic descriptions: the test area autonomous driving Baden-Württemberg dataset (TAF-BW dataset). In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–7. IEEE (2020)

    Google Scholar 

  43. Zofka, M.R., et al.: Pushing ROS towards the dark side: a ROS-based co-simulation architecture for mixed-reality test systems for autonomous vehicles. In: 2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 204–211 (2020). https://doi.org/10.1109/MFI49285.2020.9235238

Download references

Acknowledgment

This publication was written in the framework of European Union’s Horizon 2020 Research and Innovation Programme under grant agreement no 815001, project Drive2theFuture (Needs, wants and behavior of “Drivers” and automated vehicles users today and in to the future) and also partially supported by the Intel Collaborative Research Institute for Safe Automated Vehicles (ICRI-SAVe).

Author information

Authors and Affiliations

  1. FZI Research Center for Information Technology, Karlsruhe, Germany

    Lars Töttel, Maximilian Zipfl, Daniel Bogdoll, Marc René Zofka & J. Marius Zöllner

Authors
  1. Lars Töttel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maximilian Zipfl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Bogdoll
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marc René Zofka
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Marius Zöllner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maximilian Zipfl .

Editor information

Editors and Affiliations

  1. Department of Mechanical and Electrical Engineering, University of Electronic Science and Technology, Chengdu, Sichuan, China

    Zhenyuan Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Töttel, L., Zipfl, M., Bogdoll, D., Zofka, M.R., Zöllner, J.M. (2022). Reliving the Dataset: Combining the Visualization of Road Users’ Interactions with Scenario Reconstruction in Virtual Reality. In: Zhang, Z. (eds) 2021 6th International Conference on Intelligent Transportation Engineering (ICITE 2021). ICITE 2021. Lecture Notes in Electrical Engineering, vol 901. Springer, Singapore. https://doi.org/10.1007/978-981-19-2259-6_39

Download citation

Publish with us

Policies and ethics

Search

Navigation