SpringerLink
Log in

An Improved Grey Relational Theory Evaluation Method: Considering the Comprehensive Performance of Autonomous Vehicles in Virtual Test

  • Connected Automated Vehicles and ITS, Vehicle Dynamics and Control
  • Published:
International Journal of Automotive Technology Aims and scope Submit manuscript

Abstract

Reasonable test scenarios and objective evaluation methods can rapidly promote the development of autonomous vehicle technology. A new quantitative evaluation method for the comprehensive performance of autonomous vehicle is proposed in this paper. First, different test environments and test contents are combined to obtain vehicle test scenarios of different complexity. Then, the evaluation index system of autonomous vehicle is divided into target layer, total index layer, and index layer. After that, the weights of the index layer are determined by the objective weight method of Criteria Importance though Intercriteria Correlation (CRITIC) method, and the total weights of index layer are determined by the analytic hierarchy process (AHP) of subjective weight method. Finally, the improved grey relational theory method is used to quantitatively evaluate autonomous vehicles from four aspects: driving safety, riding comfort, intelligence, and efficiency. The quantitative evaluation of autonomous vehicles can reduce the influence of abnormal data on the correlation degree and increase the robustness of the evaluation algorithm. The evaluation results of the proposed method and the traditional fuzzy comprehensive evaluation method are compared by simulation experiment and evaluation. The results show that the proposed evaluation method in this paper is more objective and reasonable, which can quantitatively evaluate the comprehensive performance of autonomous vehicles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Data availability

Not applicable.

References

  • Abdel-Aty, M.; Cai, Q.; Wu, Y.; Zheng, O (2022). Evaluation of automated emergency braking system avoidance of pedestrian crashes at intersections under occluded conditions within a virtual simulator. Accident Anal Prev 176.

  • Adebisi, A., Guo, Y., Schroeder, B., Ma, J., Cesme, B., Bibeka, A., & Morgan, A. (2022). Highway capacity manual capacity adjustment factor development for connected and automated traffic at signalized intersections. J Transport Eng A-Syst, 148, 3.

    Google Scholar 

  • Akagi, Y.; Kato, R.; Kitajima, S.; Antona-Makoshi, J.; Uchida, N (2019). A Risk-index based Sampling Method to Generate Scenarios for the Evaluation of Automated Driving Vehicle Safety. In Proceedings of the IEEE Intelligent Transportation Systems Conference (IEEE-ITSC), Auckland, New Zealand, 27–30 October 2019.

  • Alghodhaifi, H., & Lakshmanan, S. (2021). Autonomous vehicle evaluation: A comprehensive survey on modeling and simulation approaches. Ieee Access, 9, 151531–151566.

    Article  Google Scholar 

  • Bae, G. H.; Lee, S. B.(2020). A Study on the Evaluation Method of Highway Driving Assist System Using Monocular Camera. Applied Sciences-Basel 10, 18.

  • Chen, J., Li, R., **ng, X., Meng, H., & Yu, Z. (2019). Survey on intelligence evaluation of autonomous vehicles. Journal of Tongji University (natural Science), 47(12), 1785–1790.

    Google Scholar 

  • Dong, F.; Zhao, Y.-N.; Gao, L.(2015). Application of Gray Correlation and Improved AHP to Evaluation on Intelligent U-turn Behavior of Unmanned Vehicles. In Proceedings of the 8th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 12–13 December 2015.

  • Du, F.; Zhao, Y.; Gao, L.; Wang, W (2017). Evaluation of the Autonomous Parking System Based on BP Neural Network. In Proceedings of the 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China, 26–27 August 2017.

  • Duan, J., Gao, F., & He, Y. (2022). Test scenario generation and optimization technology for intelligent driving systems. Ieee Intell Transp Sy, 14(1), 115–127.

    Google Scholar 

  • Feng, S., Feng, Y., Yan, X., Shen, S., Xu, S., & Liu, H. X. (2020). Safety assessment of highly automated driving systems in test tracks: A new framework. Accident Anal Prev, 144, 105664.

    Article  Google Scholar 

  • Ho, W., & Ma, X. (2018). The state-of-the-art integrations and applications of the analytic hierarchy process. European Journal of Operational Research, 267(2), 399–414.

    Article  MathSciNet  Google Scholar 

  • Hu, J., Li, X., Hu, W., et al. (2023). Decision-making for connected and automated vehicles in chanllenging traffic conditions using imitation and deep reinforcement learning. International Journal of Automotive Technology, 24, 1589–1602.

    Article  Google Scholar 

  • Hu, L., Yi, P., Huang, J., Zhang, X., & Lei, Z. (2018). A research on test scenes of two-wheeled vehicles for automatic emergency braking system based on real accident cases. Automotive Engineering, 40(12), 1435–1446.

    Google Scholar 

  • Huang, W.; Wang, K.; Lv, Y.; Zhu, F (2016). Autonomous Vehicles Testing Methods Review. In Proceedings of the 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). Rio de Janeiro, Brazil, 1–4 November.

  • Huang, Z.; Chai, J.; Li, B.; Feng, X (2017). Application of Index Weighting Based on the Determination Method of Comprehensive Weight Coefficient. In Proceedings of the 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), Bei**g, China, 24–26 November 2017.

  • Huang, Z.; Guo, Y.; Arief, M.; Lam, H.; Zhao, D (2018). In A Versatile Approach to Evaluating and Testing Automated Vehicles based on Kernel Methods, 2018 American Control Conference (ACC), IEEE, 4796–4802.

  • Krishnan, A. R., Kasim, M. M., Hamid, R., & Ghazali, M. F. (2021). A modified CRITIC method to estimate the objective weights of decision criteria. Symmetry-Basel, 13, 6.

    Google Scholar 

  • Lengyel, H., Tettamanti, T., & Szalay, Z. (2020). Conflicts of automated driving with conventional traffic infrastructure. Ieee Access, 8, 163280–163297.

    Article  Google Scholar 

  • Li, P; Zhao, Y.; Yang, J.; Cong, Z.; Zhang, J (2017). Analysis of safety evaluation on EV battery replacement station based on the gray theory. In Proceedings of the 36th Chinese Control Conference (CCC), Dalian, China, 26–28 July 2017.

  • Li, L., Huang, W., Liu, Y., Zheng, N., & Wang, F. (2016). Intelligence testing for autonomous vehicles: A new approach. Ieee Ts Intell Vehicl, 1(2), 158–166.

    Article  Google Scholar 

  • Li, S., Li, W., Li, P., Ma, P., & Yang, M. (2022). Novel test scenario generation technology for performance evaluation of automated vehicle. Int J Auto Tech-Kor, 23(5), 1295–1312.

    Article  Google Scholar 

  • Li, Y., Li, Z., Wang, H., Wang, W., & **ng, L. (2017a). Evaluating the safety impact of adaptive cruise control in traffic oscillations on freeways. Accident Anal Prev, 104, 137–145.

    Article  Google Scholar 

  • Luo, Y., **ang, D., Zhang, S., Liang, W., Sun, J., & Zhu, L. (2021). Evaluation on the fuel economy of automated vehicles with data-driven simulation method. Energy and AI, 3, 100051.

    Article  Google Scholar 

  • Schaedler, O.; Mueller, S.; Gruendl, M (2016). Experimental evaluation of the controllability of interacting advanced driver assistance systems. In Proceedings of the 24th Symposium of the International-Association-for-Vehicle-System-Dynamics (IAVSD), Graz, Austria, 17–21 August 2016.

  • Su, Y., & Wang, L. (2021). Integrated framework for test and evaluation of autonomous vehicles. Journal of Shanghai Jiaotong University (science), 26, 699–712.

    Article  Google Scholar 

  • Sun, Y.; **ong, G.; Song, W.; Gong, J.; Chen, H (2014). Test and Evaluation of Autonomous Ground Vehicles. Adv Mechl Eng.

  • Sun, Y., **ong, G., & Chen, H. (2014b). Evaluation of the intelligent behaviors of unmanned ground vehicles based on fuzzy-EAHP scheme. Automotive Engineering, 36(1), 22–27.

    Google Scholar 

  • Tan, H., Zhao, F., Zhang, W., & Liu, Z. (2023). An evaluation of the safety effectiveness and cost of autonomous vehicles based on multivariable coupling. Sensors, 23, 3.

    Article  Google Scholar 

  • Tehrani, H.; Muto, K.; Yoneda, K.; Mita, S (2014). Evaluating Human & Computer for Expressway Lane Changing. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Dearborn, MI, 8–11 June 2014.

  • Wang, R., Sun, Y., & Song, J. (2021). Evaluation method and test verification of road test scenes for autonomous vehicles. Automotive Engineering, 43(04), 620–628.

    Google Scholar 

  • Wang, T., & Lee, H. (2009). Develo** a fuzzy TOPSIS approach based on subjective weights and objective weights. Expert Systems with Applications, 36(5), 8980–8985.

    Article  Google Scholar 

  • Wang, W., Wu, L., Li, X., Qu, F., Li, W., Ma, Y., & Ma, D. (2023). An evaluation method for automated vehicles combining subjective and objective factors. Machines, 11, 6.

    Google Scholar 

  • Wang, Y., & Liu, J. (2012). Evaluation methods for the autonomy of unmanned systems. Chin Sci B-Chin, 57(26), 3409–3418.

    Article  Google Scholar 

  • **ao, D.; Geiger, W.G.; Yatbaz, H.Y.; Dianati, M.; Woodman, R (2022). Detecting Hazardous Events: A Framework for Automated Vehicle Safety Systems. In Proceedings of the IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), Macau, China, 8–12 October 2022.

  • Xu, X., Zhou, Z., Hu, W., **ao, L., Li, W., & Wang, S. (2020). Intersection test scenarios for AEB based on accident data mining. Journal of Bei**g University of Aeronautics and Astronautics, 46(10), 1817–1825.

    Google Scholar 

  • Yang, L.; Yang, Y.; Wu, G.; Zhao, X.; Fang, S.; Liao, X.; Wang, R.; Zhang, M (2022). A systematic review of autonomous emergency braking system: impact factor, technology, and performance evaluation. J Adv Transport.

  • Zhang, C.; Liu, Y.; Zhao, D.; Su, Y (2014). RoadView: A Traffic Scene Simulator for Autonomous Vehicle Simulation Testing. In Proceedings of the IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), Qingdao, China, 8–11 October 2014.

  • Zhang, J.; Wu, K.; Cheng, M.; Yang, M.; Cheng, Y.; Li, S (2020). Safety evaluation for connected and autonomous vehicles' exclusive lanes considering penetrate ratios and impact of trucks using surrogate safety measures. J Adv Transport.

  • Zhang, T., Li, Q., Zhang, C., Liang, H., Li, P., Wang, T., Li, S., Zhu, Y., & Wu, C. (2017). Current trends in the development of intelligent unmanned autonomous systems. Front Inform Tech EI, 18(1), 68–85.

    Article  Google Scholar 

  • Zhao, Y.-N.; Meng, K.; Gao, L (2015). The Entropy-Cost Function Evaluation Method for Unmanned Ground Vehicles. Mathl Probl Eng.

  • Zhao, D.; Liu, Y.; Zhang, C.; Li, Y (2015). Autonomous Driving Simulation for Unmanned Vehicles. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV 2015), Waikoloa, HI, 6–9 January 2015.

  • Zhao, J.; Liu, J.; Yang, L.; He, P (2018). Exploring the Relationships between Subjective Evaluations and Objective Metrics of Vehicle Dynamic Performance. J Adv Transport.

  • Zhao, D., Lam, H., Peng, H., Bao, S., LeBlanc, D. J., Nobukawa, K., & Pan, C. S. (2016). Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques. IEEE T Intell Transp, 18(3), 595–607.

    Article  Google Scholar 

  • Zheng, B., Hong, Z., Tang, J., Han, M., Chen, J., & Huang, X. (2023). A comprehensive method to evaluate ride comfort of autonomous vehicles under typical braking scenarios: Testing. Simulation and Analysis. Mathematics, 11, 2.

    Google Scholar 

  • Zhu, B., Zhang, P., Zhao, J., Chen, H., Xu, Z., Zhao, X., & Deng, W. (2019). Review of scenario-based virtual validation methods for automated vehicles. China Journal of Highway and Transport, 32(06), 1–19.

    Google Scholar 

Download references

Acknowledgements

This research was funded by the Scientific research project of CATARC Automotive Test Center (Tian**) Co., Ltd., (TJKY2426009).

Author information

Authors and Affiliations

  1. CATARC Automotive Test Center (Tian**) Co., Ltd., 68 **anfeng East Rd, Dongli District, Tian**, 300300, China

    Wei Wang, Wen-Bo Li, Fu-Fan Qu, Ting Dong, Li-Guang Wu & Cun-Yang Shi

  2. College of Automotive Engineering, Jilin University, No.2519 Jiefang Road, Changchun, 130022, China

    Guang-Yu Wang

Authors
  1. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Bo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fu-Fan Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ting Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guang-Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Li-Guang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cun-Yang Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ting Dong.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Li, WB., Qu, FF. et al. An Improved Grey Relational Theory Evaluation Method: Considering the Comprehensive Performance of Autonomous Vehicles in Virtual Test. Int.J Automot. Technol. (2024). https://doi.org/10.1007/s12239-024-00113-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12239-024-00113-8

Keywords

Search

Navigation