Abstract
Radar and camera sensors are commonly used in the perception stack of autonomous driving systems due to their maturity, cost-effectiveness, and robustness. The outputs from these sensors are often fused to generate more accurate perception results, due to their complementary features. In this paper, we propose a method, RCBEVD: Radar-camera fusion in Bird’s Eye View (BEV) representation for detection with velocity estimation. We introduce a novel clustering method for radar point clouds alongside a tracking algorithm to estimate the true radar point’s velocity. The estimated true radar velocities are fused with 2D object detection bounding boxes to give BEV detections. RCBEVD is evaluated on the nuScenes dataset benchmark using the Average Velocity Error (AVE) and Average Precision (AP) metrics. We outperform the baseline method CenterFusion’s Mean Average Velocity Error (mAVE) \(0.696\,\text {m/s} \,(-0.690\,\text {m/s})\), an improvement of \(49.8\%\) on the nuScenes validation dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: optimal speed and accuracy of object detection (2020) [Online]. Available: https://arxiv.org/abs/2004.10934
Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., Brox, T.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Niederlohner, D., Ulrich, M., Braun, S., Kohler, D., Faion, F., Glaser, C., Treptow, A., Blume, H.: Self-supervised velocity estimation for automotive radar object detection networks. In: 2022 IEEE Intelligent Vehicles Symposium (IV). IEEE (2022) [Online]. Available: https://doi.org/10.1109/IV51971.2022.9827295
Yao, S., Guan, R., Huang, X., Li, Z., Sha, X., Yue, Y., Lim, E.G., Seo, H., Man, K.L., Zhu, X., Yue, Y.: Radar-camera fusion for object detection and semantic segmentation in autonomous driving: a comprehensive review (2023)
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: a multimodal dataset for autonomous driving. In: CVPR (2020)
Schumann, O., Hahn, M., Scheiner, N., Weishaupt, F., Tilly, J.F., Dickmann, J., Wöhler, C.: Radarscenes: a real-world radar point cloud data set for automotive applications. In: 2021 IEEE 24th International Conference on Information Fusion (FUSION), pp. 1–8. IEEE (2021)
Zheng, L., Ma, Z., Zhu, X., Tan, B., Li, S., Long, K., Sun, W., Chen, S., Zhang, L., Wan, M., Huang, L., Bai, J.: TJ4dradset: a 4d radar dataset for autonomous driving. In: 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE (2022) [Online]. Available: https://doi.org/10.1109/ITSC55140.2022.9922539
Zhou, Y., Tuzel, O.: Voxelnet: end-to-end learning for point cloud based 3d object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2018)
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12697–12705 (2019)
Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theor. 28(2), 129–137 (1982)
Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Knowledge Discovery and Data Mining (1996)
Girshick, R.B., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation (2013) [Online]. Available: http://arxiv.org/abs/1311.2524
Duan, K., Bai, S., **e, L., Qi, H., Huang, Q., Tian, Q.: Centernet: keypoint triplets for object detection (2019) [Online]. Available: https://arxiv.org/abs/1904.08189
Xue, F., Zhuo, G., Huang, Z., Fu, W., Wu, Z., Ang, M.H.: Toward hierarchical self-supervised monocular absolute depth estimation for autonomous driving applications (2020) [Online]. Available: https://arxiv.org/abs/2004.05560
Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth prediction (2019)
Nabati, R., Qi, H.: Centerfusion: center-based radar and camera fusion for 3d object detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1527–1536 (2021)
Nobis, F., Shafiei, E., Karle, P., Betz, J., Lienkamp, M.: Radar voxel fusion for 3d object detection. In: Applied Sciences, vol. 11, no. 12 (2021) [Online]. Available: https://www.mdpi.com/2076-3417/11/12/5598
Kim, Y., Kim, S., Choi, J.W., Kum, D.: Craft: camera-radar 3d object detection with spatio-contextual fusion transformer (2022)
Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jia, Y., Lee, C.D.W., Ang, M.H. (2024). RCBEVD: Radar-Camera Fusion in Bird’s Eye View for Detection with Velocity Estimation. In: Lee, SG., An, J., Chong, N.Y., Strand, M., Kim, J.H. (eds) Intelligent Autonomous Systems 18. IAS 2023. Lecture Notes in Networks and Systems, vol 795. Springer, Cham. https://doi.org/10.1007/978-3-031-44851-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-44851-5_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44850-8
Online ISBN: 978-3-031-44851-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)