SpringerLink
Log in

On the safety of vulnerable road users by cyclist detection and tracking

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Timely detection of vulnerable road users is of great relevance to avoid accidents in the context of intelligent transportation systems. In this work, detection and tracking is acknowledged for a particularly vulnerable class of road users, the cyclists. We present a performance comparison between the main deep learning-based algorithms reported in the literature for object detection, such as SSD, Faster R-CNN and R-FCN along with InceptionV2, ResNet50, ResNet101, Mobilenet V2 feature extractors. In order to identify the cyclist heading and predict its intentions, we propose a multi-class detection with eight classes according to orientations. To do so, we introduce a new dataset called “CIMAT-Cyclist”, containing 20,229 cyclist instances over 11,103 images, labeled based on the cyclist’s orientation. To improve the performance in cyclists’ detection, the Kalman filter is used for tracking, coupled together with the Kuhn–Munkres algorithm for multi-target association. Finally, the vulnerability of the cyclists is evaluated for each instance in the field of view, taking into account their proximity and predicted intentions according to their heading angle, and a risk level is assigned to each cyclist. Experimental results validate the proposed strategy in real scenarios, showing good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ahmad, T., Ma, Y., Yahya, M., Ahmad, B., Nazir, S., et al.: Object detection through modified YOLO neural network. Sci. Program. 2020 (2020)

  2. Bar-Shalom, Y., Li, X.R., Kirubarajan, T.: Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software. Wiley, Hoboken (2004)

    Google Scholar 

  3. Basso, G.F., De Amorim, T.G.S., Brito, A.V., Nascimento, T.P.: Kalman filter with dynamical setting of optimal process noise covariance. IEEE Accessed 5, 8385–8393 (2017)

    Article  Google Scholar 

  4. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE, Phoenix Convention Centre, Phoenix, Ariz (2016)

  5. Chen, X., Kundu, K., Zhu, Y., Berneshawi, A.G., Ma, H., Fidler, S., Urtasun, R.: 3D object proposals for accurate object class detection. In: Advances in Neural Information Processing Systems, pp. 424–432 (2015)

  6. Chen, Y.Y., Jhong, S.Y., Li, G.Y., Chen, P.H.: Thermal-based pedestrian detection using faster R-CNN and region decomposition branch. In: 2019 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp. 1–2. IEEE, Taipei, Taiwan (2019)

  7. Cho, H., Rybski, P.E., Zhang, W.: Vision-based 3D bicycle tracking using deformable part model and interacting multiple model filter. In: 2011 IEEE International Conference on Robotics and Automation, pp. 4391–4398. IEEE, Shanghai (2011)

  8. Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems 29 (NIPS 2016), pp. 379–387. Barcelona, Spain (2016)

  9. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 743–761 (2012). https://doi.org/10.1109/TPAMI.2011.155

    Article  Google Scholar 

  10. Erhan, D., Szegedy, C., Toshev, A., Anguelov, D.: Scalable object detection using deep neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2155–2162. Columbus, OH, USA (2014)

  11. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  12. Ferguson, M., Law, K.: A 2d-3d object detection system for updating building information models with mobile robots. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1357–1365. IEEE (2019)

  13. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361. IEEE, Providence, Rhode Island, USA (2012)

  14. Girshick, R.: Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1440–1448. Santiago, Chile (2015). https://doi.org/10.1109/ICCV.2015.169

  15. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587. Columbus, OH, USA (2014). https://doi.org/10.1109/CVPR.2014.81

  16. Guan, K.: Computer vision based vehicle detection and tracking using tensorflow object detection api and kalman-filtering (2018)

  17. Guindel, C., Martín, D., Armingol, J.M.: Modeling traffic scenes for intelligent vehicles using CNN-based detection and orientation estimation. In: ROBOT 2017: Third Iberian Robotics Conference, pp. 487–498. Springer, Seville, Spain (2017)

  18. Guindel, C., Martin, D., Armingol, J.M.: Fast joint object detection and viewpoint estimation for traffic scene understanding. IEEE Intell. Transp. Syst. Mag. 10(4), 74–86 (2018)

    Article  Google Scholar 

  19. Guindel, C., Martin, D., Armingol, J.M.: Traffic scene awareness for intelligent vehicles using ConvNets and stereo vision. Robot. Auton. Syst. 112, 109–122 (2019)

    Article  Google Scholar 

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. Las Vegas, NV, USA (2016)

  21. Heo, D., Nam, J.Y., Ko, B.C.: Estimation of pedestrian pose orientation using soft target training based on teacher-student framework. Sensors 19(5), 1147 (2019)

    Article  Google Scholar 

  22. Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3296–3297. IEEE, Honolulu, HI, USA (2017)

  23. Hui, J.: Object detection: speed and accuracy comparison (faster R-CNN, R-FCN, SSD, FPN, RetinaNet and YOLOV3). Medium (2018). https://medium.com/@jonathan_hui/object-detection-speed-and-accuracy-comparison-faster-r-cnn-r-fcn-ssd-and-yolo-5425656ae359. Accessed on 25 July 2019

  24. Jonker, R., Volgenant, T.: Improving the Hungarian assignment algorithm. Oper. Res. Lett. 5(4), 171–175 (1986)

  25. Jung, H., Tan, J.K., Ishikawa, S., Morie, T.: Applying hog feature to the detection and tracking of a human on a bicycle. In: 2011 11th International Conference on Control, Automation and Systems, pp. 1740–1743. IEEE, Gyeonggi-do, Korea (South) (2011)

  26. Kang, Y., Yin, H., Berger, C.: Test your self-driving algorithm: an overview of publicly available driving datasets and virtual testing environments. IEEE Trans. Intell. Veh. 4(2), 171–185 (2019)

    Article  Google Scholar 

  27. Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logist. Q. 2(1–2), 83–97 (1955). https://doi.org/10.1002/nav.3800020109

  28. Lan, W., Dang, J., Wang, Y., Wang, S.: Pedestrian detection based on YOLO network model. In: 2018 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 1547–1551. IEEE, Changchun, China (2018)

  29. Li, X., Flohr, F., Yang, Y., **ong, H., Braun, M., Pan, S., Li, K., Gavrila, D.M.: A new benchmark for vision-based cyclist detection. In: 2016 IEEE Intelligent Vehicles Symposium (IV), pp. 1028–1033. IEEE, Gothenburg, Sweden (2016)

  30. Li, X., Li, L., Flohr, F., Wang, J., **ong, H., Bernhard, M., Pan, S., Gavrila, D.M., Li, K.: A unified framework for concurrent pedestrian and cyclist detection. IEEE Trans. Intell. Transp. Syst. 18(2), 269–281 (2017). https://doi.org/10.1109/TITS.2016.2567418

    Article  Google Scholar 

  31. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer, Zurich, Switzerland (2014)

  32. Liu, C., Guo, Y., Li, S., Chang, F.: ACF based region proposal extraction for YOLOv3 network towards high-performance cyclist detection in high resolution images. Sensors 19(12), 2671 (2019)

    Article  Google Scholar 

  33. Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., Pietikäinen, M.: Deep learning for generic object detection: a survey. Int. J. Comput. Vis. 128(2), 261–318 (2020)

  34. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer, Amsterdam, The Netherlands (2016)

  35. Maurya, S.K., Choudhary, A.: Deep learning based vulnerable road user detection and collision avoidance. In: 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), pp. 1–6. Madrid (2018)

  36. Ohn-Bar, E., Trivedi, M.M.: Fast and robust object detection using visual subcategories. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 179–184. Columbus, OH, USA (2014). https://doi.org/10.1109/CVPRW.2014.32

  37. Phon-Amnuaisuk, S., Murata, K.T., Pavarangkoon, P., Yamamoto, K., Mizuhara, T.: Exploring the applications of faster R-CNN and single-shot multi-box detection in a smart nursery domain. ar**v preprint ar**v:1808.08675 (2018)

  38. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788. Las Vegas, NV, USA (2016)

  39. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525. Honolulu, HI, USA (2017)

  40. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  41. Sahbani, B., Adiprawita, W.: Kalman filter and iterative-Hungarian algorithm implementation for low complexity point tracking as part of fast multiple object tracking system. In: 2016 6th International Conference on System Engineering and Technology (ICSET), pp. 109–115. IEEE, Bandung, Indonesia (2016)

  42. Saho, K.: Kalman filter for moving object tracking: performance analysis and filter design. In: Kalman Filters-Theory for Advanced Applications, pp. 233–252 (2017)

  43. Saleh, K., Hossny, M., Hossny, A., Nahavandi, S.: Cyclist detection in LIDAR scans using faster R-CNN and synthetic depth images. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), pp. 1–6. IEEE, Yokohama, Japan (2017)

  44. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. Salt Lake City, UT (2018)

  45. Saranya, K.C., Thangavelu, A., Chidambaram, A., Arumugam, S., Govindraj, S.: Cyclist detection using tiny YOLO V2. In: Soft Computing for Problem Solving (SocProS), pp. 969–979. Springer (2020)

  46. Sermanet, P., Kavukcuoglu, K., Chintala, S., Lecun, Y.: Pedestrian detection with unsupervised multi-stage feature learning. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3626–3633. Portland, OR, USA (2013). https://doi.org/10.1109/CVPR.2013.465

  47. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence. San Francisco, California, USA (2017)

  48. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826. Las Vegas, NV, USA (2016)

  49. Tian, W., Lauer, M.: Fast and robust cyclist detection for monocular camera systems. In: 10th International joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP). Berlin, Germany (2015)

  50. Tian, W., Lauer, M.: Detection and orientation estimation for cyclists by max pooled features. In: 12th International Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), pp. 17–26. Porto, Portugal (2017)

  51. Wang, K., Zhou, W.: Pedestrian and cyclist detection based on deep neural network fast R-CNN. Int. J. Adv. Robot. Syst. 16(2), 1729881419829651 (2019)

    MathSciNet  Google Scholar 

  52. World Health Organization: Global status report on road safety 2018. World Health Organization, Geneva (2018). https://www.who.int/violence_injury_prevention/road_safety_status/2018/en/

  53. World Health Organization: Road Safety 2018. https://extranet.who.int/roadsafety/death-on-the-roads/ (2020). Accessed: July 2020

  54. Xu, J.: Deep learning for object detection: a comprehensive review. Towards Data Science (2017)

  55. Yan, C., Gong, B., Wei, Y., Gao, Y.: Deep multi-view enhancement hashing for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1445–1451 (2020)

    Article  Google Scholar 

  56. Yan, C., Shao, B., Zhao, H., Ning, R., Zhang, Y., Xu, F.: 3D room layout estimation from a single RGB image. IEEE Trans. Multimedia 22(11), 3014–3024 (2020)

    Article  Google Scholar 

  57. Yang, F., Chen, H., Li, J., Li, F., Wang, L., Yan, X.: Single shot multibox detector with Kalman filter for online pedestrian detection in video. IEEE Accessed 7, 15478–15488 (2019)

    Article  Google Scholar 

  58. Zhang, M., Fu, R., Guo, Y., Wang, L., Wang, P., Deng, H.: Cyclist detection and tracking based on multi-layer laser scanner. HCIS 10, 1–18 (2020)

  59. Zhang, S., Wang, X.: Human detection and object tracking based on histograms of oriented gradients. In: 2013 Ninth International Conference on Natural Computation (ICNC), pp. 1349–1353. Shenyang, China (2013). https://doi.org/10.1109/ICNC.2013.6818189

  60. Zhao, Z.Q., Zheng, P., Xu, S.T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30(11), 3212–3232 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the Mexican National Council of Science and Technology CONACyT for the grants given and the FORDECyT project 296737 “Consorcio en Inteligencia Artificial” and also “Sportbike Jerez MTB” for allowing us to take pictures at their events.

Author information

Authors and Affiliations

  1. Center for Research in Mathematics CIMAT AC, Campus Zacatecas, Zacatecas, Mexico

    M. García-Venegas, L. A. Pinedo-Sánchez & C. A. Carballo-Monsivais

  2. Cátedras CONACyT, Center for Research in Mathematics CIMAT AC, Campus Zacatecas, Zacatecas, Mexico

    D. A. Mercado-Ravell

Authors
  1. M. García-Venegas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. A. Mercado-Ravell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. A. Pinedo-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. A. Carballo-Monsivais
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. A. Mercado-Ravell.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

García-Venegas, M., Mercado-Ravell, D.A., Pinedo-Sánchez, L.A. et al. On the safety of vulnerable road users by cyclist detection and tracking. Machine Vision and Applications 32, 109 (2021). https://doi.org/10.1007/s00138-021-01231-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01231-4

Keywords

Search

Navigation