SpringerLink
Log in

Deep-learning-based road crack detection frameworks for dashcam-captured images under different illumination conditions

  • Neural Networks
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Machine learning techniques have been used to increase detection accuracy of cracks in road surfaces. However, most studies failed to consider variable illumination conditions on the target of interest (ToI), and only focused on detecting the presence or absence of road cracks. This paper proposes a new road crack/defect detection method, IlumiCrack, which integrates Gaussian mixture models (GMM) and object detection CNN models. This work presents several contributions: firstly, a large-scale road crack/defect dataset was prepared using a dashcam with a variety of illumination scenarios. Secondly, experimental evaluations were conducted on 2–4 levels of brightness using GMM for optimal classification. Thirdly, the IlumiCrack framework integrates deep learning-based object detection framework, the YOLO and SSD, to classify road crack and defect images into eight types with high accuracy. In the model training phase, the localization loss was modified to Focal-EIoU, obtaining higher-quality anchor box. Comprehensive model precision and geometric mean (G-mean) achieve 79.1% and 77.1, respectively. Compared to YOLOv3 and SSD, IlumiCrack improves classification accuracy by at least 15.6% on two levels of brightness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Data availibility

Enquiries about data availability should be directed to the authors.

References

  • Carlile B, Delamarter G, Kinney P, Marti A (2017) Improving deep learning by inverse square root linear units (ISRLUs). ar**v preprint 2017

  • Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  • Citizen Service Hotline. Construction bureau of Taichung City government, 2023 (online). Available: https://itunes.apple.com/tw/app/%E8%87%BA%E4%B8%AD%E5%A5%BD%E5%A5%BD%E8%A1%8C/id932232922m?mt=8

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Roy Stat Soc: Ser B (methodol) 39(1):1–22

    MathSciNet  MATH  Google Scholar 

  • Du Y, Pan N, Xu Z, Deng F, Shen Y, Kang H (2021) Pavement distress detection and classification based on YOLO network. Int J Pavement Eng 22(13):1659–1672

    Article  Google Scholar 

  • Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211

    Article  Google Scholar 

  • Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2009) The pascal visual object classes (VOC) challenge. Int J Comput Vision 88:303–308

    Article  Google Scholar 

  • Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput vis 59:167–181

    Article  MATH  Google Scholar 

  • Fujita Y, Hamamoto Y (2011) A robust automatic crack detection method from noisy concrete surfaces. Mach vis Appl 22:245–254

    Article  Google Scholar 

  • Gao X, Boult TE, Coetzee F, Ramesh V (2000, June) Error analysis of background adaption. In: Proceedings IEEE conference on computer vision and pattern recognition. CVPR 2000 (Cat. No. PR00662), 2000

  • Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision

  • Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  • He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-CNN. In: Proceedings of the IEEE international conference on computer vision

  • Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  • Huang Y, Englehart KB, Hudgins B, Chan AD (2005) A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans Biomed Eng 52(11):1801–1811

    Article  Google Scholar 

  • Kuhn M, Johnson K (2013) Applied predictive modeling, vol 26. Springer, New York, p 13

    Book  MATH  Google Scholar 

  • LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  • Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  • Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: Computer vision—ECCV 2016: 14th european conference, Amsterdam, The Netherlands, 2016 October 11–14

  • Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. ar**v preprint

  • Neubeck A, Van Gool L (2006) Efficient non-maximum suppression. In 18th international conference on pattern recognition (ICPR'06) August

  • Okran AM, Abdel-Nasser M, Rashwan HA, Puig D (2022) Effective deep learning-based ensemble model for road crack detection. In: 2022 IEEE international conference on big data (Big Data), 2022 December

  • Protopapadakis E, Stentoumis C, Doulamis N, Doulamis A, Loupos K, Makantasis K, Amditis A (2016) Autonomous robotic inspection in tunnels. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci 3:167–174

    Article  Google Scholar 

  • Redmon J (2013) Darknet: open source neural networks in C (online). Available: http://pjreddie.com/darknet/

  • Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  • Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. ar**v preprint ar**v:1804.02767

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  • Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol 28

  • Saha B, Davies D, Raghavan A (2016) Day night classification of images using thresholding on HSV histogram. U.S. Patent and Trademark Office Patent U.S., Washington DC, Patent No. 9530056

  • Schwenk H, Bengio Y (1997) Training methods for adaptive boosting of neural networks. In: Advances in neural information processing systems, vol 10

  • Shi Y, Cui L, Qi Z, Meng F, Chen Z (2016) Automatic road crack detection using random structured forests. IEEE Trans Intell Transp Syst 17(12):3434–3445

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. ar**v preprint ar**v:1409.1556

  • Sinha SK, Fieguth PW (2006) Automated detection of cracks in buried concrete pipe images. Autom Constr 15(1):58–72

    Article  Google Scholar 

  • Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), 1999 June

  • Taha M, Zayed HH, Nazmy T, Khalifa M (2016) Day/night detector for vehicle tracking in traffic monitoring systems. Int J Comput Inf Eng 10(1):98–104

    Google Scholar 

  • Turkan Y, Hong J, Laflamme S, Puri N (2018) Adaptive wavelet neural network for terrestrial laser scanner-based crack detection. Autom Constr 94:191–202

    Article  Google Scholar 

  • Wan F, Sun C, He H, Lei G, Xu L, **ao T (2022) YOLO-LRDD: a lightweight method for road damage detection based on improved YOLOv5s. EURASIP J Adv Signal Process (1), 98

  • Zalama E, Gómez-García-Bermejo J, Medina R, Llamas J (2014) Road crack detection using visual features extracted by Gabor filters. Comput Aided Civ Infrastruct Eng 29(5):342–358

    Article  Google Scholar 

  • Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by gaussian mixture models. Neurocomputing 171:673–684

    Article  Google Scholar 

  • Zhang YF, Ren W, Zhang Z, Jia Z, Wang L, Tan T (2022) Focal and efficient IOU loss for accurate bounding box regression. Neurocomputing 506:146–157

    Article  Google Scholar 

  • Zhang L, Yang F, Zhang YD, Zhu YJ (2016a) Road crack detection using deep convolutional neural network. In: 2016a IEEE international conference on image processing (ICIP), 2016a, September

Download references

Acknowledgments

We would like to express our gratitude to all colleagues and students who made valuable contributions to this study. I am also grateful to my student Yu-Hao Chen, who helped in data collection. We also acknowledge the editor and reviewers for their constructive criticisms of a previous version of this paper.

Funding

This work was supported in part by National Science and Technology Council under Grant NSC111-2221-E-025-004, and National Taichung University of science and technology under Grant NTCUST111-04.

Author information

Authors and Affiliations

  1. Department of Information Management, National Taichung University of Science and Technology, No. 129, Sec. 3, Sanmin Rd., North Dist., Taichung City, 404, Taiwan, R.O.C.

    Da-Ren Chen & Wei-Min Chiu

Authors
  1. Da-Ren Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei-Min Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

D-RC have substantial contributions to the conception and design of the work. He also planned and performed the acquisition, analysis, and interpretation of data for the work. The drafting and revising work were also completed by him. W-MC performed the data acquisition, and assisted D-RC in analyzing and organizing materials for the work.

Corresponding author

Correspondence to Da-Ren Chen.

Ethics declarations

Conflict of interest

I and another author declare that we have no conflict of interest and Competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, DR., Chiu, WM. Deep-learning-based road crack detection frameworks for dashcam-captured images under different illumination conditions. Soft Comput 27, 14337–14360 (2023). https://doi.org/10.1007/s00500-023-08738-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-08738-0

Keywords

Search

Navigation