Abstract
Along with the advancement in sensing and communication technologies, the explosion in the measurement data collected by structural health monitoring (SHM) systems installed in bridges brings both opportunities and challenges to the engineering community for the SHM of bridges. Deep learning (DL), based on deep neural networks and equipped with high-end computer resources, provides a promising way of using big measurement data to address the problem and has made remarkable successes in recent years. This paper focuses on the review of the recent application of DL in SHM, particularly damage detection, and provides readers with an overall understanding of the missions faced by the SHM of the bridges. The general studies of DL in vibration-based SHM and vision-based SHM are respectively reviewed first. The applications of DL to some real bridges are then commented. A summary of limitations and prospects in the DL application for bridge health monitoring is finally given.
Similar content being viewed by others
1 Introduction
Deterioration accumulation is inevitable during the life-cycle service of bridges subjected to harsh environments, and the failure of bridges will result in considerable losses of both human life and property. Monitoring the bridge condition and detecting their damages are essential to ensure their serviceability and safety. Traditionally, visual inspection conducted by experienced inspectors is the main method adopted for this mission (Xu and ** model among pixel size, shooting distance, and focal length, based on which the actual width of the cracks could be obtained. The results of the verification experiments showed that the recognition precision has achieved at 0.01 mm.
Counting the proportion of the pixels belonging to diseases in all pixels is a workable method to quantify damages like corrosion. Wang et al (2020) proposed a standardized structural health evaluation method and based on it to quantify the damages in the photos of a steel box girder, which were synthesized into panoramas by image stitching technology, and a U-Net was employed to segment the diseases in it. For bolt losing quantification, the Hough line transform -based image processing algorithm was designed to estimate the bolt angles according to the bolt images cropped by R-CNN (Huynh et al 2019). Huynh (2021) designed an autonomous vision-based bolt-looseness detection method with a Faster R-CNN-based bolt detector, an automatic distortion corrector, an adaptive bolt-angle estimator, and a bolt-looseness classifier. Then, the method was applied in a realistic joint of the Dragon Bridge in Danang, Vietnam.
3.2 SHM-GL
3.2.1 Vibration monitoring
Apart from visible damages, vision-based methods are also efficient ways to provide vibration signals to identify invisible damages. Deng et al (2020c) developed an intelligent non-contact remote sensing method in which a uniaxial automatic cruise acquisition device was designed to collect image sequences from bridge surface before they were inputted into a three-dimensional (3D) CNN to identify the envelope spectrum of the holographic deformation. Then, the deflection curvature difference was used to identify the change of damage location and degree. Their experiments demonstrated that the holographic deformation is higher sensitive in damage identification than the limited number of measuring points.
Furthermore, cable forces estimation of urban bridges, according to the drone-captured video, has been realized by Zhang et al (2021). Firstly, a pre-trained FCN was adopted to identify bridge cables and further extract their displacement. Then, EMD was employed for extracting cable vibration signals and eliminating the effect of drone motion. Finally, natural frequencies of the cables were obtained by performing Fourier analysis on extracted cable vibration and further adopted for cable force estimation.
In traditional vision-based vibration measurement methods, template matching algorithm and corner detection algorithm are usually used to track and locate the target, but they are sensitive to the quality of images, which often is poor due to insufficient illumination or fog. Xu et al (2021) thus proposed a distraction-free displacement measurement approach by integrating DL-based Siamese tracker with correlation-based template matching. The DL-based Siamese tracker applied deep feature representations and learned similarity measures for image matching and also considered adaptive template updates with time. The method was then implemented on a short-span footbridge and a long-span road bridge, where its potential to handle challenging scenarios including illumination changes, background variations, and shade effects, was demonstrated. Shao et al (2021) combined the MagicPoint network and the SuperGlue network to achieve target-free full-field 3D vibration displacement measurement and demonstrated the combination’s accuracy compared with traditional sensors, while the combination is more cost effective. Furthermore, they (Shao et al 2022) employed a phased-based video motion magnification algorithm to achieve a higher accuracy of tiny vibrations at the submillimeter level.
3.2.2 Component identification
After various damages are detected, the rating of a structure needs to be provided by a comprehensive assessment in which importance of different components should be considered (Zhu et al 2010). This requires spatially relating identified damages with structural elements. However, inspection images, especially captured by aerial inspection platforms, usually contain complex scenes, wherein structural elements mix with a cluttered background. Extracting structural elements from complex images and sorting them is thus meaningful for SHM.
With a small dataset labeled by inspectors, Karim et al (2021) transferred a Mask R-CNN to segment multi-class bridge components from the videos captured by an UAV. False negatives were recovered by the temporal coherence analysis and a semi-supervised self-training method was developed to engage experienced inspectors in refining the network. The model’s performance reached 91.8% precision, 93.6% recall, and 92.7% F1-score.
Point clouds in 3D space can also provide sufficient information for this purpose. Kim et al (2020) extracted a high-resolution set of point clouds from the full-scale bridge by subspace partition and employed PointNet to classify the points in each subspace. Kim and Kim (2020) compared the performance of three DL models, PointNet, PointCNN, and dynamic graph CNN (DGCNN), in the classification of a point cloud of the bridge components and found that the mean interval over the unit of DGCNN was 86.85, which is higher than the others (see Fig. 10).
Identification results of points clouds in the research of Kim and Kim (2020)
3.2.3 External load
Moving vehicles are one of the main sources of live loads on bridges, and gathering their information is essential for SHM. Bridge weigh-in-motion that exploits bridge components, e.g., decks, girders, and vertical stiffeners, as weighting scales, is the most frequently adopted solution for this purpose, and DL brings efficient solutions for some of its drawbacks.
Zhang et al (2019b) proposed a novel methodology for the mission, in which a Faster R-CNN transferred from ImageNet was employed to detect different types of vehicles frame by frame. Multiple objects tracking algorithm tracked vehicles among different frames and generated the information sequence about each vehicle’s coordinate, type, lane number, and frame number. Then, the image calibration method based on moving standard vehicles was developed to calculate the vehicle length and speed. After acquiring the parameters, the spatiotemporal information could be obtained by vehicle location and the hypothesis of constant speed (see Fig. 11).
The framework for obtaining the spatiotemporal information of vehicles by Zhang et al (2019b)
However, the weight of vehicles cannot be obtained using the method proposed by Zhang et al (2019b). Jian et al (2019) combined CV with the influence line theory to acquire the time-spatial distribution of the vehicle loads on bridges. YOLO V3 was used to identify vehicle positions, types, and axle numbers. Then, vehicle weight was calculated by combining the strain influence line calibrated by field tests and the strain time-history. However, since only three scenarios of vehicle distribution were taken into consideration, the method may face obstacles in complicated traffic scenarios. To overcome this problem, a least square-based identification method that can utilize the redundant strain data measured by a network of strain sensors was proposed to distinguish complicated traffic modes and reduced the recognition errors through solving the overdetermined inverse influence equations (Pathirage et al 2019).
An approach for obtaining spatiotemporal information of vehicles on bridges based on 3D bounding box reconstruction was also proposed by Zhu et al (2021), in which CNN and YOLO were used to detect vehicles and get their 2D bounding box. A 3D bounding box reconstruction method based on the relationship between 2D and 3D bounding box was then developed to get the size and position of vehicles, and the spatiotemporal information of the vehicle could be finally obtained by using multiple objects tracking algorithm.
4 Application of DL in real bridges
The capability of DL encourages the exploration of various approaches that are able to overcome the challenges in traditional SHM, but most of them were verified just in simulation or laboratory. It cannot be denied that more details, like the platform used to collect images and the programs with user interface, need to be taken into consideration for promoting the application of these methods in practice (Xu 2018). This section summarized some efforts devoted to dealing with important details and the systems with DL that have been applied in actual structures.
A framework for autonomous bridge inspection using a UAV was proposed and applied to the Pahtajokk Bridge by Mirzazade et al (2021). Planning the most efficient flight path that could cover the damaged field with the minimum number of images was the first step. Then, three CNN models, SegNet, Inception v3, and U-Net, were trained to conduct bridge component detection, damage area recognition, and crack segmentation, respectively. The third step was to generate a dense point cloud for the damaged areas via intelligent hierarchical dense structure from motion and align it to the overall point cloud for the construction of the digital model of the bridge. Finally, damages were quantified based on the global coordinates of the detected damages.
Kruachottikul et al (2021) described a DL-based visual defect inspection system for reinforced concrete bridges, which consisted of four components. A mobile phone that could take photos was the first part. The second part identified images with defects via a modified ResNet-50, and the defects was classified using another modified ResNet-50 in the third part. Finally, damage severity was quantified by an ANN in the last part. The system’s accuracy for defect detection, classification, and severity prediction were 90.4%, 81%, and 78%, respectively, which had been accepted by Thailand’s Department of Highways for practical use.
Jang et al (2021) developed a ring-type climbing robot system composed of multiple cameras, a climbing robot, and a control computer. The raw images captured under close-up scanning conditions were proposed through feature control-based image stitching, DL-based semantic segmentation, and Euclidean distance transform-based crack quantification algorithms, based on which a digital crack map of the target bridge pier could be established. The test results conducted on the Jang-Duck bridge in South Korea revealed that the method successfully evaluated cracks of the bridge pier with a precision of 90.92% and recall of 97.47%.
Considering the difficulty to approach some parts of bridges by workforce, such as the bottom of decks, He et al (2022) proposed a smart unmanned surface vessel (USV) system for damage detection (see Fig. 12). A novel anchor-free network, CenWholeNet, which focused on center points and holistic information, was proposed, and a parallel attention module was introduced into the model innovatively. For the platform, a USV system without the global positioning systems (GPS) navigation, supporting real-time transmission of lidar and video information was designed.
The system developed by He et al (2022)
Vehicle-assisted monitoring is a promising alternative for rapid and low-cost bridge health monitoring compared with instrumentation installed on bridges. Sarwar and Cantero (2021) developed an indirect bridge monitoring system, in which a DAE was trained by the vertical acceleration responses of a fleet of vehicles passing over a healthy bridge. Then, the Kullback-Leibler divergence between the measured and the reconstructed signals was used for damage detection and severity quantification.
Mobile devices such as smartphones can be not only a sensing platform but also a computing platform to conduct on-site damage detection. However, due to the limited computing resources of mobile devices, the size of the DNN needs to be reduced. Ye et al (2022) developed pruned crack recognition network by reducing DNN size via the pruning method and designed a DL-based crack detection program for smartphones. In order to conduct crack detection by Internet of Things (IoT) devices in real-time, Kim et al (2021) proposed OleNet by fine-tuning the hyperparameters of LeNet-5. Compared with other pretrained DL models, including VGG16, Inception, and ResNet, OleNet achieved the maximum accuracy of 99.8% in the minimum computation. Shrestha and Dang (2020) developed a program integrated with CNN to realize accurate and real-time bridge vibration classification according to the multi-channel time-series signals acquired by the built-in accelerometers of smart phones.
5 Conclusions
In this paper, the applications of DL models in SHM, particularly damage detection of bridges, have been summarized systematically. It is easy to find that the excellent capability of DL models in addressing obstacles in the traditional SHM methods of the bridges has been demonstrated by the applications not only in laboratories but also in real bridges. Each of the DL models promotes the realization of a more intelligent SHM. However, it cannot be denied that drawbacks exist in every method. Some of the challenges can be listed as follows:
-
1.
Most of the current studies consider only one type of monitoring data in damage detection. If this type of monitoring data is anormal, the damage detection will fail no matter how good the damage detection method is.
-
2.
Although several attempts have been conducted to realize the targets by unsupervised learning, most of the applications still rely on pre-defined damage scenarios and training data, which pose a considerable requirement of engineering experience and labor.
-
3.
The conditions of laboratories, where the majority of methods were validated, are idealized. The robustness of DL models needs to be further enhanced to combat environmental interference in practice, such as the vibration induced by external loads and motion blur when UAVs are employed.
-
4.
The weak connection between the two levels of vision-based SHM results in difficulties in comprehensive condition assessment, for which visible defects and invisible damages need to be considered at the same time.
After considering the limitations listed above and recent achievements in DL, the following directions are promising and worthy to be further investigated:
-
1.
Fusing multiple types of information collected by SHM system: With advances of multiple types of sensors, the SHM system can provide multiple types of structural information. Fusing and leveraging the multiple types of information in structural condition assessment via DL methods is a promising way to enhance the methods’ practicality.
-
2.
Building larger training databases collected from the real world: Training DL models with the data containing actual interference is an efficient path to improve their robustness, and the availability of advanced sensors and UAVs nowadays makes it possible to build larger databases consisting of real samples.
-
3.
Utilization of mobile and IoT devices: Mobile devices, such as smartphones, can be not only a sensing platform with various built-in sensors, including magnetometer, gyroscope, accelerometer, and GPS, but also a computing platform. Leveraging them by deploying lightweight DL models makes on-site damage detection available. In addition, the IoT devices, which emerge with the innovation in data transmission and cloud-based computation, provide an efficient way to obtain and integrate different types of structural data, which will prompt a cost-minimized and automatic SHM.
-
4.
Digital twin: In order to make a reliable assessment reflecting the true condition of structural elements, an ensemble of multi-scale DL models is needed to interpret and integrate the data from both the local level and global level of SHM. Digital twin that tries to replicate physical entity in digital world (Lin et al 2021) provides a powerful platform for this mission, in which various damages can be reconstructed and evaluated at the same time. Integrating SHM and digital twin may be a promising way to realize the smart civil structure, even smart city.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Abbreviations
- 1D:
-
One-Dimensional
- 2D:
-
Two-Dimensional
- AE:
-
Autoencoder
- ANN:
-
Artificial Neural Network
- ASPP:
-
Atrous Spatial Pyramid Pooling
- CNN:
-
Convolutional Neural Network
- CV:
-
Computer Vision
- DAE:
-
Deep Autoencoder
- DBN:
-
Deep Belief Network
- DGCNN:
-
Dynamic Graph Convolutional Neural Network
- DL:
-
Deep Learning
- DNN:
-
Deep Neural Network
- EMA:
-
Expected Maximum Attention
- EMD:
-
Empirical Mode Decomposition
- FCN:
-
Fully Connected Neural Network
- FEM:
-
Finite Element Model
- FFT:
-
Fast Fourier Transform
- GAN:
-
Generative Adversarial Network
- GNN:
-
Graph Neural Network
- GPS:
-
Global Positioning Systems
- GRU:
-
Gated Recurrent Unit
- HDCB:
-
Hybrid Dilated Convolutional Block
- IoT:
-
Internet of Things
- LSTM:
-
Long Short-Term Memory Network
- MAP:
-
Maximum A-Posteriori Probabilities
- ML:
-
Machine Learning
- MTF:
-
Markov Transition Field
- RNN:
-
Recurrent Neural Network
- SAE:
-
Stacked Autoencoder
- SHM:
-
Structural Health Monitoring
- SHM-GL:
-
Structural Health Monitoring at Global Level
- SHM-LL:
-
Structural Health Monitoring at Local Level
- UAV:
-
Unmanned Aerial Vehicle
- USV:
-
Unmanned Surface Vessel
- UW:
-
Uniform Weights
- VMD:
-
Variational Mode Decomposition
- YOLO:
-
You Only Look Once
References
Adeli H, Yeh C (1989) Perceptron learning in engineering design. Comput Aided Civ Inf 4(4):247–256
Ahmed H, La HM, Gucunski N (2020) Review of non-destructive civil infrastructure evaluation for bridges: state-of-the-art robotic platforms, sensors and algorithms. Sensor 20(14):3954
Alazzawi O, Wang D (2022) A novel structural damage identification method based on the acceleration responses under ambient vibration and an optimized deep residual algorithm. Struct Health Monit 21(6):2587–2617
Ali R, Cha Y (2019) Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer. Constr Build Mater 226:376–387
Azimi M, Eslamlou AD, Pekcan G (2020) Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors 20(10):2778
Bae H, Jang K, An YK (2021) Deep super resolution crack network (SrcNet) for improving computer vision-based automated crack detectability in in situ bridges. Struct Health Monit 20(4):1428–1442
Bao Y, Li H (2021) Machine learning paradigm for structural health monitoring. Struct Health Monit 20(4):1353–1372
Byung K, Branko G, Yousok K, Hyo S (2020) Convolutional neural network–based data recovery method for structural health monitoring. Struct Health Monit 19(6):1821–1838
Cha Y, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Inf 3(9):731–747
Chen R (2021) Migration learning-based bridge structure damage detection algorithm. Sci Program Neth 2021:1102521
Chen Z, Li H, Bao Y (2019) Analyzing and modeling inter-sensor relationships for strain monitoring data and missing data imputation: a copula and functional data-analytic approach. Struct Health Monit 18(4):1168–1188
Chen Z, Wang Y, Wu J, Deng C, Hu K (2021) Sensor data-driven structural damage detection based on deep convolutional neural networks and continuous wavelet transform. Appl Intell 51(8):5598–5609
Cui M, Wu G, Chen Z, Dang J, Zhou M, Feng D (2021a) Geometric attention regularization enhancing convolutional neural networks for bridge rubber bearing damage assessment. J Perform Constr Facil 35(5):04021061
Cui X, Wang Q, Dai J, Zhang R, Li S (2021b) Intelligent recognition of erosion damage to concrete based on improved YOLO-v3. Mater Lett 302:130363
Dang HV, Tran-Ngoc H, Nguyen TV, Bui-Tien T, De Roeck G, Nguyen HX (2021) Data-driven structural health monitoring using feature fusion and hybrid deep learning. IEEE T Autom Sci Eng 18(4):2087–2103
Deng G, Zhou Z, Chu X, Shao S (2020c) Identification of behavioral features of bridge structure based on static image sequences. Adv Civ Eng 2020:2815017
Deng J, Lu Y, Lee VCS (2020a) Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network. Comput Aided Civ Inf 35(4):373–388
Deng J, Lu Y, Lee VCS (2021) Imaging-based crack detection on concrete surfaces using you only look once network. Struct Health Monit 20(2):484–499
Deng W, Mou Y, Kashiwa T, Escalera S, Nagai K, Nakayama K, Matsuo Y, Prendinger H (2020b) Vision based pixel-level bridge structural damage detection using a link ASPP network. Autom Constr 110:102973
Dong C, Catbas FN (2021) A review of computer vision-based structural health monitoring at local and global levels. Struct Health Monit 20(2):692–743
Du Y, Li L, Hou R, Wang X, Tian W, **a Y (2022) Convolutional neural network-based data anomaly detection considering class imbalance with limited data. Smart Struct Syst 29(1):63–75
Duan Y, Chen Q, Zhang H, Yun C, Wu S, Zhu Q (2019) CNN-based damage identification method of tied-arch bridge using spatial-spectral information. Smart Struct Syst 23(5):507–520
Ebenezer AS, Kanmani SD, Sheela V, Ramalakshmi K, Chandran V, Sumithra MG, Elakkiya B, Murugesan B (2021) Identification of civil infrastructure damage using ensemble transfer learning model. Adv Civ Eng 2021:5589688
Fan G, Li J, Hao H (2019) Lost data recovery for structural health monitoring based on convolutional neural networks. Struct Contrl Hlth 26(10):e2433
Farrar CR, Worden K (2012) Structural health monitoring: a machine learning perspective. Wiley, New York
Fu L, Tang Q, Gao P, **n J, Zhou J (2021) Damage identification of long-span bridges using the hybrid of convolutional neural network and long short-term memory network. Algorithms 14(6):180
Guo Q, Feng L, Zhang R, Yin H (2020) Study of damage identification for bridges based on deep belief network. Adv Struct Eng 23(8):1562–1572
Han JH, Kim IS, Lee CH, Moon YS (2020) Crack detection method for tunnel lining surfaces using ternary classifier. KSII T Internet Inf 14(9):3797–3822
He H, Zheng J, Liao L, Chen Y (2021b) Damage identification based on convolutional neural network and recurrence graph for beam bridge. Struct Health Monit 20(4):1392–1408
He Y, Chen H, Liu D, Zhang L (2021a) A framework of structural damage detection for civil structures using fast fourier transform and deep convolutional neural networks. Appl Sci Basel 11(19):9345
He Z, Jiang S, Zhang J, Wu G (2022) Automatic damage detection using anchor-free method and unmanned surface vessel. Autom Constr 133:104017
Holm E, Transeth AA, Knudsen OO, Stahl A (2020) Classification of corrosion and coating damages on bridge constructions from images using convolutional neural networks. In: 12th international conference on machine vision (ICMV 2019) 11433, p 1143320
Hormozabad SJ, Soto MG (2021) Real-time damage identification of discrete structures via neural networks subjected to dynamic loading. In: Conference on Health Monitoring of Structural and Biological Systems XV, 115932, p 115932O
Housner GW, Bergman LA, Caughey TK, Chassiakos AG, Claus RO (1997) Structural control: past, present, and future. J Eng Mech 123(9):897–971
Huynh TC (2021) Vision-based autonomous bolt-looseness detection method for splice connections: design, lab-scale evaluation, and field application. Autom Constr 124:103591
Huynh TC, Park JH, Jung HJ, Kim JT (2019) Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing. Autom Constr 105:102844
Ibrahim A, Eltawil A, Na Y, El-Tawil S (2020) A machine learning approach for structural health monitoring using noisy data sets. IEEE T Autom Sci Eng 17(2):900–908
James MWB, Alessandro DS, Xu YL, Helmut W, Emin A (2011) Vibration-based monitoring of civil infrastructure: challenges and successes. J Civ Struct Health 1:79–95
Jang K, An YK, Kim B, Cho S (2021) Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot. Comput Aided Civ Inf 36(1):14–29
Jena SP, Parhi DR (2020) Fault detection in cracked structures under moving load through a recurrent-neural-networks-based approach. Sci Iran 27(4):1886–1896
Jeong E, Seo J, Wacker J (2020) Literature review and technical survey on bridge inspection using unmanned aerial vehicles. J Perform Constr Facil 34(6):04020113
Jeong S, Ferguson M, Hou R, Lynch JP, Sohn H, Law KH (2019) Sensor data reconstruction using bidirectional recurrent neural network with application to bridge monitoring. Adv Eng Inform 42:100991
Jian X, **a Y, Lozano-Galant JA, Sun L (2019) Traffic sensing methodology combining influence line theory and computer vision techniques for girder bridges. J Sensors 2019:3409525
Jian X, Zhong H, **a Y, Sun L (2021) Faulty data detection and classification for bridge structural health monitoring via statistical and deep-learning approach. Struct Control Health Monit 28(11):e2824
Jiang H, Wan C, Yang K, Ding Y, Xue S (2021a) Continuous missing data imputation with incomplete dataset by generative adversarial networks-based unsupervised learning for long-term bridge health monitoring. Struct Health Monit 21(3):1093–1109
Jiang W, Liu M, Peng Y, Wu L, Wang Y (2021b) HDCB-net: a neural network with the hybrid dilated convolution for pixel-level crack detection on concrete bridges. IEEE Ind Inform 17(8):5485–5494
Karim MM, Qin R, Chen G, Yin Z (2021) A semi-supervised self-training method to develop assistive intelligence for segmenting multiclass bridge elements from inspection videos. Struct Health Monit 21(3):835–852
Khodabandehlou H, Pekcan G, Fadali MS (2019) Vibration-based structural condition assessment using convolution neural networks. Struct Control Hlth 26(2):e2308
Kim B, Yuvaraj N, Preethaa KRS, Pandian RA (2021) Surface crack detection using deep learning with shallow CNN architecture for enhanced computation. Neural Comput & Applic 33(15):9289–9305
Kim H, Kim C (2020) Deep-learning-based classification of point clouds for bridge inspection. Remote Sens 12(22):3757
Kim H, Yoon J, Sim SH (2020) Automated bridge component recognition from point clouds using deep learning. Struct Control Hlth 27(9):e2591
Kruachottikul P, Cooharojananone N, Phanomchoeng G, Chavarnakul T, Kovitanggoon K, Trakulwaranont D (2021) Deep learning-based visual defect-inspection system for reinforced concrete bridge substructure: a case of Thailand’s department of highways. J Civ Struct Health 11(4):949–965
Lee JS, Hwang SH, Choi IY, Choi Y (2020b) Estimation of crack width based on shape-sensitive kernels and semantic segmentation. Struct Control Hlth 27(4):e2504
Lee JS, Kim HM, Kim SI, Lee HM (2021) Evaluation of structural integrity of railway bridge using acceleration data and semi-supervised learning approach. Eng Struct 239:112330
Lee K, Byun N, Shin DH (2020a) A damage localization approach for rahmen bridge based on convolutional neural network. KSCE J Civ Eng 24(1):1–9
Li D, Ho SC, Song G, Ren L, Li H (2015) A review of damage detection methods for wind turbine blades. Smart Mater Struct 24(3):033001
Li D, Liang Z, Ren W, Yang D, Wang S, **ang S (2021c) Structural damage identification under nonstationary excitations through recurrence plot and multi-label convolutional neural network. Measurement 186:110101
Li L, Liu G, Zhang L, Li Q (2021a) FS-LSTM-based sensor fault and structural damage isolation in SHM. IEEE Sensors J 21(3):3250–3259
Li L, Zhou H, Liu H, Zhang C, Liu J (2021b) A hybrid method coupling empirical mode decomposition and a long short-term memory network to predict missing measured signal data of SHM systems. Struct Health Monit 20(4):1778–1793
Li S, Niu J, Li Z (2021d) Novelty detection of cable-stayed bridges based on cable force correlation exploration using spatiotemporal graph convolutional networks. Struct Health Monit 20(4):2216–2228
Li S, Sun L (2020) Detectability of bridge-structural damage based on fiber-optic sensing through deep-convolutional neural networks. J Bridg Eng 25(4):04020012
Li S, Zuo X, Li Z, Wang H (2020) Applying deep learning to continuous bridge deflection detected by fiber optic gyroscope for damage detection. Sensors 20(3):911
Lin K, Xu YL, Lu X, Guan Z, Li J (2021) Digital twin-based collapse fragility assessment of a long-span cable-stayed bridge under strong earthquakes. Autom Constr 123:103547
Liu G, Niu Y, Zhao W, Duan Y, Shu J (2022) Data anomaly detection for structural health monitoring using a combination network of GANomaly and CNN. Smart Struct Syst 29(1):53–62
Liu H, Ding Y, Zhao H, Wang M, Geng F (2020) Deep learning-based recovery method for missing structural temperature data using LSTM network. Struct Monit Maint 7(2):109–124
Lu W, Teng J, Li C, Cui Y (2017) Reconstruction to sensor measurements based on a correlation model of monitoring data. Appl Sci 7(3):243
Ma X, Lin Y, Nie Z, Ma H (2020) Structural damage identification based on unsupervised feature-extraction via variational auto-encoder. Measurement 160:107811
Mangalathu S, Jeon JS (2020) Ground motion-dependent rapid damage assessment of structures based on wavelet transform and image analysis techniques. J Struct Eng 146(11):04020230
Mantawy IM, Mantawy MO (2022) Convolutional neural network based structural health monitoring for rocking bridge system by encoding time-series into images. Struct Control Hlth 29(3):e2897
Miao X, Wang J, Wang Z, Sui Q, Gao Y, Jiang P (2019) Automatic recognition of highway tunnel defects based on an improved u-net model. IEEE Sensors J 19(23):11413–11423
Mirzazade A, Popescu C, Blanksvard T, Taljsten B (2021) Workflow for off-site bridge inspection using automatic damage detection-case study of the pahtajokk bridge. Remote Sens 13(14):2665
Mondal TG, Jahanshahi MR, Wu RT, Wu Z (2020) Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance. Struct Control Hlth 27(4):e2507
Mosalam K, Muin S, Gao Y (2019) New directions in structural health monitoring. NED Univ J Res 2:77–112
Mousavi M, Gandomi AH (2021a) Prediction error of Johansen cointegration residuals for structural health monitoring. Mech Syst Singal Pr 160:107847
Mousavi M, Gandomi AH (2021b) Structural health monitoring under environmental and operational variations using MCD prediction error. J Sound Vib 512:116370
Nguyen DH, Nguyen QB, Bui-Tien T, De Roeck G, Wahab MA (2020) Damage detection in girder bridges using modal curvatures gapped smoothing method and convolutional neural network: application to Bo Nghi bridge. Thero Appl Fract Mec 109:102728
Ni F, Zhang J, Chen Z (2019) Zernike-moment measurement of thin-crack width in images enabled by dual-scale deep learning. Comput Aided Civ Inf 34(5):367–384
Pal M, Palevicius P, Landauskas M, Orinaite U, Timofejeva I, Ragulskis M (2021) An overview of challenges associated with automatic detection of concrete cracks in the presence of shadows. Appl Sci Basel 11(23):11396
Pan H, Azimi M, Gui G, Yan F, Lin Z (2017) Vibration-based support vector machine for structural health monitoring. In: International Conference on Experimental Vibration Analysis for Civil Engineering Structures, pp 167–178
Pan H, Azimi M, Yan F, Lin Z (2018) Time-frequency-based data-driven structural diagnosis and damage detection for cable-stayed bridges. J Bridg Eng 23(6):04018033
Pathirage CSN, Li J, Li L, Hao H, Liu W, Ni P (2018) Structural damage identification based on autoencoder neural networks and deep learning. Eng Struct 172:13–28
Pathirage CSN, Li J, Li L, Hao H, Liu W, Wang R (2019) Development and application of a deep learning–based sparse autoencoder framework for structural damage identification. Struct Health Monit 18(1):103–122
Perez H, Tah JHM, Mosavi A (2019) Deep learning for detecting building defects using convolutional neural networks. Sensors 19(16):3556
Qiao W, Ma B, Liu Q, Wu X, Li G (2021) Computer vision-based bridge damage detection using deep convolutional networks with expectation maximum attention module. Sensors 21(3):824
Quqa S, Martakis P, Movsessian A, Pai S, Reuland Y, Chatzi E (2022) Two-step approach for fatigue crack detection in steel bridges using convolutional neural networks. J Civ Struct Health 12(1):127–140
Rastin Z, Amiri GG, Darvishan E (2021a) Generative adversarial network for damage identification in civil structures. Shock Vib 2021:3987835
Rastin Z, Amiri GG, Darvishan E (2021b) Unsupervised structural damage detection technique based on a deep convolutional autoencoder. Shock Vib 2021:6658575
Rubio JJ， Kashiwa T, Laiteerapong T, Deng W, Nagai K, Escalera S, Nakayama K, Matsuo Y, Prendinger H (2019) Multi-class structural damage segmentation using fully convolutional networks. Comput Ind 112: 103121
Sajedi SO, Liang X (2019) A convolutional cost-sensitive crack localization algorithm for automated and reliable RC bridge inspection. In: Risk-based bridge engineering: proceedings of the 10th New York City bridge conference 2019, p 229
Sarwar MZ, Cantero D (2021) Deep autoencoder architecture for bridge damage assessment using responses from several vehicles. Eng Struct 246:113064
Savino P, Tondolo F (2021) Automated classification of civil structure defects based on convolutional neural network. Front Struct Civ Eng 15(2):305–317
Shajihan S, Wang S, Zhai G, Spencer BF (2022) CNN based data anomaly detection using multi-channel imagery for structural health monitoring. Smart Struct Syst 29(1):181–193
Shao Y, Li L, Li J, An S, Hao H (2021) Computer vision based target-free 3D vibration displacement measurement of structures. Eng Struct 246:113040
Shao Y, Li L, Li J, An S, Hao H (2022) Target-free 3D tiny structural vibration measurement based on deep learning and motion magnification. J Sound Vib 538(10):117244
Sharma S, Sen S (2020) One-dimensional convolutional neural network-based damage detection in structural joints. J Civ Struct Health 10(5):1057–1072
Shin HK, Ahn YH, Lee SH, Kim HY (2020) Automatic concrete damage recognition using multi-level attention convolutional neural network. Materials 13(23):5549
Shrestha A, Dang J (2020) Deep learning-based real-time auto classification of smartphone measured bridge vibration data. Sensor 20(9):2710
Silva M, Santos A, Santos R, Figueiredo E, Sales C, Costa JC (2019) Deep principal component analysis: an enhanced approach for structural damage identification. Struct Health Monit 18(5–6):1444–1463
Silva MF, Santos A, Santos R, Figueiredo E, Costa JCWA (2021) Damage-sensitive feature extraction with stacked autoencoders for unsupervised damage detection. Struct Control Hlth 28(5):e2714
Sofi A, Regita JJ, Rane B, Lau HH (2022) Structural health monitoring using wireless smart sensor network-an overview. Mech Syst Signal Pr 163:108113
Son H, Pham VT, Jang Y, Kim SE (2021) Damage localization and severity assessment of a cable-stayed bridge using a message passing neural network. Sensors 21(9):3118
Song Q, Chen Y, Oskoui EA, Fang Z, Taylor T (2020) Micro-crack detection method of steel beam surface using stacked autoencoders on massive full-scale sensing strains. Struct Health Monit 19(4):1175–1187
Sony S, Gamage S, Sadhu A, Samarabandu J (2022) Vibration-based multiclass damage detection and localization using long short-term memory networks. Structures 35:436–451
Sun L, Shang Z, **a Y, Bhowmick S, Nagarajaiah S (2020) Review of bridge structural health monitoring aided by big data and artificial intelligence: from condition assessment to damage detection. J Struct Eng 146(5):04020073
Tang Z, Chen Z, Bao Y, Li H (2019) Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Health Monit 26(1):e2296
Teng S, Chen G, Liu Z, Cheng L, Sun X (2021) Multi-sensor and decision-level fusion-based structural damage detection using a one-dimensional convolutional neural network. Sensors 21(12):3950
Teng Z, Teng S, Zhang J, Chen G, Cui F (2020) Structural damage detection based on real-time vibration signal and convolutional neural network. Appl Sci 10(14):4720
Wang D, Zhang Y, Pan Y, Peng B, Liu H, Ma R (2020) An automated inspection method for the steel box girder bottom of long-span bridges based on deep learning. IEEE Access 8:94010–94023
Wang R, Chencho ASJ, Li J, Hao H, Liu W (2021) Deep residual network framework for structural health monitoring. Struct Health Monit 20(4):1443–1461
Wang W, Su C (2022) Automatic classification of reinforced concrete bridge defects using the hybrid network. Arab J Sci Eng 47(4):5187–5197
**ao H, Wang W, Dong L, Ogai H (2021) A novel bridge damage diagnosis algorithm based on deep learning with gray relational analysis for intelligent bridge monitoring system. IEEJ T Electr 16(5):743–753
**ao X, Xu YL, Zhu Q (2015) Multi-scale modelling and model updating of a cable-stayed bridge, part II: model updating using modal frequencies and influence lines. J Bridg Eng 20(10):04014113
Xu YL (2018) Making good use of structural health monitoring systems of long-span cable-supported bridges. J Civ Struct Health 8(3):477–497
Xu YL, **a Y (2012) Structural health monitoring of long-span suspension bridges. Spon Press (Taylor& Francis), UK
Xu Y, Zhang J, Brownjohn J (2021) An accurate and distraction-free vision-based structural displacement measurement method integrating Siamese network-based tracker and correlation-based template matching. Measurement 179:109506
Yang J, Zhang L, Chen C, Li Y, Li R, Wang G, Jiang S, Zeng Z (2020) A hierarchical deep convolutional neural network and gated recurrent unit framework for structural damage detection. Inf Sci 540:117–130
Yang K, Ding Y, Sun P, Jiang H, Wang Z (2021) Computer vision-based crack width identification using F-CNN model and pixel nonlinear calibration. Struct Infrastruct E 2021:1994617
Yang S, Huang Y (2021) Damage identification method of prestressed concrete beam bridge based on convolutional neural network. Neural Comput & Applic 33(2):535–545
Yao Y, Tung STE, Glisic B (2014) Crack detection and characterization techniques-An overview. Struct Control Hlth 21(12):1387–1413
Ye X, ** T, Chen PY (2019) Structural crack detection using deep learning-based fully convolutional networks. Adv Struct Eng 22(16):3412–3419
Ye X, Li Z, ** T (2022) Smartphone-based structural crack detection using pruned fully convolutional networks and edge computing. Smart Struct Syst 29(1):141–151
Yin H, Gai K (2015) An empirical study on preprocessing high-dimensional class-imbalanced data for classification. In: 2015 IEEE 17th international conference on high performance computing and communications, 2015 IEEE 7th international symposium on cyberspace safety and security, and 2015 IEEE 12th international conference on embedded software and systems, pp 1314–1319
Zhang B, Zhou L, Zhang J (2019b) A methodology for obtaining spatiotemporal information of the vehicles on bridges based on computer vision. Comput Aided Civ Inf 34(6):471–487
Zhang C, Chang C, Jamshidi M (2020) Concrete bridge surface damage detection using a single-stage detector. Comput Aided Civ Inf 35(4):389–409
Zhang C, Tian Y, Zhang J (2021) Complex image background segmentation for cable force estimation of urban bridges with drone-captured video and deep learning. Struct Contrl Hlth 29(4):e2910
Zhang L, Shen J, Zhu B (2022a) A review of the research and application of deep learning-based computer vision in structural damage detection. Earthq Eng Vib 21(1):1–21
Zhang Y, Lei Y (2021) Data anomaly detection of bridge structures using convolutional neural network based on structural vibration signals. Symmetry-Basel 13(7):1186
Zhang Y, Miyamori Y, Mikami S, Saito T (2019a) Vibration-based structural state identification by a 1-dimensional convolutional neural network. Comput Aided Civ Inf 34(9):822–839
Zhang Z, Yan J, Li L, Pan H, Dong C (2022b) Condition assessment of stay cables through enhanced time series classification using a deep learning approach. Smart Struct Syst 29(1):105–116
Zhao R, Yan R, Chen Z, Mao K, Wang P, Gao R (2019) Deep learning and its applications to machine health monitoring. Mech Syst Signal Pr 115:213–237
Zhu J, Li X, Zhang C, Shi T (2021) An accurate approach for obtaining spatiotemporal information of vehicle loads on bridges based on 3D bounding box reconstruction with computer vision. Measurement 181:109657
Zhu J, Zhang C, Qi H, Lu Z (2020) Vision-based defects detection for bridges using transfer learning and convolutional neural networks. Struct Infrastruct E 16(7):1037–1049
Zhu Q, Xu YL, **ao X (2015) Multi-scale modelling and model updating of a cable-stayed bridge, part I: modelling and influence line analysis. J Bridg Eng 20(10):04014112
Zhu Z, German S, Brilakis I (2010) Detection of large-scale concrete columns for automated bridge inspection. Autom Constr 19(8):1047–1055
Acknowledgements
The works described in this paper are financially supported by the Changjiang Scholars Program of the Ministry of Education of China (SWJTU-YH1199911012201) to which the authors are most grateful. Any opinions and conclusions presented in this paper are entirely those of the authors.
Funding
The Changjiang Scholars Program of the Ministry of Education of China: SWJTU-YH1199911012201.
Author information
Authors and Affiliations
Contributions
Guoqing Zhang reviewed the correlated literature and drafted the manuscript. Bin Wang and Jun Li proposed the frame of the paper and provided worthy advice. You-lin Xu revised the manuscript and advised the summary and prospects. The author(s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
You-lin Xu is an honorary adviser and Bin Wang is a manager editor for Advances in Bridge Engineering. They were not involved in the editorial review, or the decision to publish this article. All authors declare that there are no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhang, GQ., Wang, B., Li, J. et al. The application of deep learning in bridge health monitoring: a literature review. ABEN 3, 22 (2022). https://doi.org/10.1186/s43251-022-00078-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s43251-022-00078-7