SpringerLink
Log in

A machine learning-based data augmentation strategy for structural damage classification in civil infrastructure system

  • Original Paper
  • Published:
Journal of Civil Structural Health Monitoring Aims and scope Submit manuscript

Abstract

With the explosive growth of available data collected from various sensors and increasingly powerful computing resources, recent advanced in machine learning (ML) and other data-driven techniques have yielded transformative results in structural health monitoring (SHM). A common challenge in applying ML algorithms to SHM problems in a supervised learning manner is that the ML models need a comprehensive and systematic training process, typically requiring large datasets representing structures under different damage conditions, a requirement that is usually not met when dealing with real-life civil structures. In this study, we propose a novel data augmentation strategy within a vibration-based SHM framework, based on a conditional variational autoencoder (CVAE) to generate additional data samples of the power cepstral coefficients of the structural acceleration response in various damage conditions. The developed CVAE architecture can be trained to model various statistical distributions of the power cepstral coefficients simultaneously by incorporating newly defined conditional variables, where the decoder of the trained CVAE is subsequently separated and used to generate new samples for the data augmentation. A new type of the power cepstral coefficients is used to boost the performance and robustness of the data augmentation and consequently of the subsequent damage classification task. The augmented training datasets can be used to better train a probabilistic linear discriminant analysis (PLDA) model that can be employed for structural damage identification and classification in both supervised and unsupervised learning strategies. The proposed method has been validated by numerical simulations and experimental data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The data that support the findings of this study are not openly available due to reasons of sensitivity. The data are, however, available from the authors upon reasonable request and with permission from the Department of Civil Engineering and Engineering Mechanics of Columbia University and the Structural Mechanics Section of KU Leuven.

References

  1. Azimi M, Eslamlou AD, Pekcan G (2020) Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors 20(10):2778

    Article  Google Scholar 

  2. Fritzen CP (2005) Vibration-based structural health monitoring–concepts and applications. Key Eng Mater 293:3–20

    Article  Google Scholar 

  3. Avci O, Abdeljaber O, Kiranyaz S, Hussein M, Gabbouj M, Inman DJ (2021) A review of vibration-based damage detection in civil structures: from traditional methods to machine learning and deep learning applications. Mech Syst Signal Process 147:107077. https://doi.org/10.1016/j.ymssp.2020.107077

    Article  Google Scholar 

  4. Farrar CR, Worden K (2012) Structural health monitoring: a machine learning perspective. Wiley

    Book  Google Scholar 

  5. O'Shea K, Nash R (2015) An introduction to convolutional neural networks. ar**v preprint ar**v:1511.08458. https://doi.org/10.48550/ar**v.1511.08458

  6. Ni F, Zhang J, Noori MN (2020) Deep learning for data anomaly detection and data compression of a long-span suspension bridge. Comput-Aided Civ Infrastruct Eng 35(7):685–700. https://doi.org/10.1111/mice.12528

    Article  Google Scholar 

  7. Azimi M, Pekcan G (2020) Structural health monitoring using extremely compressed data through deep learning. Comput-Aided Civ Infrastruct Eng 35(6):597–614. https://doi.org/10.1111/mice.12517

    Article  Google Scholar 

  8. Abdeljaber O, Avci O, Kiranyaz S, Gabbouj M, Inman DJ (2017) Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. J Sound Vib 388:154–170. https://doi.org/10.1016/j.jsv.2016.10.043

    Article  Google Scholar 

  9. Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Comput-Aided Civ Infrastruct Eng 32(5):361–378. https://doi.org/10.1111/mice.12263

    Article  Google Scholar 

  10. Wang Z, Cha YJ (2021) Unsupervised deep learning approach using a deep auto-encoder with a one-class support vector machine to detect damage. Struct Health Monit 20(1):406–425. https://doi.org/10.1177/1475921720934051

    Article  Google Scholar 

  11. Pathirage CS, Li J, Li L, Hao H, Liu W, Ni P (2018) Structural damage identification based on autoencoder neural networks and deep learning. Eng Struct 172:13–28. https://doi.org/10.1016/j.engstruct.2018.05.109

    Article  Google Scholar 

  12. Li L, Morgantini M, Betti R (2023) Structural damage assessment through a new generalized autoencoder with features in the quefrency domain. Mech Syst Signal Process 184:109713. https://doi.org/10.1016/j.ymssp.2022.109713

    Article  Google Scholar 

  13. Morgantini M, Betti R, Balsamo L (2021) Structural damage assessment through features in quefrency domain. Mech Syst Signal Process 147:107017. https://doi.org/10.1016/j.ymssp.2020.107017

    Article  Google Scholar 

  14. Dike HU, Zhou Y, Deveerasetty KK, Wu Q (2018) Unsupervised learning based on artificial neural network: a review. In 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), 322–327. https://doi.org/10.1109/CBS.2018.8612259

  15. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big data 3(1):1–40. https://doi.org/10.1186/s40537-016-0043-6

    Article  Google Scholar 

  16. Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowl-Based Syst 80:14–23. https://doi.org/10.1016/j.knosys.2015.01.010

    Article  Google Scholar 

  17. Gao Y, Mosalam KM (2018) Deep transfer learning for image-based structural damage recognition. Comput-Aided Civ Infrastruct Eng 33(9):748–768. https://doi.org/10.1111/mice.12363

    Article  Google Scholar 

  18. Tronci EM, Beigi H, Feng MQ, Betti R (2022) Transfer learning from audio domains a valuable tool for structural health monitoring. Dynamics of civil structures, vol 2. Springer, Cham, pp 99–107. https://doi.org/10.1007/978-3-030-77143-0_11

    Chapter  Google Scholar 

  19. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48. https://doi.org/10.1186/s40537-019-0197-0

    Article  Google Scholar 

  20. Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning. ar**v preprint ar**v:1712.04621. https://doi.org/10.48550/ar**v.1712.04621

  21. Feng SY, Gangal V, Wei J, Chandar S, Vosoughi S, Mitamura T, Hovy E (2021) A survey of data augmentation approaches for NLP. ar**v preprint ar**v:2105.03075. https://doi.org/10.48550/ar**v.2105.03075

  22. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks: an overview. IEEE Signal Process Mag 35(1):53–65. https://doi.org/10.1109/MSP.2017.2765202

    Article  Google Scholar 

  23. Luleci F, Catbas FN, Avci O (2022) A literature review: generative adversarial networks for civil structural health monitoring. Front Built Environ Struct Sens Control Asset Manag 8:1027379. https://doi.org/10.3389/fbuil.2022.1027379/full

    Article  Google Scholar 

  24. Wan P, He H, Guo L, Yang J, Li J (2021) InfoGAN-MSF: a data augmentation approach for correlative bridge monitoring factors. Meas Sci Technol 32(11):114008. https://doi.org/10.1088/1361-6501/ac0744/meta

    Article  Google Scholar 

  25. Luleci F, Catbas FN, Avci O (2023) Generative adversarial networks for labeled acceleration data augmentation for structural damage detection. J Civil Struct Health Monit 13(1):181–98. https://doi.org/10.1007/s13349-022-00627-8

    Article  Google Scholar 

  26. Kingma DP, Welling M (2013) Auto-encoding variational bayes, ar**v preprint ar**v:1312.6114. https://doi.org/10.48550/ar**v.1312.6114

  27. Ma X, Lin Y, Nie Z, Ma H (2020) Structural damage identification based on unsupervised feature-extraction via variational Auto-encoder. Measurement 160:107811. https://doi.org/10.1016/j.measurement.2020.107811

    Article  Google Scholar 

  28. Zhang Y, **e X, Li H, Zhou B (2022) An unsupervised tunnel damage identification method based on convolutional variational auto-encoder and wavelet packet analysis. Sensors 22(6):2412. https://doi.org/10.3390/s22062412

    Article  Google Scholar 

  29. Sajedi S, Liang X (2022) Deep generative Bayesian optimization for sensor placement in structural health monitoring. Comput-Aided Civil Infrastruct Eng 37(9):1109–1127. https://doi.org/10.1111/mice.12799

    Article  Google Scholar 

  30. F. Luleci, F.N. Catbas (2022) A Brief Introduction to Deep Generative Models for Civil Structural Health Monitoring, Civil Infrastructure Technologies for Resilience and Safety (CITRS). https://www.researchgate.net/publication/366422643_A_Brief_Introductory_Review_to_Deep_Generative_Models_for_Civil_Structural_Health_Monitoring . Retrieved 19 Dec 2022

  31. Bogert BP (1963) The quefrency alanysis of time series for echoes: Cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking. In Proc. Symposium Time Series Analysis. pp. 209–243. https://cir.nii.ac.jp/crid/1570854175999207936

  32. Balsamo L, Betti R, Beigi H (2014) A structural health monitoring strategy using cepstral features. J Sound Vib 333(19):4526–4542. https://doi.org/10.1016/j.jsv.2014.04.062

    Article  Google Scholar 

  33. Ioffe S (2006) Probabilistic linear discriminant analysis. In European Conference on Computer Vision, Springer, Berlin, Heidelberg. 531–542. https://doi.org/10.1007/11744085_41

  34. Theis L, Oord AV, Bethge M (2015) A note on the evaluation of generative models. ar**v preprint ar**v:1511.01844. https://doi.org/10.48550/ar**v.1511.01844

  35. Kingma DP, Welling M (2019) An introduction to variational autoencoders. Found Trends Mach Learn 12(4):307–92. https://doi.org/10.1561/2200000056

    Article  MATH  Google Scholar 

  36. Gulrajani I, Kumar K, Ahmed F, Taiga AA, Visin F, Vazquez D, et al. (2016) Pixelvae: a latent variable model for natural images. ar**v preprint ar**v:1611.05013. https://doi.org/10.48550/ar**v.1611.05013

  37. Villalba J, Brümmer N, Dehak N (2017) Tied Variational Autoencoder Backends for i-Vector Speaker Recognition. InInterspeech. 1004–1008. https://www.isca-speech.org/archive_v0/Interspeech_2017/pdfs/1018.PDF. Retrieved 20 Dec 2022

  38. Jordan MI (2004) Graphical models. Stat Sci 19(1):140–155. https://doi.org/10.1214/088342304000000026

    Article  MathSciNet  MATH  Google Scholar 

  39. Wu Z, Wang S, Qian Y, Yu K. Data Augmentation Using Variational Autoencoder for Embedding Based Speaker Verification. In INTERSPEECH. 1163–1167. https://www.isca-speech.org/archive_v0/Interspeech_2019/pdfs/2248.pdf. Retrieved 20 Dec 2022

  40. Sohn K, Lee H, Yan X (2015) Learning structured output representation using deep conditional generative models. Advances in neural information processing systems. 28. https://openreview.net/forum?id=rJWXGDWd-H. Retrieved 21 Dec 2022

  41. Kullback S (1997) Information theory and statistics. Courier Corporation

    MATH  Google Scholar 

  42. Doersch C (2016) Tutorial on variational autoencoders. ar**v preprint ar**v:1606.05908. https://doi.org/10.48550/ar**v.1606.05908

  43. Kingma DP, Mohamed S, Jimenez Rezende D, Welling M (2014) Semi-supervised learning with deep generative models. Advances in neural information processing systems. 27. https://ui.adsabs.harvard.edu/link_gateway/2014ar**v1406.5298K/arxiv:1406.5298

  44. An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE. 2(1):1–8. http://dm.snu.ac.kr/static/docs/TR/SNUDM-TR-2015-03.pdf

  45. Taud H, Mas JF (2018) Multilayer perceptron (MLP). Geomatic approaches for modeling land change scenarios. Springer, pp 451–455. https://doi.org/10.1007/978-3-319-60801-3_27.pdf

    Chapter  Google Scholar 

  46. Izenman AJ (2013) Linear discriminant analysis. Modern multivariate statistical techniques. New York, Springer, pp 237–280. https://doi.org/10.1007/978-0-387-78189-1_8.pdf

    Chapter  Google Scholar 

  47. Sizov A, Lee KA, Kinnunen T (2014) Unifying probabilistic linear discriminant analysis variants in biometric authentication. In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), Springer, Berlin, Heidelberg. 464–475. https://doi.org/10.1007/978-3-662-44415-3_47

  48. Rossi RJ (2018) Mathematical statistics: an introduction to likelihood based inference. Wiley, Cham

    Book  MATH  Google Scholar 

  49. Nandakumar K, Chen Y, Dass SC, Jain A (2007) Likelihood ratio-based biometric score fusion. IEEE Trans Pattern Anal Mach Intell 30(2):342–347. https://doi.org/10.1109/TPAMI.2007.70796

    Article  Google Scholar 

  50. Boger Z, Guterman H (1997) Knowledge extraction from artificial neural network models. In1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, IEEE. 4:3030–3035. https://doi.org/10.1109/ICSMC.1997.633051

  51. Zhai G, Narazaki Y, Wang S, Shajihan SA, Spencer BF Jr (2022) Synthetic data augmentation for pixel-wise steel fatigue crack identification using fully convolutional networks. Smart Struct Syst 29(1):237–50. https://doi.org/10.12989/sss.2022.29.1.237

    Article  Google Scholar 

  52. Jurafsky D (2000) Speech and language processing. Pearson Education India

  53. Krämer C, De Smet CA, De Roeck G (1999) Z24 bridge damage detection tests. In IMAC 17, the International Modal Analysis Conference, Society of Photo-optical Instrumentation Engineers 3727:1023–1029. https://lirias.kuleuven.be/1123428?limo=0

  54. Reynders E, De Roeck G Vibration-based damage identification: the Z24 benchmark. https://lirias.kuleuven.be/1725994?limo=0

  55. Giglioni V, Venanzi I, Baia AE, Poggioni V, Milani A, Ubertini F (2023) Deep autoencoders for unsupervised damage detection with application to the Z24 benchmark bridge. In: Yen T (ed) European workshop on structural health monitoring. Springer, Cham, pp 1048–1057. https://doi.org/10.1007/978-3-031-07258-1_105

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the Brite EuRam Programme BE-3157 SIMCES, the European Commission and the Structural Mechanics Section of KU Leuven for gathering and sharing the data of the Z24 bridge. We also would like to thank Ms. Sanaa Mouzahir for critically reading the final manuscript.

Author information

Authors and Affiliations

  1. Department of Civil Engineering and Engineering Mechanics, Columbia University, New York, NY, 10027, USA

    Lechen Li & Raimondo Betti

Authors
  1. Lechen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raimondo Betti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lechen Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Betti, R. A machine learning-based data augmentation strategy for structural damage classification in civil infrastructure system. J Civil Struct Health Monit 13, 1265–1285 (2023). https://doi.org/10.1007/s13349-023-00705-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13349-023-00705-5

Keywords

Search

Navigation