SpringerLink
Log in

Enabling cognitive and autonomous infrastructure in extreme events through computer vision

  • Technical paper
  • Published:
Innovative Infrastructure Solutions Aims and scope Submit manuscript

Abstract

As advent by the continuous inertia toward integrating artificial intelligence into daily operations, it is a matter of time before artificial intelligence reforms the field of structural engineering. From this point of view, this paper explores how computer vision and deep learning can be applied, in combination with advanced finite element analysis, to realize cognitive (self-diagnosing) and autonomous infrastructure. The outcome of this study demonstrates that computer vision not only can enable a structure to understand that it is undergoing an extreme event but can also allow it to trace its own performance and to independently respond to mitigate prominent failure/collapse. Findings of this work infer that computer vision can serve as an intelligent, and scalable agent to accurately trace structural response, identify different damage mechanisms and propose suitable repair strategies whether during or in the aftermath of a traumatic event (i.e., fire, earthquake). Finally, a series of challenges and future research directions are outlined toward the end of this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. For the sake of discussion carried out herein, Beam 2 is examined first without accounting for the contribution of CFRP attachments (i.e., transforming this beam into a traditional RC beam). Beam 2 is then re-examined (with CFRP attachments) toward the end of this section.

  2. For instance, Deepomatic suggests the use of large volumes of imagery (i.e., exceeding 1000 + images) in order to achieve high confidence CV models.

References

  1. McNichol E (2017) It’s time for states to invest in infrastructure, center on budget and policy priorities, Cent. Budg. Policy Priorities. https://www.cbpp.org/research/state-budget-and-tax/its-time-for-states-to-invest-in-infrastructure. Accessed 3rd Jan 2019

  2. Haigh R (2014) Enhancing resilience of critical road infrastructure: bridges, culverts and floodways. Int J Disaster Resil Built Environ. https://doi.org/10.1108/IJDRBE-05-2014-0038

    Article  Google Scholar 

  3. Kodur V, Yahyai M, Rezaeian A, Eslami M, Poormohamadi A (2017) Residual mechanical properties of high strength steel bolts subjected to heating-cooling cycle. J Constr Steel Res 131:122–131. https://doi.org/10.1016/J.JCSR.2017.01.007

    Article  Google Scholar 

  4. Abdalla JA, Abu-Obeidah AS, Hawileh RA, Rasheed HA (2016) Shear strengthening of reinforced concrete beams using externally-bonded aluminum alloy plates: an experimental study. Constr Build Mater 128:24–37. https://doi.org/10.1016/J.CONBUILDMAT.2016.10.071

    Article  Google Scholar 

  5. Barros JAO, Dias SJE, Baghi H, Ventura-Gouveia A (2016) New shear strengthening configurations of near-surface-mounted CFRP laminates for RC beams. ACI Struct J. https://doi.org/10.14359/51689029

    Article  Google Scholar 

  6. Mylrea M, Gourisetti NG (2017) Cybersecurity and optimization in smart “autonomous” buildings: a threat or Savior? Auton Artif Intell. https://doi.org/10.1007/978-3-319-59719-5_12

    Article  Google Scholar 

  7. Naser MZ (2019) Autonomous and resilient infrastructure with cognitive and self-deployable load-bearing structural components. Autom Constr 99:59–67. https://doi.org/10.1016/J.AUTCON.2018.11.032

    Article  Google Scholar 

  8. Murase M, Tsuji M, Takewaki I (2013) Smart passive control of buildings with higher redundancy and robustness using base-isolation and inter-connection. Earthq Struct. https://doi.org/10.12989/eas.2013.4.6.649

    Article  Google Scholar 

  9. Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, Murphy K (n.d.) Speed/accuracy trade-offs for modern convolutional object detectors. https://arxiv.org/pdf/1611.10012.pdf. Accessed 10 Jan 2019

  10. Shreyas SK, Dey A (2019) Application of soft computing techniques in tunnelling and underground excavations: state of the art and future prospects. Innov Infrastruct Solut. https://doi.org/10.1007/s41062-019-0234-z

    Article  Google Scholar 

  11. Shahri AA, Asheghi R (2018) Optimized developed artificial neural network-based models to predict the blast-induced ground vibration. Innov Infrastruct Solut. https://doi.org/10.1007/s41062-018-0137-4

    Article  Google Scholar 

  12. Moeinossadat SR, Ahangari K, Shahriar K (2018) Modeling maximum surface settlement due to EPBM tunneling by various soft computing techniques. Innov Infrastruct Solut. https://doi.org/10.1007/s41062-017-0114-3

    Article  Google Scholar 

  13. Kumar V, Kumar A (2019) Studying the behavior of neural models under hybrid and reinforced foundations. Innov Infrastruct Solut. https://doi.org/10.1007/s41062-019-0208-1

    Article  Google Scholar 

  14. Naser MZ (2019) Fire resistance evaluation through artificial intelligence—a case for timber structures. Fire Saf J 105:1–18. https://doi.org/10.1016/j.firesaf.2019.02.002

    Article  Google Scholar 

  15. Naser MZ (2019) AI-based cognitive framework for evaluating response of concrete structures in extreme conditions. Eng Appl Artif Intell 81:437–449. https://doi.org/10.1016/J.ENGAPPAI.2019.03.004

    Article  Google Scholar 

  16. Hasni H, Jiao P, Lajnef N, Alavi AH (2018) Damage localization and quantification in gusset plates: a battery-free sensing approach. Struct Control Health Monit. https://doi.org/10.1002/stc.2158

    Article  Google Scholar 

  17. Kim J, Zhou Y, Schiavon S, Raftery P, Brager G (2018) Personal comfort models: Predicting individuals’ thermal preference using occupant heating and cooling behavior and machine learning. Build Environ. https://doi.org/10.1016/j.buildenv.2017.12.011

    Article  Google Scholar 

  18. Shukla H, Piratla K (2020) Leakage detection in water pipelines using supervised classification of acceleration signals. Autom Constr. https://doi.org/10.1016/j.autcon.2020.103256

    Article  Google Scholar 

  19. Szeliski R (2010) Computer vision: algorithms and applications, Springer. https://books.google.com/books?hl=en&lr=&id=bXzAlkODwa8C&oi=fnd&pg=PR4&dq=computer+vision+engineering&ots=g_-890jDHE&sig=0PSy5CeZBkmM9aZBmKIhE3f7C40. Accessed 3 Jan 2019

  20. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst 3104–3112. http://papers.nips.cc/paper/5346-sequence-to-sequence-learnin. Accessed 16 Jan 2019

  21. Graves A, Mohamed A, Hinton G (2013) Speech recognition with deep recurrent neural networks. IEEE Int Conf Acoust Speech Signal Process 2013:6645–6649. https://doi.org/10.1109/ICASSP.2013.6638947

    Article  Google Scholar 

  22. Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. http://arxiv.org/abs/1605.06409. Accessed 10 Jan 2019

  23. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of IEEE Conf. Comput. Vis. Pattern Recognit. http://pjreddie.com/yolo9000/. Accessed 10 Jan 2019

  24. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/J.NEUNET.2014.09.003

    Article  Google Scholar 

  25. Luo H, **ong C, Fang W, Love PED, Zhang B, Ouyang X (2018) Convolutional neural networks: computer vision-based workforce activity assessment in construction. Autom Constr. https://doi.org/10.1016/j.autcon.2018.06.007

    Article  Google Scholar 

  26. Li D, Menassa CC, Kamat VR (2019) Feasibility of low-cost infrared thermal imaging to assess occupants’ thermal comfort. In: Comput. Civ. Eng. 2019 Smart Cities, Sustain. Resil. - Sel. Pap. from ASCE Int. Conf. Comput. Civ. Eng. 2019. https://doi.org/10.1061/9780784482445.008

  27. Sohn H, Farrar C, Hemez F, … DS-LAN (2003) A review of structural health monitoring literature: 1996–2001, Library.Lanl.Gov. (n.d.). https://library.lanl.gov/cgi-bin/getfile?00796820.pdf. Accessed 4 Jan 2019

  28. Simoen E, De Roeck G, Lombaert G (2015) Dealing with uncertainty in model updating for damage assessment: a review. Mech Syst Signal Process 56–57:123–149. https://doi.org/10.1016/J.YMSSP.2014.11.001

    Article  Google Scholar 

  29. **e Y, Sichani MEB, Padgett JE, DesRoches R (2020) The promise of implementing machine learning in earthquake engineering: a state-of-the-art review. Earthq Spectra. https://doi.org/10.1177/8755293020919419

    Article  Google Scholar 

  30. Naser MZ, Kodur VKR (2020) Concepts and applications for integrating unmanned aerial vehicles (UAV’s) in disaster management. Adv Comput Des 5(1):91–109

    Google Scholar 

  31. Naser MZ, Kodur VKR (2018) Cognitive infrastructure—a modern concept for resilient performance under extreme events. Autom Constr 90:253–264. https://doi.org/10.1016/j.autcon.2018.03.004

    Article  Google Scholar 

  32. Naser M (2016) Response of steel and composite beams subjected to combined shear and fire loading, Michigan State University. https://d.lib.msu.edu/etd/4107

  33. Hawileh RA, Abdalla JA, Tanarslan MH, Naser MZ (2011) Modeling of nonlinear cyclic response of shear-deficient RC T-beams strengthened with side bonded CFRP fabric strips. Comput Concr 8:193–206

    Article  Google Scholar 

  34. BSI (2004) European Committee for Standardization, design of concrete structures - part 1-2: general rules - structural fire design. https://doi.org/10.1002/jcp.25002

  35. ECS (2005) EN 1993-1-2: Eurocode 3: design of steel structures - part 1-2: general rules - structural fire design, European committee for standardisation, free download, borrow, and streaming, internet archive

  36. Rahnavard R, Thomas RJ (2018) Numerical evaluation of the effects of fire on steel connections; Part 1: simulation techniques. Case Stud Therm Eng. https://doi.org/10.1016/j.csite.2018.06.003

    Article  Google Scholar 

  37. Kodur VKR, Dwaikat MMS (2010) Effect of high temperature creep on the fire response of restrained steel beams. Mater Struct Constr. https://doi.org/10.1617/s11527-010-9583-y

    Article  Google Scholar 

  38. ASTM (2016) E119-16 - Standard test methods for fire tests of building construction and materials. Am Soc Test Mater. https://doi.org/10.1520/E0119-10B.1.2

  39. Naser MZ, Kodur VKR (2017) Comparative fire behavior of composite girders under flexural and shear loading. Thin-Walled Struct 116:82–90. https://doi.org/10.1016/j.tws.2017.03.003

    Article  Google Scholar 

  40. Hongnestad E, Hanson NW, Mchenry D (1955) Concrete stress distribution in ultimate strength design. J Am Concr Inst 57(2):875–928. https://doi.org/10.14359/8051

    Article  Google Scholar 

  41. Willam K, Warnke E (1975) Constitutive model for the triaxial behavior of concrete. Proc Int Assoc Bridg Struct Eng 19:1–30. https://doi.org/10.5169/seals-17526

    Article  Google Scholar 

  42. Xu XP, Needleman A (1994) Numerical simulations of fast crack growth in brittle solids. J Mech Phys Solids. https://doi.org/10.1016/0022-5096(94)90003-5

    Article  Google Scholar 

  43. Teng JG, Chen J, Smith S, Lam L (2002) FRP-strengthened RC structures, Wiley, 2002. https://www.wiley.com/en-us/FRP%3A+Strengthened+RC+Structures-p-9780471487067. Accessed 28 Nov 2018

  44. Zhang HY, Lv HR, Kodur V, Qi SL (2018) Performance comparison of fiber sheet strengthened RC beams bonded with geopolymer and epoxy resin under ambient and fire conditions. J Struct Fire Eng 9:174–188. https://doi.org/10.1108/JSFE-01-2017-0023

    Article  Google Scholar 

  45. Al-Mosawe A, Kalfat R, Al-Mahaidi R (2017) Strength of Cfrp-steel double strap joints under impact loads using genetic programming. Compos Struct. https://doi.org/10.1016/j.compstruct.2016.11.016

    Article  Google Scholar 

  46. Firmo JP, Correia JR, Bisby LA (2015) Fire behaviour of FRP-strengthened reinforced concrete structural elements: a state-of-the-art review. Compos Part B Eng 80:198–216. https://doi.org/10.1016/J.COMPOSITESB.2015.05.045

    Article  Google Scholar 

  47. Moreu F, Li X, Li S, Zhang D (2018) Technical specifications of structural health monitoring for highway bridges: new chinese structural health monitoring code. Front Built Environ 4:10. https://doi.org/10.3389/fbuil.2018.00010

    Article  Google Scholar 

  48. Chang PC, Flatau A, Liu SC (2003) Review paper: health monitoring of civil infrastructure. Struct Heal Monit Int J 2:257–267. https://doi.org/10.1177/1475921703036169

    Article  Google Scholar 

  49. Delgado J, Oyedele L, Ajayi A, Akanbi L, Owolabi L (2019) Robotics and automated systems in construction: understanding industry-specific challenges for adoption. J Build Eng 26:100868

    Article  Google Scholar 

  50. Dong CZ, Catbas FN (2019) A non-target structural displacement measurement method using advanced feature matching strategy. Adv Struct Eng. https://doi.org/10.1177/1369433219856171

    Article  Google Scholar 

  51. Koch C, Paal S, Rashidi A, Zhu Z, König M, Brilakis I (2014) Achievements and challenges in machine vision-based inspection of large concrete structures. Adv Struct Eng. https://doi.org/10.1260/1369-4332.17.3.303

    Article  Google Scholar 

  52. Gao X, Jian M, Hu M, Tanniru M, Li S (2019) Faster multi-defect detection system in shield tunnel using combination of FCN and faster RCNN. Adv Struct Eng. https://doi.org/10.1177/1369433219849829

    Article  Google Scholar 

Download references

Acknowledgements

The author would like to thank the support and constructive comments of the Editor and Reviewers.

Author information

Authors and Affiliations

  1. Glenn Department of Civil Engineering, Clemson University, Clemson, SC, 29634, USA

    M. Z. Naser

Authors
  1. M. Z. Naser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Z. Naser.

Ethics declarations

Conflict of interest

The author does not have any conflict of interest.

Ethical approval

This article does not contain any studies with human participants.

Informed consent

For this type of study formal consent is not required.

Appendix

Appendix

Figure 11 can be used to benchmark structural and construction engineering-based CV software (for a composite steel beam loaded in shear and subjected to standard fire conditions).

Fig. 11
figure 11figure 11figure 11figure 11figure 11figure 11figure 11figure 11

Image collected to be used in benchmarking VC model for tracing temperature-induced instability (Time is displayed in seconds)—minor buckling occurs between 15 and 35 min, major buckling occurs between 35 and 65 min, and failure through buckling occurs between 65 and 70 min

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naser, M.Z. Enabling cognitive and autonomous infrastructure in extreme events through computer vision. Innov. Infrastruct. Solut. 5, 99 (2020). https://doi.org/10.1007/s41062-020-00351-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41062-020-00351-6

Keywords

Search

Navigation