Abstract
The AI4TWINNING project aims at the automated generation of a system of inter-related digital twins of the built environment spanning multiple resolution scales providing rich semantics and coherent geometry. To this end, an interdisciplinary group of researchers develops a multi-scale, multi-sensor, multi-method approach combining terrestrial, airborne, and spaceborne acquisition, different sensor types (visible, thermal, LiDAR, Radar) and different processing methods integrating top-down and bottom-up AI approaches. The key concept of the project lies in intelligently fusing the data from different sources by AI-based methods, thus closing information gaps and increasing completeness, accuracy and reliance of the resulting digital twins. To facilitate the process and improve the results, the project makes extensive use of informed machine learning by exploiting explicit knowledge on the design and construction of built facilities. The final goal of the project is not to create a single monolithic digital twin, but instead a system of interlinked twins across different scales, providing the opportunity to seamlessly blend city, district and building models while kee** them up-to-date and consistent. As testbed and demonstration scenario serves a urban zone around the city campus of TUM, for which large data sets from various sensors are available.
This article was selected based on the results of a double-blind review of an extended abstract.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
BBSR (2017) Nutzungsmischung und die Bewältigung von Nutzungskonflikten in Innenstädten, Stadt- und Ortsteilzentren - Chancen und Hemmnisse. (https://www.bbsr.bund.de/BBSR/DE/veroeffentlichungen/bbsr-online/2017/bbsr-online-23-2017-dl.pdf), Accessed 26 Jul 2023
Berger R, Berger E (1999) Bauwerke betrachten, erfassen, beurteilen: Wege zum Verständnis klassicher und moderner Architektur
Bleek J (2022) Fassade und Ornament. (Brill Fink,2022,7)
Chen Z, Ledoux H, Khademi S, Nan L (2022) Reconstructing compact building models from point clouds using deep implicit fields. ISPRS J Photogram Remote Sens 194:58–73
Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation
Chen Z, Shi Y, Nan L, **ong Z, Zhu XX (2023) PolyGNN: polyhedron-based graph neural network for 3D building reconstruction from point clouds. ar**v preprint ar**v:2307.08636
Chua F, Duffy N (2021) DeepCPCFG: deep learning and context free grammars for end-to-end information extraction. Doc Anal Recogn—ICDAR 2021, 838–853
Cremers J (2015) Die gestalterische Wirkung von Öffnungen in der Fassade. Atlas Gebäudeöffnungen, 24–31
Garau N, Bisagno N, Sambugaro Z, Conci N (2022) Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13689–13698
Heeramaglore M, Kolbe T (2022) Semantically enriched voxels as a common representation for comparison and evaluation of 3D building models. ISPRS Annals Photogram Remote Sens Spatial Inf Sci X-4/W2-2022. 89-96. 10.5194/isprs-annals-X-4-W2-2022-89-2022
Herzog T, Krippner R, Lang W (2016) Fassaden atlas. (DETAIL,2016,12)
Hoegner L, Stilla U (2018) Mobile thermal map** for matching of infrared images with 3D building models and 3D point clouds. Quant Infrared Thermography J 15(2):252–270
Kemp W (2006) Architektur analysieren: Katalog Haus der Kunst München. (Schirmer Mosel)
Krijnen T, El-Diraby T, Konomi T, Attalla A (2021) Thermal analysis of IFC building models using voxelized geometries. In: Proceedings of the 38th international conference of CIB W78. (CIB W78 conference series), pp 437–446. https://itc.scix.net/paper/w78-2021-paper-044
Li W, Zlatanova S, Gorte B (2020) Voxel data management and analysis in PostgreSQL/PostGIS under different data layouts. ISPRS Ann Photogram Remote Sens Spatial Inf Sci VI-3/W1-2020, 35–42
Mehranfar M, Braun A, Borrmann A (2022) A hybrid top-down, bottom-up approach for 3D space parsing using dense RGB point clouds. Proceedings of European conference on product and process modeling
Mehranfar M, Braun A, Borrmann A (2023) Automatic creation of digital building twins with rich semantics from dense RGB point clouds through semantic segmentation and model fitting. In EG-ICE 2023; 30th international conference on intelligent computing in engineering
Mitchell W (1998) The logic of architecture: design, computation, and cognition. MIT Press
Nguyen S, Yao Z, Kolbe T (2017) Spatio-semantic comparison of large 3D city models in CityGML using a graph database. In: Proceedings of the 12th international 3D GeoInfo conference 2017 (ISPRS annals of the photogrammetry, remote sensing and spatial information sciences), ISPRS, 99–106
Nourian P, Gonçalves R, Zlatanova S, Ohori KA, Vu Vo A (2016) Voxelization algorithms for geospatial applications: computational methods for voxelating spatial datasets of 3D city models containing 3D surface, curve and point data models. MethodsX 3:69–86
Poux F, Billen R (2019) Voxel-based 3D point cloud semantic segmentation: unsupervised geometric and relationship featuring vs deep learning methods. ISPRS Int J Geo-Inf 8(5)
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR)
Robert H (2009) Wege zur Kunst: begriffe und Methoden für den Umgang mit Architektur. Schroedel
Socher R, Lin C, Ng A, Manning C (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on international conference on machine learning, pp 129–136
Stadler A, Kolbe TH (2007) Spatio-semantic coherence in the integration of 3D city models. In: Proceedings of the 5th international ISPRS symposium on spatial data quality ISSDQ 2007 in enschede, The Netherlands, 13–15 June 2007 (ISPRS Archives), ISPRS
Sun Y, Mou L, Wang Y, Montazeri S, Zhu XX (2022) Large-scale building height retrieval from single SAR imagery based on bounding box regression networks. ISPRS J Photogram Remote Sens 184:79–95
Wang Q, Zuo W, Guo Z, Li Q, Mei T, Qiao S (2020) BIM voxelization method supporting cell-based creation of a path-planning environment. J Construct Eng Manage 146:04020080
**e S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)
Zhu J, Xu Y, Ye Z, Hoegner L, Stilla U (2021) Fusion of urban 3D point clouds with thermal attributes using MLS data and TIR image sequences. Infrared Phys Technol 113:103622
Acknowledgements
The AI4TWINNING project is funded by the TUM Georg Nemetschek Institute for Artificial Intelligence for the Built World, which is thankfully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Borrmann, A. et al. (2024). Artificial Intelligence for the Automated Creation of Multi-scale Digital Twins of the Built World—AI4TWINNING. In: Kolbe, T.H., Donaubauer, A., Beil, C. (eds) Recent Advances in 3D Geoinformation Science. 3DGeoInfo 2023. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-031-43699-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-43699-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43698-7
Online ISBN: 978-3-031-43699-4
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)