Abstract
Digital metric documentation of historical city centers is challenging because of the complexity of the buildings and monuments, which feature different geometries, construction technologies, and materials. We propose a solution for rapid documentation and classification of such complex spaces using 360° video cameras, which can capture the entire scene and can be pointed in any direction, making data acquisition rapid and straightforward. The high framerate during image acquisition allows users to capture overlap** images that can be used for photogrammetric applications. This paper aims to quickly capture 360° videos with low-cost cameras and then generate dense point clouds using the photogrammetric/structure from motion pipeline for 3D modeling. Point cloud classification is the prerequisite for such applications. Numerous deep learning methods (DL) have been developed to classify point clouds due to the expansion of artificial intelligence (AI) capabilities. We aim to pave the way toward utilizing the convolutional neural network (CNN) to classify point clouds generated by 360° videos of historic cities. A preliminary case study in a historic city center demonstrates that our method achieves promising results in the generation and classification of point clouds, with an overall classification accuracy of 96% using the following categories: ground, buildings, poles, bollards, cars, and natural.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barazzetti, L., Previtali, M., Roncoroni, F.: 3D modeling with 5K 360° videos. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. XLVI-2/W1-2022, 65–71. https://doi.org/10.5194/isprs-archives-XLVI-2-W1-2022-65-2022
Sánchez-Aparicio, L.J., Del Pozo, S., Ramos, L.F., Arce, A., Fernandes, F.: Heritage site preservation with combined radiometric and geometric analysis of TLS data. Autom. Constr. 85, 24–39 (2018)
Weinmann, M., Jutzi, B., Mallet, C., Weinmann, M.: Geometric features and their relevance for 3D point cloud classification. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. IV-1/W1, 157–164 (2017)
Grilli, E., Farella, E.M., Torresani, A., Remondino, F.: Geometric features analysis for the classification of cultural heritage point clouds. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. XLII-2/W15, 541–548 (2019)
Bassier, M., Van Genechten, B., Vergauwen, M.: Classification of sensor independent point cloud data of building objects using random forests. J. Build. Eng. 21, 468–477 (2019)
Babahajiani, P., Fan, L., Kamarainen, J., Gabbouj, M.: Automated super-voxel based features classification of urban environments by integrating 3D point cloud and image content. In: 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 372–377 (2015)
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 605–613 (2017)
Riegler, G., Osman Ulusoy, A., Geiger, A.: OctNet: learning deep 3D representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3577–3586 (2017)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: KPConv: flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6411–6420 (2019)
Cao, Y., Scaioni, M.: 3DLEB-Net: label-efficient deep learning-based semantic segmentation of building point clouds at LoD3 level. Appl. Sci. 11(19), 8996 (2021)
Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K., Pollefeys, M.: Semantic3D.net:: a new large scale point cloud classification benchmark. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. IV-1-W1, pp. 91–98 (2017)
Roynard, X., Deschaud, J.E., Goulette, F.: Paris-Lille-3D: a large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification. Int. J. Robot. Res. 37(6), 545–557 (2018)
Fangi, G.: Multiscale multiresolution spherical photogrammetry with long focal lenses for architectural surveys. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 38(Part 5), 1–6 (2010)
Fangi, G.: The Multi-image spherical panoramas as a tool for architectural survey. CIPA Herit. Doc. 21, 311–316 (2011)
Fangi, G.: The Book of Spherical Photogrammetry: Theory and Experiences. Edizioni Accademiche Italiane (2017). 300 pages
Boulch, A.: ConvPoint: continuous convolutions for point cloud processing. Comput. Graph. 88, 24–34 (2020)
Hu, Q., et al.: RandLA-Net: efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11108–11117 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, Y., Previtali, M., Barazzetti, L., Scaioni, M. (2022). Integration of Point Clouds from 360° Videos and Deep Learning Techniques for Rapid Documentation and Classification in Historical City Centers. In: Mazzeo, P.L., Frontoni, E., Sclaroff, S., Distante, C. (eds) Image Analysis and Processing. ICIAP 2022 Workshops. ICIAP 2022. Lecture Notes in Computer Science, vol 13374. Springer, Cham. https://doi.org/10.1007/978-3-031-13324-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-13324-4_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13323-7
Online ISBN: 978-3-031-13324-4
eBook Packages: Computer ScienceComputer Science (R0)