Introduction

Three-dimensional capture of an objects shape and appearance has many applied and theorized uses, and current technology has made the acquisition of such data more approachable for both experts and novices than what it used to be. By using different non-contact techniques one is able to capture the coordinates of different parts of an object in a 3D space, which can be used to visualize the object in several different ways. This approach is applicable for any object size as long as one is able to collect images of good quality or maintain line-of-sight with the object during acquisition, and is extensively researched and applied to the medical field [1], construction [2], and indeed cultural heritage (CH). This review creates an overview and critical analysis of the current application and implementation of 3D data to CH objects of any kind, independent of purpose or approach.

Collection of 3D data can be done by a variety of methods using a variety of tools, and has been extensively explored and described in prior reviews [Footnote 5 The Louvre Museum,Footnote 6 and The British MuseumFootnote 7are some examples.

The virtual museum is perhaps the application which represents the elements from the traditional heritage conservation perspective and computer science perspective most equally, and museum researcher Suzanne Keene captures an important aspect of moving towards CH in the digital space with the quote: “We used to build collections of objects. Now we can make collections of information, too” [8]. While her comment regards several things, including metadata and semantics, 3D objects might be appended with additional information as well. Cross sections from x-ray data, annotations, and different textures are a few. But, it is important to remember that 3D objects are not tangible objects in themselves, rather visualization of information about tangible objects. The visualizations of CH in virtual museums are only based on the information we are able to acquire in an acquisition process. Careful ethical considerations must then be made on how to visualize and communicate this to an audience, without introducing conjecture or misinterpretation in the presentation. This is perhaps especially true in the museum setting, but has equal relevance for any application of reality-based 3D methods. Heritage objects curated by museum organizations follows standardization guidelines provided by organizations like ICOM [9] and ICOMOS [10], and while work is being done on implementing 3D in virtual museums in a standardized way [11,12,13], there is still a lot of gaps in knowledge and ethically justifiable workflows. As such, the virtual museum is an example of the general digitization process of the CH field, but the contemporary and state-of-the-art projects and implementations reviewed here often take several steps away from the traditional museum setting.

Application of 3D data of CH objects is only partially covered by museums, either physical or virtual. Standalone projects, non-profit organizations, research institutions, production companies, and private persons contribute a significant portion of the 3D data that is available online. Furthermore, the data of this varied field is being used for many other applications than just digital viewing of the object, even though this is an obvious, popular, and easily implemented utilization. As this varied ensemble might not be bound to some institution or larger organization, they might not adhere to any sort of museum or heritage standards or regulations in the case of digitizing CH, and are therefore free to do “what they want” in order to get the best results. What can be classified as the “best result” again depends on the application, and literature and projects often lists the many platforms 3D data can be applied to.

Some recurring proposed and tested applications for 3D models of CH objects are: visualization and dissemination [14,15,16], simulation [17, 18] education and training [19,20,21], and research [22,23,24]. Additionally, audience interaction is a factor explored in implementations that feature gamification principles. At a glance, these are very broad suggestive implementations that are very different in their context and content. Papers and projects that utilize 3D for CH often phrases that it might lead to new knowledge and insights [16], but this application is in most cases still in a very early phase. New developments for utilization of 3D also gives virtual reality (VR), augmented reality (AR), and mixed reality (MR) increased uses and applications in many fields, but similarly lacks designated guidelines for ethical implementations and requirements in terms of quality and semantics. Especially with CH objects that are sensitive to conjecture.

Currently there is a lot of work focused on making the application of 3D data for CH more concrete; researching and develo** both standards and workflows that should help institutions apply 3D to their own collection in a more uniform way. The prior mentioned reports and research papers agree that there is still a lack of knowledge for CH institutions regarding 3D implementation and processing, and that this is required to make the full utilization of 3D for CH a reality.

There is on the other hand no shortage of research conducted on the use of 3D in the CH sector. New applications that are being investigated includes conservation [25], change monitoring [26, 27], visualization [28], additive manufacturing [29, 30], BIM (Building Information Modeling), also known as HBIM (Heritage Building Information Modeling) [31, 32], and dissemination methods [33] just to mention a few. Recent investments and grants like CHANGE,Footnote 8 the n-Dame Heritage ERC project,Footnote 9 Data Service for Complex 3D Data in the Arts and Humanities,Footnote 10 JPI CH,Footnote 11 and PerceiveFootnote 12 also signifies that this field will continue to grow in the future.

This paper reviews results from contemporary and state-of-the-art projects, and how they make use of 3D for CH purposes to see if the initial imagined potential and value of 3D digitization has been met, surpassed, or limited. Additionally, we take a close look on the various frameworks in which 3D data is presented, and if interoperability might be a concern between projects. Longevity and use of the end results is also something that is scrutinized, in an attempt to evaluate the impact that these projects might have had based on the results they present. The reviewed projects, institutions, guidelines, and tools have been selected based on their recurring appearance in academic papers as well as their visibility when browsing for the subject online. We deem that since these projects are the most visible, they might also be the most influential for new projects in the future.

Section "Prior reviews and existing projects" presents various prior reviews on 3D CH data, and takes a close look at some of the most relevant selected projects. Workflow proposals for data acquisition and processing is presented and scrutinized in Sect. "Workflows for acquisition and processing of 3D data", and Sect. "Heterogeneous data and interoperability issues" presents some apparent issues with utilizing 3D CH data. In this section we also highlight a few options for solutions that are not as visible as the reviewed projects. A discussion on the presented data is done in Sect. "Discussion", before summarizing and providing conclusions in Sect. "Conclusion".

Prior reviews and existing projects

Previous reviews of applications of 3D data in the CH sector has mostly been from specific approaches, for example acquisition methods [71, 72] in relation to the 3D topic covered here. An example of effects from these factors can be seen in Fig. 4.

Fig. 4
figure 4

Left: No visible UV seams or pixelization. Middle: Visible UV seams and pixelization when zooming. Right: UV segmentation. Model: “Nile” (https://skfb.ly/6TDJI) by Rigsters

Full size image

These are just a few of the issues which projects utilizing 3D would have to make decisions about, and documentation of their selected approach and workflow is essential for an evaluation of the end result. But while the documentation of these technical specifications and approaches are integral to a 3D project, in many cases they also drown out the question of why the digitization was conducted in the first place. This observation was first reported by Pfarr-Harfst in 2016[48], and while there have been improvements to contextualizing the data in contemporary research, many projects still suffers from the same issue.

In many projects the objective is summarized as the paraphrase: “the 3D documentation of CH to provide open access for education, research, and audiences”. While noble, this is very open ended and could potentially have limited use if the aspects of the data is not of a high enough quality for a certain application. The projects and workflows reviewed here may in some cases produce the data, but not the tools or platform for which they created them. Figure 5 highlights the importance of quality control for the 3D models of CH, as some published data has many apparent faults in their accuracy.

Fig. 5
figure 5

Left: Image of Lamassu from the British Museum Right: Model of the same Lamassu by CyArk

Full size image

In her review, Pfarr-Harfst proposed a documentation practice that highlights the ’prior’, the ’during’, and ’subsequent’ situation of both the heritage object and the project data[48]. This might be an important step the field should take to be more academically recognized, as we would be able to more clearly move away from using CH as an object for computer graphics visualization and towards using computer science as a tool for CH preservation.

Current reviews for 3D implementation in CH highlight the necessity to quantify the accuracy of the 3D data in an objective, homogeneous, and semantic way, while suggested workflows provide subjective and indeterminate tools to do so. This divide is a cause for concern for the merit of 3D CH data, and future research should exert itself to contribute to this gap in information. But the apparent and necessary variation in approaches, along with the great variation of the objects themselves, signifies that a single, standardized workflow for 3D in CH might be an impractical approach.

Heterogeneous data and interoperability issues

Another possible reason for the data clutter in the 3D CH field is precisely this variation of objectives with limited specifying documentation. This ties back to the lack of long-term support and application of the collected data, and overemphasis on acquisition methods relative to research questions. What we have ended up with is heterogeneous data that might have limited interoperability and little contribution to other areas than computer graphics visualization, which is not unique to the CH field. There is a significant semantic difference between using 3D for visualization and for the research of the tangible objects, as one emphasizes observer perception and the other measures quantitative parameters. Various approaches will also weigh different parameters of an acquisition process in a unique way, perhaps leading to specialized data that is not easily used for other applications. While it may not be an objective for some research approaches to make their data universally applicable, extensive restrictions on the workflow and post-processing might render the data or research results to not be reproducible in other environments. If this ends up being the case, the legitimacy of the methodology might be jeopardized, as it might only be valid under very specific conditions.

Digital projects for CH that utilize 3D data will always be multimodal, and weighs different data types based on the project objective. It is an intrinsic part of any 3D workflow that the data undergoes a lot of change and travel through different software. Even projects that exclusively want to capture the geometry of objects, disregarding color and texture, are still dependent on temporary data like 2D images or point clouds, meaning that there is no clear divide between 2D and 3D workflows. Maintaining the different modalities, even if it might have no practical use to the current project might lessen this heterogeneity issue. This ties back to Pfarr-Harfst’s notion of documenting the prior, the current, and the subsequent in 3D digitization processes, and different data formats validating this could provide sufficient academic evidence of the results of a 3D processing stage for CH.

For example, while research have been conducted on reconstructing the missing shape of CH objects using Poisson reconstruction [28] and shape recognition[73], there is no way of validating how accurate this is in reality. And while this is the case, there might be little difference between using this method and modeling by hand, apart from ethical or subjective considerations. A model that has been processed in such a way would also be unsuitable for suggested applications like change monitoring, where the introduced conjecture already renders the 3D object ethically improper for ground truth comparisons. Other application workflows, like 3D printing, uses a file format which reads the data in way where a watertight 3D model is essential. In which case a hole-filling process is inevitable. ScanTheWorldFootnote 70 is a repository designed for sharing 3D data of cultural artifacts for the purpose of 3D printing, and therefore does not support objects or formats which are unsuitable for this task. Other applications might not have this demand, and while a 3D object with no holes might be more visually appealing to look at, introducing algorithms to fill these holes will make conjecture unavoidable.

The issue with the current proposed standard parameters and workflows referenced in this review is that they are not quantifiable to the degree of being objective. As such, different researchers, producers, and curators will weigh them differently based on their needs, and a unified and reproducible metric for all forms of 3D data is unachievable.

Standardized formats and their use

Specialized production also often results in data formats that supports the primary objective of each specific data acquisition process, possibly limiting the interoperability or applied use of the approach for other means. A few papers and projects investigate the creation of evaluation datasets, but this approach has yet to see too much development in the field. Using such tools, different approaches could tested on the same object which might give a better baseline evaluation of an applied methodology or processing technique.

Tools like the H3D dataset from [74] released in 2021 is an example of how it might be useful for researchers and developers to test out their new processing workflows, and quantitatively compare them to others who have used the same dataset. H3D is a UAV LiDAR dataset depicting the town of Hessigheim, Germany in several epochs, and terrestrial data acquired by UAV LiDAR are often used for researching heritage sites and buildings[75,76,77]. The work has 32 citations at the time of writing, and note that this is not a collectively approved standard. Other examples of such datasets include CO3D [78] from Meta, HM3D [Footnote 100 and the Open 3D FoundationFootnote 101 consist of large actors within the 3D field that work towards open-source developments.

Discussion

Through this review, we have found that several of the original theorized applications for 3D in CH have been explored in various ways, as different projects implements 3D for CH for means of education, dissemination, and simulation. Visualization seems to still be the primary result of many of these projects, either for the purpose of visualizing to an audience or exploring different data acquisition methodologies. But in many ways the various projects remain fragmented and isolated from each other. Ad hoc solutions for implementation mirrors the ad hoc acquisition workflows of data collectors, making it almost impossible to quantitatively compare two similar implementations by the same parameters. Evaluation of 3D project yields is therefore still very subjective in the CH field, as several root issues are yet to be tackled. In this review we have explored the most prominent and recurring issues with data acquisition, data storage, file formats, and standardization along with 3D object quality assessment, workflow variation, data actualisation, and limited research focus within the field of 3D in cultural heritage. It is clear that while there exist many great tools to aid and develop this process, there are still several shortcomings that are integral. Numeric evaluation tools on objective variables and statistics should be a priority for the field in the coming years, so that the variability of 3D could at least be quantified in the most recurring dimensions. Variables that we deem the most important for this are:

  1. 1.

    Geometric accuracy and its alterations and reductions in a 3D process.

  2. 2.

    3D resolution levels utilized to digitize certain objects and surfaces, with distance measurements of whats captured within the resolution.

  3. 3.

    Processing power and computer memory required to utilize 3D objects.

  4. 4.

    Characterization of color and surface acquisition, and texture projection protocols.

Even though 3D can be used for a great variety of applications, and as a result will look significantly different within such measurements, it would be a tool for categorization and evaluation of 3D objects depending on the most important variables within its specific application. Subcategories would be created, that would narrow down the idea of 3D objects and what they include depending on the utilization. This could potentially lead to better standardization practices within each subcategory, and avoid the issue of attempting to develop a one-tool-fits-all standard.

While institutions would perhaps be more inclined to subscribe to agreed-upon standards, private enthusiasts of the general public might adhere to no such regulations. Public production is only set to increase in the future if the current trend continues, so it is vital for the research-field on 3D CH to separate 3D objects that artistically represents CH from research based 3D that attempts to visualize, archive, and analyze the truthful presentation of tangible CH objects. Development of standards would have to tackle a lot of different issues, as the heterogeneous and alterable nature of both CH and computer science will stretch and strain regulations in many different directions. First drafts would, and arguably should, therefore not encompass all variations, but attempt to establish some ground-rules based on the most recurring characteristics. The standardization is nonetheless vital for the research on the field, to be able to approach the data from a common scientific and quantifiable way. It would be the means to separate the generic visualization of computer graphics from the quantifiable research data of computer science, and elevate appearance acquisition and analysis for CH to a more concrete field. Variables of 3D implementations that are not part of an objects numerical evaluation that we suggest to pay more attention to are:

  1. 1.

    Extent of human intervention in the digitization process, compared to purely algorithmic.

  2. 2.

    Lifespan of the 3D object, both in terms of utilization and quality compared to current state-of-the-art.

  3. 3.

    Subsequent use of the 3D assets after initial acquisition and visualization.

  4. 4.

    Semantic and objective descriptions of the 3D asset and its use, instead of generic and open-ended.

Even with these suggestions, the field in relevance currently still very much depends on a lot of different actors from different backgrounds. Be it for interdisciplinary research or not. Researchers in this field must be aware of the interdisciplinary, commercial, public, and non-profit developments being made, as the public interactions with the research results is one of the primary objectives of 3D CH projects. As such, it would be beneficial to find some way to develop a workflow that does not narrow the 3D data acquisition to a specific research question or creative application, thereby making the data more universally relevant. Specialized training and education would allow for more nuanced and knowledgeable approaches, but the field is still too fragmented for such aspirations to emerge by themselves. A research gap for methods of validation and quality control of 3D objects is still prominent, especially in the CH sector where the conservation of object appearance, both shape and surface, is the main aspiration. If this is not considered by future projects, the heterogeneity of the field is only set to increase.

Europeana’s controlled and organized approach to 3D data storage shows promise for a more officially-recognized standard, and similar institutions provide valuable input that gradually provides more guidelines for 3D data collectors. But still there is lacking some implementation to verify that their hosted objects are of a high quality. Suggestions of peer-reviews of uploaded 3D data is a very interesting notion that should be explored further, and attempt to develop a framework from which 3D models could be evaluated. Other quantitative approaches could also be made, like relating the tangible object’s size or geometric variation to resolution requirements. We already have great quantities of 3D CH data available on the web, collected using various acquisition paradigms. Attempting to extract what research data we can from these pre-existing models would show us where they excel, where they fall short, and what characteristics are lacking for various research applications. Another project that is promising is the development of The European Collaborative Cloud for CH, which released its stakeholder survey in December 2022 [93]. This survey repeats a lot of what is noted in prior reports, and we hope that the development of this platform will take the shortcomings highlighted in this review into consideration.

One more segment that is seeing more development is the 3D viewing platforms. For a long time Sketchfab has reigned supreme as the 3D viewer of choice on the web, and while it offers some possibilities for 3D model inspection and provides a good API, we argue that it should not be deemed fit for hosting research-oriented 3D CH models apart from secondary-objective visualization. Some projects have opted for other 3D viewers that fits their format of 3D data, like PoTree, which comes with their own limitations or restrictions. General shortcomings of 3D viewers seems to be universal tools and format support, as well as object quality validation and embedding of metadata. While simpler tools like annotation is often listed as a feature of high priority, and is indeed implemented in most 3D viewers, the field should move towards using 3D viewers that opens more for processing of the dataset directly. Open Source projects have shown to provide good solutions for hosting 3D research data, and the transparency in development and code integration should be prioritized over proprietary, black-box viewers.

Challenges and opportunities are apparent in every stage of a 3D project, from project planning to data implementation, and CH provide various challenges that strains creative workflows. Substantial work is being done in improving many of these stages, but we have highlighted a few that is key to elevating the research-field.

Conclusion

This review looks at different projects working with 3D CH data, including data collectors, data repositories, suggestions for standards and workflows, as well as viewing platforms and processing engines. We have highlighted some of the important developments in each segment, and proposed directions to where research should head into the future. There is no shortcoming of work being done, but we hope to see a bit more in depth application research in the future, as well as emphasis on research questions for 3D CH that is subsequent for acquisition.

The research field is an interdisciplinary field, and as such includes a great variation of competent institutions that could to contribute to standardization agreements, research, and development activities. However, in many cases the approach each institution selects is incompatible with the approach of another institution, limiting the interoperability of the implementation and reduces the possibility for sharing data directly. Current work shows great promise in providing solutions for these issues, but there is a lot of work still left to do.