Visualization of Cultural Heritage Collection Data: State of the Art and Future Challenges

CataAnno: An Ancient Catalog Annotator for Annotation Cleaning by Recommendation

Classical bibliography, by researching preserved catalogs from both official archives and personal collections of accumulated books, examines the books throughout history, thereby revealing cultural development across historical periods. In this work, we collaborate with domain experts to accomplish the task of data annotation concerning Chinese ancient catalogs. We introduce the CataAnno system that facilitates users in completing annotations more efficiently through cross-linked views, recommendation methods and convenient annotation interactions. The recommendation method can learn the background knowledge and annotation patterns that experts subconsciously integrate into the data during prior annotation processes. CataAnno searches for the most relevant examples previously annotated and recommends to the user. Meanwhile, the cross-linked views assist users in comprehending the correlations between entries and offer explanations for these recommendations. Evaluation and expert feedback confirm that the CataAnno system, by offering high-quality recommendations and visualizing the relationships between entries, can mitigate the necessity for specialized knowledge during the annotation process. This results in enhanced accuracy and consistency in annotations, thereby enhancing the overall efficiency.

From Invisible to Visible: Impacts of Metadata in Communicative Data Visualization

Leaving the context of visualizations invisible can have negative impacts on understanding and transparency. While common wisdom suggests that recontextualizing visualizations with metadata (e.g., disclosing the data source or instructions for decoding the visualizations' encoding) may counter these effects, the impact remains largely unknown. To fill this gap, we conducted two experiments. In Experiment 1, we explored how chart type, topic, and user goal impacted which categories of metadata participants deemed most relevant. We presented 64 participants with four real-world visualizations. For each visualization, participants were given four goals and selected the type of metadata they most wanted from a set of 18 types. Our results indicated that participants were most interested in metadata which explained the visualization's encoding for goals related to understanding and metadata about the source of the data for assessing trustworthiness. In Experiment 2, we explored how these two types of metadata impact transparency, trustworthiness and persuasiveness, information relevance, and understanding. We asked 144 participants to explain the main message of two pairs of visualizations (one with metadata and one without); rate them on scales of transparency and relevance; and then predict the likelihood that they were selected for a presentation to policymakers. Our results suggested that visualizations with metadata were perceived as more thorough than those without metadata, but similarly relevant, accurate, clear, and complete. Additionally, we found that metadata did not impact the accuracy of the information extracted from visualizations, but may have influenced which information participants remembered as important or interesting.

Communicating Uncertainty in Digital Humanities Visualization Research

Due to their historical nature, humanistic data encompass multiple sources of uncertainty. While humanists are accustomed to handling such uncertainty with their established methods, they are cautious of visualizations that appear overly objective and fail to communicate this uncertainty. To design more trustworthy visualizations for humanistic research, therefore, a deeper understanding of its relation to uncertainty is needed. We systematically reviewed 126 publications from digital humanities literature that use visualization as part of their research process, and examined how uncertainty was handled and represented in their visualizations. Crossing these dimensions with the visualization type and use, we identified that uncertainty originated from multiple steps in the research process from the source artifacts to their datafication. We also noted how besides known uncertainty coping strategies, such as excluding data and evaluating its effects, humanists also embraced uncertainty as a separate dimension important to retain. By mapping how the visualizations encoded uncertainty, we identified four approaches that varied in terms of explicitness and customization. This work contributes with two empirical taxonomies of uncertainty and it's corresponding coping strategies, as well as with the foundation of a research agenda for uncertainty visualization in the digital humanities. Our findings further the synergy among humanists and visualization researchers, and ultimately contribute to the development of more trustworthy, uncertainty-aware visualizations.

HisVA: A Visual Analytics System for Studying History

Studying history involves many difficult tasks. Examples include searching for proper data in a large event space, understanding stories of historical events by time and space, and finding relationships among events that may not be apparent. Instructors who extensively use well-organized and well-argued materials (e.g., textbooks and online resources) can lead students to a narrow perspective in understanding history and prevent spontaneous investigation of historical events, with the students asking their own questions. In this article, we proposed HisVA, a visual analytics system that allows the efficient exploration of historical events from Wikipedia using three views: event, map, and resource. HisVA provides an effective event exploration space, where users can investigate relationships among historical events by reviewing and linking them in terms of space and time. To evaluate our system, we present two usage scenarios, a user study with a qualitative analysis of user exploration strategies, and in-class deployment results.

Implicit Error, Uncertainty and Confidence in Visualization: An Archaeological Case Study

While we know that the visualization of quantifiable uncertainty impacts the confidence in insights, little is known about whether the same is true for uncertainty that originates from aspects so inherent to the data that they can only be accounted for qualitatively. Being embedded within an archaeological project, we realized how assessing such qualitative uncertainty is crucial in gaining a holistic and accurate understanding of regional spatio-temporal patterns of human settlements over millennia. We therefore investigated the impact of visualizing qualitative implicit errors on the sense-making process via a probe that deliberately represented three distinct implicit errors, i.e., differing collection methods, subjectivity of data interpretations and assumptions on temporal continuity. By analyzing the interactions of 14 archaeologists with different levels of domain expertise, we discovered that novices became more actively aware of typically overlooked data issues and domain experts became more confident of the visualization itself. We observed how participants quoted social factors to alleviate some uncertainty, while in order to minimize it they requested additional contextual breadth or depth of the data. While our visualization did not alleviate all uncertainty, we recognized how it sparked reflective meta-insights regarding methodological directions of the data. We believe our findings inform future visualizations on how to handle the complexity of implicit errors for a range of user typologies and for highly data-critical application domains such as the digital humanities.

New Media and Space: An Empirical Study of Learning and Enjoyment Through Museum Hybrid Space

A museum hybrid space combines physical artifacts co-located with virtual and augmented reality displays. Although the technology exists to provide museums with hybrid space, there are no empirical studies on effectiveness of the museum hybrid space in terms of learning and enjoyment. This article takes an experimental approach and measures the enjoyment and learning (dependent variables) of participants in response to selected environments (independent variables) including a traditional environment (based on photos and labels), a video-enhanced environment (based on projected video clips), and a VR-enhanced environment (based on video game). The main outcome of this article is demonstrating that the use of VR technology and the resulting hybrid space (i.e., VR-enhanced environment) results in novel museum experiences that provide greater impacts on audience in terms of learning and enjoyment.

User Evaluation and Metrics Analysis of a Prototype Web-Based Federated Search Engine for Art and Cultural Heritage

Content and metadata concerning a specialized field such as Art and Cultural Heritage are often scattered throughout the World Wide Web, making it hard for end-users to find, especially amid the vast and often commercialized general content of the Web. This paper presents the process of designing and developing a Federated Search Engine (FSE) that collects such content from multiple credible sources of the world of Art and Culture and presents it to the user in a unified user-oriented manner, enhancing it with added functionality. The study focuses on the challenges such an endeavor presents and the technological tools, design decisions and methodology that lead to a fully functional, Web-based platform. This implemented search engine was evaluated by a group of stakeholders from the wider fields of art, culture and media during a closed test and the insights and feedback gained by these tests are herein analyzed and presented. These insights contain both the quantitative metrics of user engagement during the testing period and the qualitative information presented by the stakeholders through interviews. The above findings are thoroughly discussed and lead to conclusions regarding the usefulness and viability of Web applications in the aggregation and diffusion of Art and Cultural Heritage related content.

Integrating, Indexing and Querying the Tangible and Intangible Cultural Heritage Available Online: The QueryLab Portal

Cultural heritage inventories have been created to collect and preserve the culture and to allow the participation of stakeholders and communities, promoting and disseminating their knowledges. There are two types of inventories: those who give data access via web services or open data, and others which are closed to external access and can be visited only through dedicated web sites, generating data silo problems. The integration of data harvested from different archives enables to compare the cultures and traditions of places from opposite sides of the world, showing how people have more in common than expected. The purpose of the developed portal is to provide query tools managing the web services provided by cultural heritage databases in a transparent way, allowing the user to make a single query and obtain results from all inventories considered at the same time. Moreover, with the introduction of the ICH-Light model, specifically studied for the mapping of intangible heritage, data from inventories of this domain can also be harvested, indexed and integrated into the portal, allowing the creation of an environment dedicated to intangible data where traditions, knowledges, rituals and festive events can be found and searched all together.

Too Low Motivation, Too High Authority? Digital Media Support for Co-Curation in Local Cultural Heritage Communities

Over the last decades, a shift towards participatory approaches could be observed in cultural heritage institutions. In co-curation processes, museums collaborate with public audiences to identify, select, prepare, and interpret cultural materials. This article focuses on the question how to engage and motivate local communities or individuals in rethinking dominant discourses or expert narratives regarding cultural heritage and bringing in their own experiences and knowledge. Based on four case studies of cultural co-curation, we delineate two basic challenges for this process: (1) Authority—even though museums strive to involve the public, there is still an imbalance in participation due to the museums’ authoritative status. (2) Motivation—participation in co-curation processes requires high levels of motivation, which are difficult to achieve. Based on the media synchronicity theory, we discuss which characteristics of new media technologies can be helpful to overcome these challenges. Media can increase awareness on counternarratives and blind spots in cultural collections. They can provide a setting where the participants can easily contribute, feel competent to do so, are empowered to rethink dominant discourses, develop a sense of relatedness with other contributors, and maintain autonomy in how and to which degree they engage in the discourse.

Impresso Inspect and Compare. Visual Comparison of Semantically Enriched Historical Newspaper Articles

The automated enrichment of mass-digitised document collections using techniques such as text mining is becoming increasingly popular. Enriched collections offer new opportunities for interface design to allow data-driven and visualisation-based search, exploration and interpretation. Most such interfaces integrate close and distant reading and represent semantic, spatial, social or temporal relations, but often lack contrastive views. Inspect and Compare (I&C) contributes to the current state of the art in interface design for historical newspapers with highly versatile side-by-side comparisons of query results and curated article sets based on metadata and semantic enrichments. I&C takes search queries and pre-curated article sets as inputs and allows comparisons based on the distributions of newspaper titles, publication dates and automatically generated enrichments, such as language, article types, topics and named entities. Contrastive views of such data reveal patterns, help humanities scholars to improve search strategies and to facilitate a critical assessment of the overall data quality. I&C is part of the impresso interface for the exploration of digitised and semantically enriched historical newspapers.

Semantic enrichment on large scanned collections through their “satellite texts”: the paradigm of Migne’s Patrologia Graeca

Purpose

This paper aims to present a methodology for the semantic enrichment on the scanned collection of Migne’s Patrologia Graeca (PG), attempting to easily locate on the Web domain the scanned PG source, when a reference of this source is described and commented on another scanned or textual document, and to semantically enrich PG through related scanned or textual documents named “satellite texts” published by third people. The present enrichment of PG uses as satellite texts the Dorotheos Scholarios's Synoptic Index (DSSI) which act as metadata for PG.

Design/methodology/approach

The methodology consists of two parts. The first part addresses the DSSI transcription via a proper web tool. The second part is divided into two subsections: the accomplishment of interlinking the printed column numbers of each scanned PG page with its actual filename, which is the build of a matching function, and the build of a web interface for PG, based on the generated Uniform Resource Identifiers (URIs) of the above first subsection.

Findings

The result of the implemented methodology is a Web portal, capable of providing server-less search of topics with direct (single click) navigation to sources. The produced system is static, scalable, easy to be managed and requires minimal cost to be completed and maintained. The produced data sets of transcribed DSSI and the JavaScript Object Notation (JSON) matching functions are available for personal use of students and scholars under Creative Commons license (CC-BY-NC-SA).

Social implications

Scholars or anyone interested in a particular subject can easily locate topics in PG and reference them, using URIs that are easy to remember. This fact contributes significantly to the related scientific dialogue.

Originality/value

The methodology uses the transcribed satellite texts of DSSI, which act as metadata for PG, to semantically enrich PG collection. Furthermore, the built PG Web interface can be used by other satellite texts as a reference basis to further enrich PG, as it provides a direct identification of sources. The presented methodology is general and can be applied to any scanned collection using its own satellite texts.

Multi-Purpose Ontology-Based Visualization of Spatio-Temporal Data: A Case Study on Silk Heritage

Due to the increasing use of data analytics, information visualization is getting more and more important. However, as data get more complex, so does visualization, often leading to ad hoc and cumbersome solutions. A recent alternative is the use of the so-called knowledge-assisted visualization tools. In this paper, we present STMaps (Spatio-Temporal Maps), a multipurpose knowledge-assisted ontology-based visualization tool of spatio-temporal data. STMaps has been (originally) designed to show, by means of an interactive map, the content of the SILKNOW project, a European research project on silk heritage. It is entirely based on ontology support, as it gets the source data from an ontology and uses also another ontology to define how data should be visualized. STMaps provides some unique features. First, it is a multi-platform application. It can work embedded in an HTML page and can also work as a standalone application over several computer architectures. Second, it can be used for multiple purposes by just changing its configuration files and/or the ontologies on which it works. As STMaps relies on visualizing spatio-temporal data provided by an ontology, the tool could be used to visualize the results of any domain (in other cultural and non-cultural contexts), provided that its datasets contain spatio-temporal information. The visualization mechanisms can also be changed by changing the visualization ontology. Third, it provides different solutions to show spatio-temporal data, and also deals with uncertain and missing information. STMaps has been tested to browse silk-related objects, discovering some interesting relationships between different objects, showing the versatility and power of the different visualization tools proposed in this paper. To the best of our knowledge, this is also the first ontology-based visualization tool applied to silk-related heritage.

Introducing Layers of Meaning (LoM): A Framework to Reduce Semantic Distance of Visualization In Humanistic Research

Information visualization (infovis) is a powerful tool for exploring rich datasets. Within humanistic research, rich qualitative data and domain culture make traditional infovis approaches appear reductive and disconnected, leading to low adoption. In this paper, we use a multi-step approach to scrutinize the relationship between infovis and the humanities and suggest new directions for it. We first look into infovis from the humanistic perspective by exploring the humanistic literature around infovis. We validate and expand those findings though a co-design workshop with humanist and infovis experts. Then, we translate our findings into guidelines for designers and conduct a design critique exercise to explore their effect on the perception of humanist researchers. Based on these steps, we introduce Layers of Meaning, a framework to reduce the semantic distance between humanist researchers and visualizations of their research material, by grounding infovis tools in time and space, physicality, terminology, nuance, and provenance.

Analysis of Cultural Meme Characteristics for Big Data of Cultural Relics

The cultural meme is the smallest unit constituting a dynasty′s culture, which has the same inheritance and variability as biological genes. Here, based on the name of cultural relics, we extract cultural memes through semantic word segmentation, word frequency statistics, and the synonym merging method, and construct dynasty cultural meme vectors. We analyzed color, auxiliary, texture, shape, and overall networks of five types of model to construct the culture network, using the social network analysis method, and explored the clustering and degrees of centrality characteristics of cultural memes. We then analyzed the similarities and differences among cultures of the dynasties. The main conclusions are as follows: (1) Cultural memes represent different cultural characteristics of dynasties, and the inheritance and differences among dynasties’ cultures are closely related to their historical background. (2) Prevalence memes construct the cultural label of dynasties, which can roughly restore the cultural appearance of dynasties through fewer prevalence memes. (3) The use of community detection with a cultural meme network can determine the clustering of dynasties′ cultures, and the degree of centrality further reflects the main cultural characteristics presented by successive dynasties.

Multi-scale optical coherence tomography imaging and visualization of Vermeer's Girl with a Pearl Earring

We demonstrate multi-scale multi-parameter optical coherence tomography (OCT) imaging and visualization of Johannes Vermeer's painting Girl with a Pearl Earring. Through automated acquisition, OCT image segmentation, and 3D volume stitching we realize OCT imaging at the scale of an entire painting. This makes it possible to image, with micrometer axial and lateral resolution, an entire painting over more than 5 orders of length scale. From the multi-scale OCT data we quantify multiple parameters in a fully automated way: the surface height, the scattering strength, and the combined glaze and varnish layer thickness. The multi-parameter OCT data of Girl with a Pearl Earring shows various features: Vermeer's brushstrokes, surface craquelure, paint losses, and restorations. Through an interactive visualization of the Girl, based on the OCT data and the optical properties of historical reconstructions of Vermeer's paint, we can virtually study the effect of the lighting condition, viewing angle, zoom level and presence/absence of glaze layer. The interactive visualization shows various new painting features. It demonstrates that the glaze layer structure and its optical properties were essential to Vermeer to create an extremely strong light to dark contrast between the figure and the background that gives the painting such an iconic aesthetic appeal.

Many Views Are Not Enough: Designing for Synoptic Insights in Cultural Collections

Cultural object collections attract and delight spectators since ancient times. Yet, they also easily overwhelm visitors due to their perceptual richness and associated information. Similarly, digitized collections appear as complex, multifaceted phenomena, which can be challenging to grasp and navigate. Though visualizations can create various types of collection overviews for that matter, they do not easily assemble into a "big picture" or lead to an integrated understanding. We introduce coherence techniques to maximize connections between multiple views and apply them to the prototype PolyCube system of collection visualization: with map, set, and network visualizations it makes spatial, categorical, and relational collection aspects visible. For the essential temporal dimension, it offers four different views: superimposition, animation, juxtaposition, and space-time cube representations. A user study confirmed that better integrated visualizations support synoptic, cross-dimensional insights. An outlook is dedicated to the system's applicability within other arts and humanities data domains.

A Data-Driven Introduction to Authors, Readings, and Techniques in Visualization for the Digital Humanities

The newly rediscovered frontier between data visualization and the digital humanities has proven to be an exciting field of experimentation for scholars from both disciplines. This fruitful collaboration is attracting researchers from other areas of science who may be willing to create visual analysis tools that promote humanities research in its many forms. However, as the collaboration grows in complexity, it may become intimidating for these scholars to get engaged in the discipline. To facilitate this task, we have built an introduction to visualization for the digital humanities that sits on a data-driven stance adopted by the authors. In order to construct a dataset representative of the discipline, we analyze citations from a core corpus on 300 publications in visualization for the humanities obtained from recent editions of the InfoVis Vis4DH workshop, the ADHO Digital Humanities Conference, and the specialized digital humanities journal Digital Humanities Quarterly. From here, we extract referenced works and analyze more than 1900 publications in search of citation patterns, prominent authors in the field, and other interesting insights. Finally, following the path set by other researchers in the visualization and Human-Computer Interaction (HCI) communities, we analyze paper keywords to identify significant themes and research opportunities in the field.

Exhibiting Uncertainty: Visualizing Data Quality Indicators for Cultural Collections

Uncertainty is a standard condition under which large parts of art-historical and curatorial knowledge creation and communication are operating. In contrast to standard levels of data quality in non-historical research domains, historical object and knowledge collections contain substantial amounts of uncertain, ambiguous, contested, or plainly missing data. Visualization approaches and interfaces to cultural collections have started to represent data quality and uncertainty metrics, yet all existing work is limited to representations for isolated metadata dimensions only. With this article, we advocate for a more systematic, synoptic and self-conscious approach to uncertainty visualization for cultural collections. We introduce omnipresent types of data uncertainty and discuss reasons for their frequent omission by interfaces for galleries, libraries, archives and museums. On this basis we argue for a coordinated counter strategy for uncertainty visualization in this field, which will also raise the efforts going into complex interface design and conceptualization. Building on the PolyCube framework for collection visualization, we showcase how multiple uncertainty representation techniques can be assessed and coordinated in a multi-perspective environment. As for an outlook, we reflect on both the strengths and limitations of making the actual wealth of data quality questions transparent with regard to different target and user groups.