Applications of collaborative annotation system in digital curation, crowdsourcing, and digital humanities

Evaluating a Taxonomy of Textual Uncertainty for Collaborative Visualisation in the Digital Humanities

The capture, modelling and visualisation of uncertainty has become a hot topic in many areas of science, such as the digital humanities (DH). Fuelled by critical voices among the DH community, DH scholars are becoming more aware of the intrinsic advantages that incorporating the notion of uncertainty into their workflows may bring. Additionally, the increasing availability of ubiquitous, web-based technologies has given rise to many collaborative tools that aim to support DH scholars in performing remote work alongside distant peers from other parts of the world. In this context, this paper describes two user studies seeking to evaluate a taxonomy of textual uncertainty aimed at enabling remote collaborations on digital humanities (DH) research objects in a digital medium. Our study focuses on the task of free annotation of uncertainty in texts in two different scenarios, seeking to establish the requirements of the underlying data and uncertainty models that would be needed to implement a hypothetical collaborative annotation system (CAS) that uses information visualisation and visual analytics techniques to leverage the cognitive effort implied by these tasks. To identify user needs and other requirements, we held two user-driven design experiences with DH experts and lay users, focusing on the annotation of uncertainty in historical recipes and literary texts. The lessons learned from these experiments are gathered in a series of insights and observations on how these different user groups collaborated to adapt an uncertainty taxonomy to solve the proposed exercises. Furthermore, we extract a series of recommendations and future lines of work that we share with the community in an attempt to establish a common agenda of DH research that focuses on collaboration around the idea of uncertainty.

A character social network relationship map tool to facilitate digital humanities research

Purpose

Digital humanities aim to use a digital-based revolutionary new way to carry out enhanced forms of humanities research more effectively and efficiently. This study develops a character social network relationship map tool (CSNRMT) that can semi-automatically assist digital humanists through human-computer interaction to more efficiently and accurately explore the character social network relationships from Chinese ancient texts for useful research findings.

Design/methodology/approach

With a counterbalanced design, semi-structured in-depth interview, and lag sequential analysis, a total of 21 research subjects participated in an experiment to examine the system effectiveness and technology acceptance of adopting the ancient book digital humanities research platform with and without the CSNRMT to interpret the characters and character social network relationships.

Findings

The experimental results reveal that the experimental group with the CSNRMT support appears higher system effectiveness on the interpretation of characters and character social network relationships than the control group without the CSNRMT, but does not achieve a statistically significant difference. Encouragingly, the experimental group with the CSNRMT support presents remarkably higher technology acceptance than the control group without the CSNRMT. Furthermore, use behaviors analyzed by lag sequential analysis reveal that the CSNRMT could assist digital humanists in the interpretation of character social network relationships. The results of the interview present positive opinions on the integration of system interface, smoothness of operation, and external search function.

Research limitations/implications

Currently, the system effectiveness of exploring the character social network relationships from texts for useful research findings by using the CSNRMT developed in this study will be significantly affected by the accuracy of recognizing character names and character social network relationships from Chinese ancient texts. The developed CSNRMT will be more practical when the offered information about character names and character social network relationships is more accurate and broad.

Practical implications

This study develops an ancient book digital humanities research platform with an emerging CSNRMT that provides an easy-to-use real-time interaction interface to semi-automatically support digital humanists to perform digital humanities research with the need of exploring character social network relationships.

Originality/value

At present, a real-time social network analysis tool to provide a friendly interaction interface and effectively assist digital humanists in the digital humanities research with character social networks analysis is still lacked. This study thus presents the CSNRMT that can semi-automatically identify character names from Chinese ancient texts and provide an easy-to-use real-time interaction interface for supporting digital humanities research so that digital humanists could more efficiently and accurately establish character social network relationships from the analyzed texts to explore complicated character social networks relationship and find out useful research findings.

An innovative citation recommendation model for draft papers with varying degrees of information completeness

Purpose

As researchers are writing a draft paper with incomplete structure or text, one of burdensome tasks is to deliberate about which references should be cited for one sentence or paragraph of this draft. In view of the rapid increase in the number of research papers, researchers desire to figure out a better way to do citation recommendations in developing their draft papers. The purpose of this paper is to propose citation recommendation algorithms that enable the acquisition of relevant citations for research papers that are still at the drafting stage. This study attempts to help researchers to select appropriate references among the vast amount of available papers and make draft papers complete in reference citation.

Design/methodology/approach

This study adopts a model for recommending citations for incomplete drafts. Four algorithms are proposed in this study. The first and second algorithms are unsupervised models, applying term frequency-inverse document frequency and WordNet technologies, respectively. The third and fourth algorithms are based on the second algorithm to integrate different weight adjustment strategies to improve performance.

Findings

The proposed recommendation method adopts three techniques, including using WordNet to transform vector and setting adjustment weights according to structural factors and the information completeness degree, to generate citation recommendation for incomplete drafts. The experiments show that all these three techniques can significantly improve the recommendation accuracy.

Originality/value

None of the methods employed in previous studies can recommend articles as references for incomplete drafts. This paper addresses the situation that a draft paper can be incomplete either in structure or text or both. Recommended references, however, can be still generated and inserted into any desired sentence of the draft paper.

Development and evaluation of an automatic text annotation system for supporting digital humanities research

Purpose

An automatic text annotation system (ATAS) that can collect resources from different databases through Linked Data (LD) for automatically annotating ancient texts was developed in this study to support digital humanities research. It allows the humanists referring to resources from diverse databases when interpreting ancient texts as well as provides a friendly text annotation reader for humanists interpreting ancient text through reading. The paper aims to discuss whether the ATAS is helpful to support digital humanities research or not.

Design/methodology/approach

Based on the quasi-experimental design, the ATAS developed in this study and MARKUS semi-ATAS were compared whether the significant differences in the reading effectiveness and technology acceptance for supporting humanists interpreting ancient text of the Ming dynasty’s collections existed or not. Additionally, lag sequential analysis was also used to analyze users’ operation behaviors on the ATAS. A semi-structured in-depth interview was also applied to understand users’ opinions and perception of using the ATAS to interpret ancient texts through reading.

Findings

The experimental results reveal that the ATAS has higher reading effectiveness than MARKUS semi-ATAS, but not reaching the statistically significant difference. The technology acceptance of the ATAS is significantly higher than that of MARKUS semi-ATAS. Particularly, the function comparison of the two systems shows that the ATAS presents more perceived ease of use on the functions of term search, connection to source websites and adding annotation than MARKUS semi-ATAS. Furthermore, the reading interface of ATAS is simple and understandable and is more suitable for reading than MARKUS semi-ATAS. Among all the considered LD sources, Moedict, which is an online Chinese dictionary, was confirmed as the most helpful one.

Research limitations/implications

This study adopted Jieba Chinese parser to perform the word segmentation process based on a parser lexicon for the Chinese ancient texts of the Ming dynasty’s collections. The accuracy of word segmentation to a lexicon-based Chinese parser is limited due to ignoring the grammar and semantics of ancient texts. Moreover, the original parser lexicon used in Jieba Chinese parser only contains the modern words. This will reduce the accuracy of word segmentation for Chinese ancient texts. The two limitations that affect Jieba Chinese parser to correctly perform the word segmentation process for Chinese ancient texts will significantly affect the effectiveness of using ATAS to support digital humanities research. This study thus proposed a practicable scheme by adding new terms into the parser lexicon based on humanists’ self-judgment to improve the accuracy of word segmentation of Jieba Chinese parser.

Practical implications

Although some digital humanities platforms have been successfully developed to support digital humanities research for humanists, most of them have still not provided a friendly digital reading environment to support humanists on interpreting texts. For this reason, this study developed an ATAS that can automatically retrieve LD sources from different databases on the Internet to supply rich annotation information on reading texts to help humanists interpret texts. This study brings digital humanities research to a new ground.

Originality/value

This study proposed a novel ATAS that can automatically annotate useful information on an ancient text to increase the readability of the ancient text based on LD sources from different databases, thus helping humanists obtain a deeper and broader understanding in the ancient text. Currently, there is no this kind of tool developed for humanists to support digital humanities research.

A visual interactive reading system based on eye tracking technology to improve digital reading performance

Purpose

Developing attention-aware systems and interfaces based on eye tracking technology could revolutionize mainstream human–computer interaction to make the interaction between human beings and computers more intuitive, effective and immersive than can be achieved traditionally using a computer mouse. This paper aims to propose an eye-controlled interactive reading system (ECIRS) that uses human eyes instead of the traditional mouse to control digital text to support screen-based digital reading.

Design/methodology/approach

This study uses a quasi-experimental design to examine the effects of an experimental group and a control group of learners who, respectively, used the ECIRS and a mouse-controlled interactive reading system (MCIRS) to conduct their reading of two types of English-language text online – pure text and Q&A-type articles on reading comprehension, cognitive load, technology acceptance, and reading behavioural characteristics. Additionally, the effects of learners with field-independent (FI) and field-dependence (FD) cognitive styles who, respectively, used the ECIRS and MCIRS to conduct their reading of two types of English-language text online – pure text and Q&A-type articles on reading comprehension are also examined.

Findings

Analytical results reveal that the reading comprehension of learners in the experimental group significantly exceeded those in the control group for the Q&A article, but the difference was insignificant for the pure text article. Moreover, the ECIRS improved the reading comprehension of field-independent learners more than it did that of field-dependent learners. Moreover, neither the cognitive loads of the two groups nor their acceptance of the technology differed significantly, whereas the reading time of the experimental group significantly exceeded that of the control group. Interestingly, for all articles, the control group of learners read mostly from top to bottom without repetition, whereas most of the learners in the experimental group read most paragraphs more than once. Clearly, the proposed ECIRS supports deeper digital reading than does the MCIRS.

Originality/value

This study proposes an emerging ECIRS that can automatically provide supplementary information to a reader and control a reading text based on a reader’s eye movement to replace the widely used mouse-controlled reading system on a computer screen to effectively support digital reading for English language learning. The implications of this study are that the highly interactive reading patterns of digital text with ECIRS support increase motivation and willingness to learn while giving learners a more intuitive and natural reading experience as well as reading an article online with ECIRS support guides learners’ attention in deeper digital reading than does the MCIRS because of simultaneously integrating perceptual and cognitive processes of selection, awareness and control based on human eye movement.

A Chinese ancient book digital humanities research platform to support digital humanities research

Purpose

With the rapid development of digital humanities, some digital humanities platforms have been successfully developed to support digital humanities research for humanists. However, most of them have still not provided a friendly digital reading environment and practicable social network analysis tool to support humanists on interpreting texts and exploring characters’ social network relationships. Moreover, the advancement of digitization technologies for the retrieval and use of Chinese ancient books is arising an unprecedented challenge and opportunity. For these reasons, this paper aims to present a Chinese ancient books digital humanities research platform (CABDHRP) to support historical China studies. In addition to providing digital archives, digital reading, basic search and advanced search functions for Chinese ancient books, this platform still provides two novel functions that can more effectively support digital humanities research, including an automatic text annotation system (ATAS) for interpreting texts and a character social network relationship map tool (CSNRMT) for exploring characters’ social network relationships.

Design/methodology/approach

This study adopted DSpace, an open-source institutional repository system, to serve as a digital archives system for archiving scanned images, metadata, and full texts to develop the CABDHRP for supporting digital humanities (DH) research. Moreover, the ATAS developed in the CABDHRP used the Node.js framework to implement the system’s front- and back-end services, as well as application programming interfaces (APIs) provided by different databases, such as China Biographical Database (CBDB) and TGAZ, used to retrieve the useful linked data (LD) sources for interpreting ancient texts. Also, Neo4j which is an open-source graph database management system was used to implement the CSNRMT of the CABDHRP. Finally, JavaScript and jQuery were applied to develop a monitoring program embedded in the CABDHRP to record the use processes from humanists based on xAPI (experience API). To understand the research participants’ perception when interpreting the historical texts and characters’ social network relationships with the support of ATAS and CSNRMT, semi-structured interviews with 21 research participants were conducted.

Findings

An ATAS embedded in the reading interface of CABDHRP can collect resources from different databases through LD for automatically annotating ancient texts to support digital humanities research. It allows the humanists to refer to resources from diverse databases when interpreting ancient texts, as well as provides a friendly text annotation reader for humanists to interpret ancient text through reading. Additionally, the CSNRMT provided by the CABDHRP can semi-automatically identify characters’ names based on Chinese word segmentation technology and humanists’ support to confirm and analyze characters’ social network relationships from Chinese ancient books based on visualizing characters’ social networks as a knowledge graph. The CABDHRP not only can stimulate humanists to explore new viewpoints in a humanistic research, but also can promote the public to emerge the learning interest and awareness of Chinese ancient books.

Originality/value

This study proposed a novel CABDHRP that provides the advanced features, including the automatic word segmentation of Chinese text, automatic Chinese text annotation, semi-automatic character social network analysis and user behavior analysis, that are different from other existed digital humanities platforms. Currently, there is no this kind of digital humanities platform developed for humanists to support digital humanities research.