Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections

Personalized Language Model Selection Through Gamified Elicitation of Contrastive Concept Preferences

Language models are widely used for different Natural Language Processing tasks while suffering from a lack of personalization. Personalization can be achieved by, e.g., fine-tuning the model on training data that is created by the user (e.g., social media posts). Previous work shows that the acquisition of such data can be challenging. Instead of adapting the model's parameters, we thus suggest selecting a model that matches the user's mental model of different thematic concepts in language. In this article, we attempt to capture such individual language understanding of users. In this process, two challenges have to be considered. First, we need to counteract disengagement since the task of communicating one's language understanding typically encompasses repetitive and time-consuming actions. Second, we need to enable users to externalize their mental models in different contexts, considering that language use changes depending on the environment. In this article, we integrate methods of gamification into a visual analytics (VA) workflow to engage users in sharing their knowledge within various contexts. In particular, we contribute the design of a gameful VA playground called Concept Universe. During the four-phased game, the users build personalized concept descriptions by explaining given concept names through representative keywords. Based on their performance, the system reacts with constant visual, verbal, and auditory feedback. We evaluate the system in a user study with six participants, showing that users are engaged and provide more specific input when facing a virtual opponent. We use the generated concepts to make personalized language model suggestions.

TopicRefiner: Coherence-Guided Steerable LDA for Visual Topic Enhancement

This article presents a new Human-steerable Topic Modeling (HSTM) technique. Unlike existing techniques commonly relying on matrix decomposition-based topic models, we extend LDA as the fundamental component for extracting topics. LDA's high popularity and technical characteristics, such as better topic quality and no need to cherry-pick terms to construct the document-term matrix, ensure better applicability. Our research revolves around two inherent limitations of LDA. First, the principle of LDA is complex. Its calculation process is stochastic and difficult to control. We thus give a weighting method to incorporate users' refinements into the Gibbs sampling to control LDA. Second, LDA often runs on a corpus with massive terms and documents, forming a vast search space for users to find semantically relevant or irrelevant objects. We thus design a visual editing framework based on the coherence metric, proven to be the most consistent with human perception in assessing topic quality, to guide users' interactive refinements. Cases on two open real-world datasets, participants' performance in a user study, and quantitative experiment results demonstrate the usability and effectiveness of the proposed technique.

Impact of word embedding models on text analytics in deep learning environment: a review

The selection of word embedding and deep learning models for better outcomes is vital. Word embeddings are an n-dimensional distributed representation of a text that attempts to capture the meanings of the words. Deep learning models utilize multiple computing layers to learn hierarchical representations of data. The word embedding technique represented by deep learning has received much attention. It is used in various natural language processing (NLP) applications, such as text classification, sentiment analysis, named entity recognition, topic modeling, etc. This paper reviews the representative methods of the most prominent word embedding and deep learning models. It presents an overview of recent research trends in NLP and a detailed understanding of how to use these models to achieve efficient results on text analytics tasks. The review summarizes, contrasts, and compares numerous word embedding and deep learning models and includes a list of prominent datasets, tools, APIs, and popular publications. A reference for selecting a suitable word embedding and deep learning approach is presented based on a comparative analysis of different techniques to perform text analytics tasks. This paper can serve as a quick reference for learning the basics, benefits, and challenges of various word representation approaches and deep learning models, with their application to text analytics and a future outlook on research. It can be concluded from the findings of this study that domain-specific word embedding and the long short term memory model can be employed to improve overall text analytics task performance.

A Survey of Domain Knowledge Elicitation in Applied Machine Learning

Eliciting knowledge from domain experts can play an important role throughout the machine learning process, from correctly specifying the task to evaluating model results. However, knowledge elicitation is also fraught with challenges. In this work, we consider why and how machine learning researchers elicit knowledge from experts in the model development process. We develop a taxonomy to characterize elicitation approaches according to the elicitation goal, elicitation target, elicitation process, and use of elicited knowledge. We analyze the elicitation trends observed in 28 papers with this taxonomy and identify opportunities for adding rigor to these elicitation approaches. We suggest future directions for research in elicitation for machine learning by highlighting avenues for further exploration and drawing on what we can learn from elicitation research in other fields.

Defragmenting Research Areas with Knowledge Visualization and Visual Text Analytics

The increasing specialization of science is motivating the fragmentation of traditional and well-established research areas into interdisciplinary communities of practice that focus on cooperation between experts to solve problems in a wide range of domains. This is the case of problem-driven visualization research (PDVR), in which groups of scholars use visualization techniques in different application domains such as the digital humanities, bioinformatics, sports science, or computer security. In this paper, we employ the findings obtained during the development of a novel visual text analytics tool we built in previous studies, GlassViz, to automatically detect interesting knowledge associations and groups of common interests between these communities of practice. Our proposed method relies on the statistical modeling of author-assigned keywords to make its findings, which are demonstrated in two use cases. The results show that it is possible to propose interactive, semisupervised visual approaches that aim at defragmenting a body of research using text-based, automatic literature analysis methods.

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models; (2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phases of the suggested pipeline within the commonly used TensorBoard environment. We performed a user-study with nine participants across different expertise levels to examine their perception of our workflow and to collect suggestions to fill the gap between our system and framework. The evaluation confirms that our tightly integrated system leads to an informed machine learning process while disclosing opportunities for further extensions.