The reality of doing things with (thousands of) words in applied research and clinical settings: A commentary on Clarke et al. (2020)

Towards a temporospatial framework for measurements of disorganization in speech using semantic vectors

Incoherent speech in schizophrenia has long been described as the mind making "leaps" of large distances between thoughts and ideas. Such a view seems intuitive, and for almost two decades, attempts to operationalize these conceptual "leaps" in spoken word meanings have used language-based embedding spaces. An embedding space represents meaning of words as numerical vectors where a greater proximity between word vectors represents more shared meaning. However, there are limitations with word vector-based operationalizations of coherence which can limit their appeal and utility in clinical practice. First, the use of esoteric word embeddings can be conceptually hard to grasp, and this is complicated by several different operationalizations of incoherent speech. This problem can be overcome by a better visualization of methods. Second, temporal information from the act of speaking has been largely neglected since models have been built using written text, yet speech is spoken in real time. This issue can be resolved by leveraging time stamped transcripts of speech. Third, contextual information - namely the situation of where something is spoken - has often only been inferred and never explicitly modeled. Addressing this situational issue opens up new possibilities for models with increased temporal resolution and contextual relevance. In this paper, direct visualizations of semantic distances are used to enable the inspection of examples of incoherent speech. Some common operationalizations of incoherence are illustrated, and suggestions are made for how temporal and spatial contextual information can be integrated in future implementations of measures of incoherence.

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders.

Reflections on measuring disordered thoughts as expressed via language

Thought disorder, as inferred from disorganized and incoherent speech, is an important part of the clinical presentation in schizophrenia. Traditional measurement approaches essentially count occurrences of certain speech events which may have restricted their usefulness. Applying speech technologies in assessment can help automate traditional clinical rating tasks and thereby complement the process. Adopting these computational approaches affords clinical translational opportunities to enhance the traditional assessment by applying such methods remotely and scoring various parts of the assessment automatically. Further, digital measures of language may help detect subtle clinically significant signs and thus potentially disrupt the usual manner by which things are conducted. If proven beneficial to patient care, methods where patients' voice are the primary data source could become core components of future clinical decision support systems that improve risk assessment. However, even if it is possible to measure thought disorder in a sensitive, reliable and efficient manner, there remain many challenges to then translate into a clinically implementable tool that can contribute towards providing better care. Indeed, embracing technology - notably artificial intelligence - requires vigorous standards for reporting underlying assumptions so as to ensure a trustworthy and ethical clinical science.