Quantifying the Interplay of Semantics and Phonology During Failures of Word Retrieval by People With Aphasia Using a Multiplex Lexical Network

Using network science to examine audio-visual speech perception with a multi-layer graph

To examine visual speech perception (i.e., lip-reading), we created a multi-layer network (the AV-net) that contained: (1) an auditory layer with nodes representing phonological word-forms and edges connecting words that were phonologically related, and (2) a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme (and additional edges to connect related nodes in the two layers). The results of several computer simulations (in which activation diffused across the network to simulate word identification) are reported and compared to the performance of human participants who identified the same words in a condition in which audio and visual information were both presented (Simulation 1), in an audio-only presentation condition (Simulation 2), and a visual-only presentation condition (Simulation 3). Another simulation (Simulation 4) examined the influence of phonological information on visual speech perception by comparing performance in the multi-layer AV-net to a single-layer network that contained only a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme. We also report the results of several analyses of the errors made by human participants in the visual-only presentation condition. The results of our analyses have implications for future research and training of lip-reading, and for the development of automatic lip-reading devices and software for individuals with certain developmental or acquired disorders or for listeners with normal hearing in noisy conditions.

Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges

The mental lexicon is a complex cognitive system representing information about the words/concepts that one knows. Over decades psychological experiments have shown that conceptual associations across multiple, interactive cognitive levels can greatly influence word acquisition, storage, and processing. How can semantic, phonological, syntactic, and other types of conceptual associations be mapped within a coherent mathematical framework to study how the mental lexicon works? Here we review cognitive multilayer networks as a promising quantitative and interpretative framework for investigating the mental lexicon. Cognitive multilayer networks can map multiple types of information at once, thus capturing how different layers of associations might co-exist within the mental lexicon and influence cognitive processing. This review starts with a gentle introduction to the structure and formalism of multilayer networks. We then discuss quantitative mechanisms of psychological phenomena that could not be observed in single-layer networks and were only unveiled by combining multiple layers of the lexicon: (i) multiplex viability highlights language kernels and facilitative effects of knowledge processing in healthy and clinical populations; (ii) multilayer community detection enables contextual meaning reconstruction depending on psycholinguistic features; (iii) layer analysis can mediate latent interactions of mediation, suppression, and facilitation for lexical access. By outlining novel quantitative perspectives where multilayer networks can shed light on cognitive knowledge representations, including in next-generation brain/mind models, we discuss key limitations and promising directions for cutting-edge future research.

Multilevel factors predict treatment response following semantic feature-based intervention in bilingual aphasia

Semantic feature-based treatments (SFTs) are effective rehabilitation strategies for word retrieval deficits in bilinguals with aphasia (BWA). However, few studies have prospectively evaluated the effects of key parameters of these interventions on treatment outcomes. This study examined the influence of intervention-level (i.e., treatment language and treatment sessions), individual-level (baseline naming severity and age), and stimulus-level (i.e., lexical frequency, phonological length, and phonological neighborhood density) factors on naming improvement in a treated and untreated language for 34 Spanish-English BWA who completed 40 hours of SFT. Results revealed significant improvement over time in both languages. In the treated language, individuals who received therapy in their L1 improved more. Additionally, higher pre-treatment naming scores predicted greater response to treatment. Finally, a frequency effect on baseline naming accuracy and phonological effects on accuracy over time were associated with differential treatment gains. These findings indicate that multilevel factors are influential predictors of bilingual treatment outcomes.

Using Complex Networks in the Hearing Sciences

In this Point of View, we review a number of recent discoveries from the emerging, interdisciplinary field of Network Science , which uses graph theoretic techniques to understand complex systems. In the network science approach, nodes represent entities in a system, and connections are placed between nodes that are related to each other to form a web-like network . We discuss several studies that demonstrate how the micro-, meso-, and macro-level structure of a network of phonological word-forms influence spoken word recognition in listeners with normal hearing and in listeners with hearing loss. Given the discoveries made possible by this new approach and the influence of several complex network measures on spoken word recognition performance we argue that speech recognition measures-originally developed in the late 1940s and routinely used in clinical audiometry-should be revised to reflect our current understanding of spoken word recognition. We also discuss other ways in which the tools of network science can be used in Speech and Hearing Sciences and Audiology more broadly.

Using Network Science and Psycholinguistic Megastudies to Examine the Dimensions of Phonological Similarity

Network science was used to examine different dimensions of phonological similarity in English. Data from a phonological associate task and an identification of words in noise task were used to create a phonological association network and a misperception network. These networks were compared to a network formed by a computational metric widely used to assess phonological similarity (i.e., one-phoneme metric). The phonological association network and the misperception network were topographically more similar to each other than either were to the one-phoneme metric network, but there were several network features in common between the one-phoneme metric network and the phonological association network. To assess the influence of network structure on processing, we compared the influence of degree (i.e., neighborhood density) from each of the networks on visual and auditory lexical decision reaction times obtained from two psycholinguistic megastudies. The effect of degree differed across network types and tasks. We discuss the use of each approach to determine phonological similarity and a possible direction forward for language research through the use of multiplex networks.

The Resilience of the Phonological Network May Have Implications for Developmental and Acquired Disorders

A central tenet of network science states that the structure of the network influences processing. In this study of a phonological network of English words we asked: how does damage alter the network structure (Study 1)? How does the damaged structure influence lexical processing (Study 2)? How does the structure of the intact network "protect" processing with a less efficient algorithm (Study 3)? In Study 1, connections in the network were randomly removed to increasingly damage the network. Various measures showed the network remained well-connected (i.e., it is resilient to damage) until ~90% of the connections were removed. In Study 2, computer simulations examined the retrieval of a set of words. The performance of the model was positively correlated with naming accuracy by people with aphasia (PWA) on the Philadelphia Naming Test (PNT) across four types of aphasia. In Study 3, we demonstrated another way to model developmental or acquired disorders by manipulating how efficiently activation spread through the network. We found that the structure of the network "protects" word retrieval despite decreases in processing efficiency; words that are relatively easy to retrieve with efficient transmission of priming remain relatively easy to retrieve with less efficient transmission of priming. Cognitive network science and computer simulations may provide insight to a wide range of speech, language, hearing, and cognitive disorders.

Stimulus- and Person-Level Variables Influence Word Production and Response to Anomia Treatment for Individuals With Chronic Poststroke Aphasia

PURPOSE

The impact of stimulus-level psycholinguistic variables and person-level semantic and phonological processing skills on treatment outcomes in individuals with aphasia requires further examination to inform clinical decision making in treatment prescription and stimuli selection. This study investigated the influence of stimulus-level psycholinguistic properties and person-level semantic and phonological processing skills on word production accuracy and treatment response.

METHOD

This retrospective analysis included 35 individuals with chronic, poststroke aphasia, 30 of whom completed typicality-based semantic feature treatment. Mixed-effects logistic regression models were used to predict binary naming accuracy (a) at baseline and (b) over the course of treatment using stimulus-level psycholinguistic word properties and person-level semantic and phonological processing skills as predictors.

RESULTS

In baseline naming, words with less complex lexical-semantic and phonological properties showed greater predicted accuracy. There was also an interaction at baseline between stimulus-level lexical-semantic properties and person-level semantic processing skills in predicting baseline naming accuracy. With treatment, words that were more complex from a lexical-semantic standpoint (vs. less complex) and less complex from a phonological standpoint (vs. more complex) improved more. Individuals with greater baseline semantic and phonological processing skills showed a greater treatment response.

CONCLUSIONS

This study suggests that future clinical research and clinical work should consider semantic and phonological properties of words in selecting stimuli for semantically based treatment. Furthermore, future clinical research should continue to evaluate baseline individual semantic and phonological profiles as predictors of response to semantically based treatment.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21256341.

DASentimental: Detecting Depression, Anxiety, and Stress in Texts via Emotional Recall, Cognitive Networks, and Machine Learning

Most current affect scales and sentiment analysis on written text focus on quantifying valence/sentiment, the primary dimension of emotion. Distinguishing broader, more complex negative emotions of similar valence is key to evaluating mental health. We propose a semi-supervised machine learning model, DASentimental, to extract depression, anxiety, and stress from written text. We trained DASentimental to identify how N = 200 sequences of recalled emotional words correlate with recallers’ depression, anxiety, and stress from the Depression Anxiety Stress Scale (DASS-21). Using cognitive network science, we modeled every recall list as a bag-of-words (BOW) vector and as a walk over a network representation of semantic memory—in this case, free associations. This weights BOW entries according to their centrality (degree) in semantic memory and informs recalls using semantic network distances, thus embedding recalls in a cognitive representation. This embedding translated into state-of-the-art, cross-validated predictions for depression (R = 0.7), anxiety (R = 0.44), and stress (R = 0.52), equivalent to previous results employing additional human data. Powered by a multilayer perceptron neural network, DASentimental opens the door to probing the semantic organizations of emotional distress. We found that semantic distances between recalls (i.e., walk coverage), was key for estimating depression levels but redundant for anxiety and stress levels. Semantic distances from “fear” boosted anxiety predictions but were redundant when the “sad–happy” dyad was considered. We applied DASentimental to a clinical dataset of 142 suicide notes and found that the predicted depression and anxiety levels (high/low) corresponded to differences in valence and arousal as expected from a circumplex model of affect. We discuss key directions for future research enabled by artificial intelligence detecting stress, anxiety, and depression in texts.

What Do Cognitive Networks Do? Simulations of Spoken Word Recognition Using the Cognitive Network Science Approach

Cognitive network science is an emerging approach that uses the mathematical tools of network science to map the relationships among representations stored in memory to examine how that structure might influence processing. In the present study, we used computer simulations to compare the ability of a well-known model of spoken word recognition, TRACE, to the ability of a cognitive network model with a spreading activation-like process to account for the findings from several previously published behavioral studies of language processing. In all four simulations, the TRACE model failed to retrieve a sufficient number of words to assess if it could replicate the behavioral findings. The cognitive network model successfully replicated the behavioral findings in Simulations 1 and 2. However, in Simulation 3a, the cognitive network did not replicate the behavioral findings, perhaps because an additional mechanism was not implemented in the model. However, in Simulation 3b, when the decay parameter in spreadr was manipulated to model this mechanism the cognitive network model successfully replicated the behavioral findings. The results suggest that models of cognition need to take into account the multi-scale structure that exists among representations in memory, and how that structure can influence processing.

Exploring How Phonotactic Knowledge Can Be Represented in Cognitive Networks

In Linguistics and Psycholinguistics, phonotactics refers to the constraints on individual sounds in a given language that restrict how those sounds can be ordered to form words in that language. Previous empirical work in Psycholinguistics demonstrated that phonotactic knowledge influenced how quickly and accurately listeners retrieved words from that part of memory known as the mental lexicon. In the present study, we used three computer simulations to explore how three different cognitive network architectures could account for the previously observed effects of phonotactics on processing. The results of Simulation 1 showed that some—but not all—effects of phonotactics could be accounted for in a network where nodes represent words and edges connect words that are phonologically related to each other. In Simulation 2, a different network architecture was used to again account for some—but not all—effects of phonotactics and phonological neighborhood density. A bipartite network was used in Simulation 3 to account for many of the previously observed effects of phonotactic knowledge on spoken word recognition. The value of using computer simulations to explore different network architectures is discussed.

What Can Network Science Tell Us About Phonology and Language Processing?

Contemporary psycholinguistic models place significant emphasis on the cognitive processes involved in the acquisition, recognition, and production of language but neglect many issues related to the representation of language‐related information in the mental lexicon. In contrast, a central tenet of network science is that the structure of a network influences the processes that operate in that system, making process and representation inextricably connected. Here, we consider how the structure found across phonological networks of several languages from different language families may influence language processing as we age and experience diseases that affect cognition during the typical and atypical acquisition of new words, during typical perception and production of speech in adults, and during language change over time. We conclude that the network science approach may not only provide insights into specific language processes but also provide a way to connect the work from these domains, which are becoming increasingly balkanized.

Methodological Considerations for Incorporating Clinical Data Into a Network Model of Retrieval Failures

Difficulty retrieving information (e.g., words) from memory is prevalent in neurogenic communication disorders (e.g., aphasia and dementia). Theoretical modeling of retrieval failures often relies on clinical data, despite methodological limitations (e.g., locus of retrieval failure, heterogeneity of individuals, and progression of disorder/disease). Techniques from network science are naturally capable of handling these limitations. This paper reviews recent work using a multiplex lexical network to account for word retrieval failures and highlights how network science can address the limitations of clinical data. Critically, any model we employ could impact clinical practice and patient lives, harkening the need for theoretically well-informed network models.