Interactive activation and competition models and semantic context: From behavioral to brain data

Revealing brain's cognitive process deeply: a study of the consistent EEG patterns of audio-visual perceptual holistic

Introduction

To investigate the brain's cognitive process and perceptual holistic, we have developed a novel method that focuses on the informational attributes of stimuli.

Methods

We recorded EEG signals during visual and auditory perceptual cognition experiments and conducted ERP analyses to observe specific positive and negative components occurring after 400ms during both visual and auditory perceptual processes. These ERP components represent the brain's perceptual holistic processing activities, which we have named Information-Related Potentials (IRPs). We combined IRPs with machine learning methods to decode cognitive processes in the brain.

Results

Our experimental results indicate that IRPs can better characterize information processing, particularly perceptual holism. Additionally, we conducted a brain network analysis and found that visual and auditory perceptual holistic processing share consistent neural pathways.

Discussion

Our efforts not only demonstrate the specificity, significance, and reliability of IRPs but also reveal their great potential for future brain mechanism research and BCI applications.

Neuropsychological differential diagnosis of Alzheimer's disease and vascular dementia: a systematic review with meta-regressions

Introduction

Diagnostic classification systems and guidelines posit distinguishing patterns of impairment in Alzheimer's (AD) and vascular dementia (VaD). In our study, we aim to identify which diagnostic instruments distinguish them.

Methods

We searched PubMed and PsychInfo for empirical studies published until December 2020, which investigated differences in cognitive, behavioral, psychiatric, and functional measures in patients older than 64 years and reported information on VaD subtype, age, education, dementia severity, and proportion of women. We systematically reviewed these studies and conducted Bayesian hierarchical meta-regressions to quantify the evidence for differences using the Bayes factor (BF). The risk of bias was assessed using the Newcastle-Ottawa-Scale and funnel plots.

Results

We identified 122 studies with 17,850 AD and 5,247 VaD patients. Methodological limitations of the included studies are low comparability of patient groups and an untransparent patient selection process. In the digit span backward task, AD patients were nine times more probable (BF = 9.38) to outperform VaD patients (β g = 0.33, 95% ETI = 0.12, 0.52). In the phonemic fluency task, AD patients outperformed subcortical VaD (sVaD) patients (β g = 0.51, 95% ETI = 0.22, 0.77, BF = 42.36). VaD patients, in contrast, outperformed AD patients in verbal (β g = -0.61, 95% ETI = -0.97, -0.26, BF = 22.71) and visual (β g = -0.85, 95% ETI = -1.29, -0.32, BF = 13.67) delayed recall. We found the greatest difference in verbal memory, showing that sVaD patients outperform AD patients (β g = -0.64, 95% ETI = -0.88, -0.36, BF = 72.97). Finally, AD patients performed worse than sVaD patients in recognition memory tasks (β g = -0.76, 95% ETI = -1.26, -0.26, BF = 11.50).

Conclusion

Our findings show inferior performance of AD in episodic memory and superior performance in working memory. We found little support for other differences proposed by diagnostic systems and diagnostic guidelines. The utility of cognitive, behavioral, psychiatric, and functional measures in differential diagnosis is limited and should be complemented by other information. Finally, we identify research areas and avenues, which could significantly improve the diagnostic value of cognitive measures.

Verbal working memory encodes phonological and semantic information differently

Working memory (WM) is often tested through immediate serial recall of word lists. Performance in such tasks is negatively influenced by phonological similarity: People more often get the order of words wrong when they are phonologically similar to each other (e.g., cat, fat, mat). This phonological-similarity effect shows that phonology plays an important role for the representation of serial order in these tasks. By contrast, semantic similarity usually does not impact performance negatively. To resolve and understand this discrepancy, we tested the effects of phonological and semantic similarity for the retention of positional information in WM. Across six experiments (all Ns = 60 young adults), we manipulated between-item semantic and phonological similarity in tasks requiring participants to form and maintain new item-context bindings in WM. Participants were asked to retrieve items from their context, or the contexts from their item. For both retrieval directions, phonological similarity impaired WM for item-context bindings across all experiments. Semantic similarity did not. These results demonstrate that WM encodes phonological and semantic information differently. We propose a WM model accounting for semantic-similarity effects in WM, in which semantic knowledge supports WM through activated long-term memory.

Dual-Language Testing of Emotional Verbal Fluency: A Closer Look at "Joy," "Sadness," "Fear," "Anger," and "Disgust"

OBJECTIVE

Assessing neuropsychological functions of dual-language speakers with semantic verbal fluency should consider the impact of language proficiency. Much evidence for the proficiency effect is found in tasks targeting neutral words. The proficiency effect on emotional verbal fluency (Emo-VF) that targets emotional word retrieval, however, is far from conclusive. This study aimed to clarify the proficiency effect on Emo-VF, specifically the extent to which language proficiency impacted positive and negative word retrieval comparably. In addition, the study examined the extent to which dual-languages speakers produced duplicated items and unique, non-duplicated items in each of two languages tested.

METHOD

Thirty-two adult participants completed Emo-VF tasks that targeted a comprehensive set of basic emotions ("joy," "sadness," "fear," "anger," "disgust") in English and Spanish in two sessions separated by at least 72 h. All participants exhibited greater proficiency in English than in Spanish according to subjective and objective measures of language proficiency.

RESULTS

Verbal productivity was comparably lowered for all emotions in the less proficient language. Differences among categories were consistent between languages, with "joy" eliciting more words than other emotions, and "fear" yielding the highest productivity among negative emotions. Finally, dual-language speakers produced a significant number of unique items in different languages, especially for "fear."

CONCLUSION

Language proficiency exerts a general effect on positive and negative word retrieval. The study extends previous work by revealing differences between "fear" and other negative categories in addition to replicating positivity biases in unbalanced dual-language speakers. Dual-language testing is valuable in capturing the richness of emotional word repertoire associated with different languages.

Structural gray matter features and behavioral preliterate skills predict future literacy - A machine learning approach

When children learn to read, their neural system undergoes major changes to become responsive to print. There seem to be nuanced interindividual differences in the neurostructural anatomy of regions that later become integral parts of the reading network. These differences might affect literacy acquisition and, in some cases, might result in developmental disorders like dyslexia. Consequently, the main objective of this longitudinal study was to investigate those interindividual differences in gray matter morphology that might facilitate or hamper future reading acquisition. We used a machine learning approach to examine to what extent gray matter macrostructural features and cognitive-linguistic skills measured before formal literacy teaching could predict literacy 2 years later. Forty-two native German-speaking children underwent T1-weighted magnetic resonance imaging and psychometric testing at the end of kindergarten. They were tested again 2 years later to assess their literacy skills. A leave-one-out cross-validated machine-learning regression approach was applied to identify the best predictors of future literacy based on cognitive-linguistic preliterate behavioral skills and cortical measures in a priori selected areas of the future reading network. With surprisingly high accuracy, future literacy was predicted, predominantly based on gray matter volume in the left occipito-temporal cortex and local gyrification in the left insular, inferior frontal, and supramarginal gyri. Furthermore, phonological awareness significantly predicted future literacy. In sum, the results indicate that the brain morphology of the large-scale reading network at a preliterate age can predict how well children learn to read.

Semantic feature activation takes time: longer SOA elicits earlier priming effects during reading

While most previous studies of "semantic" priming confound associative and semantic relations, here we use a simple co-occurrence-based approach to examine "pure" semantic priming, while experimentally controlling for associative relations. We define associative relations by the co-occurrence of words in the sentences of a large text corpus. Contextual-semantic feature overlap, in contrast, is defined by the number of common associates that the prime shares with the target. Then we revisit the spreading activation theory and examine whether a long vs. short time available for semantic feature activation leads to early vs. late viewing time effects on the target words of a sentence reading experiment. We independently manipulate contextual-semantic feature overlap of two primes with one target word in sentences of the form pronoun, verb prime, article, adjective prime and target noun, e. g., "She rides the gray elephant." The results showed that long-SOA (verb-noun) overlap reduces early single and first fixation durations of the target noun, and short-SOA (adjective-noun) overlap reduces late go-past durations. This result pattern can be explained by the spreading activation theory: The semantic features of the prime words need some time to become sufficiently active before they can reliably affect target processing. Therefore, the verb can act on the target noun's early eye-movement measures presented three words later, while the adjective is presented immediately prior to the target-thus a difficult adjective-noun semantic integration leads to a late sentence re-examination of the preceding words.

Computational Models of Readers' Apperceptive Mass

Recent progress in machine-learning-based distributed semantic models (DSMs) offers new ways to simulate the apperceptive mass (AM; Kintsch, 1980) of reader groups or individual readers and to predict their performance in reading-related tasks. The AM integrates the mental lexicon with world knowledge, as for example, acquired via reading books. Following pioneering work by Denhière and Lemaire (2004), here, we computed DSMs based on a representative corpus of German children and youth literature (Jacobs et al., 2020) as null models of the part of the AM that represents distributional semantic input, for readers of different reading ages (grades 1–2, 3–4, and 5–6). After a series of DSM quality tests, we evaluated the performance of these models quantitatively in various tasks to simulate the different reader groups' hypothetical semantic and syntactic skills. In a final study, we compared the models' performance with that of human adult and children readers in two rating tasks. Overall, the results show that with increasing reading age performance in practically all tasks becomes better. The approach taken in these studies reveals the limits of DSMs for simulating human AM and their potential for applications in scientific studies of literature, research in education, or developmental science.

Micropoetry Meets Neurocognitive Poetics: Influence of Associations on the Reception of Poetry

Reading and understanding poetic texts is often described as an interactive process influenced by the words and phrases building the poems and all associations and images induced by them in the readers mind. Iser, for example, described the understanding process as the closing of a good Gestalt promoted by mental images. Here, we investigate the effect that semantic cohesion, that is the internal connection of a list words, has on understanding and appreciation of poetic texts. To do this, word lists are presented as modern micropoems to the participants and the (ease of) extraction of underlying concepts as well as the affective and aesthetic responses are implicitly and explicitly measured. We found that a unifying concept is found more easily and unifying concepts vary significantly less between participants when the words composing a micropoem are semantically related. Moreover these items are liked better and are understood more easily. Our study shows evidence for the assumed relationship between building spontaneous associations, forming mental imagery, and understanding and appreciation of poetic texts. In addition, we introduced a new method well-suited to manipulate backgrounding features independently of foregrounding features which allows to disentangle the effects of both on poetry reception.

Creativity Is Optimal Novelty and Maximal Positive Affect: A New Definition Based on the Spreading Activation Model

Creativity is commonly defined as a process that leads to a novel and useful outcome (an idea, product, or expression). However, two dilemmas about this definition remain unresolved: (1) A strict application of usefulness is difficult to apply to artistic works: who decides what artwork is useful, and how it is useful? (2) The implied boundary conditions of novelty are problematic: The default perspective is that novelty has a monotonic increasing relationship with creativity, or it is categorical-i.e., novel or not. To address these dilemmas, this paper proposes a spreading activation model of creativity (SAMOC), a model built on a brain-architecture-inspired vast interconnected network of nodes, each node representing information, and assigned meanings through interaction with the environment. Nodes are linked to each other according to principles of temporal contiguity (linking objects/events in time) and similarity (linking objects/events by shared features). A node activated by attention spreads through the network through previously linked nodes. Nodes that are well connected activate each other easily, while those that are weakly connected do not. Net total activation corresponds to positive affect (e.g., pleasure), and this is proposed as an essential criteria for a creative work of art, instead of usefulness. SAMOC also predicts that creativity will be optimized at an intermediate, not extreme, level of novelty. Too much activation will occur with the activation of preexisting ideas (hence reproduction rather than creativity), and too much novelty will not produce spread of activation. The two functions (spreading activation and the novelty curve) are superposed to demonstrate this optimal novelty hypothesis. Early evidence of the hypothesis comes from the data that some great works of art were critically rejected at premiers (suggesting excessive novelty), but after sufficient repetition (and therefore linking) became suitably associated and commenced generating activation. The hypothesis has important implications for future empirical research programs on creativity, and for the definition of creativity itself.

Neuroimaging of valence decisions in children and adults

To date, the neural underpinnings of affective components in language processing in children remain largely unknown. To fill this gap, the present study examined behavioural and neural correlates of children and adults performing the same auditory valence decision task with an event-related fMRI paradigm. Based on previous findings in adults, activations in anterior and posterior cingulate cortex, orbitofrontal cortex and left inferior frontal gyrus were expected for both positive and negative valence categories. Recent behavioural findings on valence decisions showed similar ratings and reaction time patterns in children and adults. This finding was successfully replicated in the present study. On a neural level, our analysis of affective language processing showed activations in regions associated with both semantic (superior and middle temporal and frontal) and affective (anterior and posterior cingulate, orbitofrontal and inferior frontal, insula and amygdala) processing. Neural activations in children and adults were systematically different in explicit affective word processing. In particular, adults showed a more distributed semantic network activation while children recruited additional subcortical structures.

Neural correlates of affective contributions to lexical decisions in children and adults

The goal of the present study was to investigate whether 6-9-year old children and adults show similar neural responses to affective words. An event-related neuroimaging paradigm was used in which both age cohorts performed the same auditory lexical decision task (LDT). The results show similarities in (auditory) lexico-semantic network activation as well as in areas associated with affective information. In both age cohorts' activations were stronger for positive than for negative words, thus exhibiting a positivity superiority effect. Children showed less activation in areas associated with affective information in response to all three valence categories than adults. Our results are discussed in the light of computational models of word recognition, and previous findings of affective contributions to LDT in adults.

Cloze enough? Hemodynamic effects of predictive processing during natural reading

Evidence accrues that readers form multiple hypotheses about upcoming words. The present study investigated the hemodynamic effects of predictive processing during natural reading by means of combining fMRI and eye movement recordings. In particular, we investigated the neural and behavioral correlates of precision-weighted prediction errors, which are thought to be indicative of subsequent belief updating. Participants silently read sentences in which we manipulated the cloze probability and the semantic congruency of the final word that served as an index for precision and prediction error respectively. With respect to the neural correlates, our findings indicate an enhanced activation within the left inferior frontal and middle temporal gyrus suggesting an effect of precision on prediction update in higher (lexico-)semantic levels. Despite being evident at the neural level, we did not observe any evidence that this mechanism resulted in disproportionate reading times on participants' eye movements. The results speak against discrete predictions, but favor the notion that multiple words are activated in parallel during reading.

Sentiment Analysis of Children and Youth Literature: Is There a Pollyanna Effect?

If the words of natural human language possess a universal positivity bias, as assumed by Boucher and Osgood’s (1969) famous Pollyanna hypothesis and computationally confirmed for large text corpora in several languages (Dodds et al., 2015), then children and youth literature (CYL) should also show a Pollyanna effect. Here we tested this prediction applying an unsupervised vector space model-based sentiment analysis tool called SentiArt (Jacobs, 2019) to two CYL corpora, one in English (372 books) and one in German (500 books). Pitching our analysis at the sentence level, and assessing semantic as well as lexico-grammatical information, both corpora show the Pollyanna effect and thus add further evidence to the universality hypothesis. The results of our multivariate sentiment analyses provide interesting testable predictions for future scientific studies of literature.

The Emotional Verbal Fluency Task: A Close Examination of Verbal Productivity and Lexical-Semantic Properties

Purpose Emotional verbal fluency (Emo-VF) has the potential to expand neuropsychological assessment by providing information about affective memory retrieval. The usability of Emo-VF is limited, however, by significant variations in task administration and the lack of information about Emo-VF responses. This study investigated verbal productivity and the lexical-semantic properties of responses on positive and negative Emo-VF tasks. Comparing Emo-VF to non-Emo-VF tasks used regularly in neuropsychological assessment provided additional information about the basic characteristics of Emo-VF tasks. Method Twenty-five adult native speakers provided verbal responses to three Emo-VF ("happy," "sad," "negative emotions") and two non-Emo-VF categories ("animals," "things people do"). Verbal productivity was measured at the word and syllable levels. Multiple large-scale data corpora were used to estimate the lexical-semantic properties of the verbal responses. Results There was a robust positivity bias in verbal productivity within Emo-VF tasks. Emo-VF tasks tended to elicit longer words than "animals" and "things people do," which might impact the results of verbal productivity analyses, especially in comparisons with "things people do." Within Emo-VF tasks, negative Emo-VF elicited words from a wider range of valence than positive Emo-VF tasks. Similarities (e.g., word length and complexity) and differences (e.g., concreteness, age of acquisition) were found between positive and negative Emo-VF tasks. Conclusions The study provided information about the basic characteristics of Emo-VF tasks, which included evidence for a robust positivity bias, suggestions for analyses of verbal productivity (e.g., consideration of word length), and lexical-semantic properties associated with positive and negative Emo-VF tasks using corpora data.

Representation of associative and affective semantic similarity of abstract words in the lateral temporal perisylvian language regions

The examination of semantic cognition has traditionally identified word concreteness as well as valence as two of the principal dimensions in the representation of conceptual knowledge. More recently, corpus-based vector space models as well as graph-theoretical analysis of large-scale task-related behavioural responses have revolutionized our insight into how the meaning of words is structured. In this fMRI study, we apply representational similarity analysis to investigate the conceptual representation of abstract words. Brain activity patterns were related to a cued-association based graph as well as to a vector-based co-occurrence model of word meaning. Twenty-six subjects (19 females and 7 males) performed an overt repetition task during fMRI. First, we performed a searchlight classification procedure to identify regions where activity is discriminable between abstract and concrete words. These regions were left inferior frontal gyrus, the upper and lower bank of the superior temporal sulcus bilaterally, posterior middle temporal gyrus and left fusiform gyrus. Representational Similarity Analysis demonstrated that for abstract words, the similarity of activity patterns in the cortex surrounding the superior temporal sulcus bilaterally and in the left anterior superior temporal gyrus reflects the similarity in word meaning. These effects were strongest for semantic similarity derived from the cued association-based graph and for affective similarity derived from either of the two models. The latter effect was mainly driven by positive valence words. This research highlights the close neurobiological link between the information structure of abstract and affective word content and the similarity in activity pattern in the lateral and anterior temporal language system.

The SLS-Berlin: Validation of a German Computer-Based Screening Test to Measure Reading Proficiency in Early and Late Adulthood

Reading proficiency, i.e., successfully integrating early word-based information and utilizing this information in later processes of sentence and text comprehension, and its assessment is subject to extensive research. However, screening tests for German adults across the life span are basically non-existent. Therefore, the present article introduces a standardized computerized sentence-based screening measure for German adult readers to assess reading proficiency including norm data from 2,148 participants covering an age range from 16 to 88 years. The test was developed in accordance with the children's version of the Salzburger LeseScreening (SLS, Wimmer and Mayringer, 2014). The SLS-Berlin has a high reliability and can easily be implemented in any research setting using German language. We present a detailed description of the test and report the distribution of SLS-Berlin scores for the norm sample as well as for two subsamples of younger (below 60 years) and older adults (60 and older). For all three samples, we conducted regression analyses to investigate the relationship between sentence characteristics and SLS-Berlin scores. In a second validation study, SLS-Berlin scores were compared with two (pseudo)word reading tests, a test measuring attention and processing speed and eye-movements recorded during expository text reading. Our results confirm the SLS-Berlin's sensitivity to capture early word decoding and later text related comprehension processes. The test distinguished very well between skilled and less skilled readers and also within less skilled readers and is therefore a powerful and efficient screening test for German adults to assess interindividual levels of reading proficiency.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Steady state visual evoked potentials in reading aloud: Effects of lexicality, frequency and orthographic familiarity

The present study explored the possibility to use Steady-State Visual Evoked Potentials (SSVEPs) as a tool to investigate the core mechanisms in visual word recognition. In particular, we investigated three benchmark effects of reading aloud: lexicality (words vs. pseudowords), frequency (high-frequency vs. low-frequency words), and orthographic familiarity ('familiar' versus 'unfamiliar' pseudowords). We found that words and pseudowords elicited robust SSVEPs. Words showed larger SSVEPs than pseudowords and high-frequency words showed larger SSVEPs than low-frequency words. SSVEPs were not sensitive to orthographic familiarity. We further localized the neural generators of the SSVEP effects. The lexicality effect was located in areas associated with early level of visual processing, i.e. in the right occipital lobe and in the right precuneus. Pseudowords produced more activation than words in left sensorimotor areas, rolandic operculum, insula, supramarginal gyrus and in the right temporal gyrus. These areas are devoted to speech processing and/or spelling-to-sound conversion. The frequency effect involved the left temporal pole and orbitofrontal cortex, areas previously implicated in semantic processing and stimulus-response associations respectively, and the right postcentral and parietal inferior gyri, possibly indicating the involvement of the right attentional network.

A model-guided dissociation between subcortical and cortical contributions to word recognition

Neurocognitive studies of visual word recognition have provided information about brain activity correlated with orthographic processing. Some of these studies related the orthographic neighborhood density of letter strings to the amount of hypothetical global lexical activity (GLA) in the brain as simulated by computational models of word recognition. To further investigate this issue, we used GLA of words and nonwords from the multiple read-out model of visual word recognition (MROM) and related this activity to neural correlates of orthographic processing in the brain by using functional magnetic resonance imaging (fMRI). Words and nonwords elicited linear effects in the cortex with increasing BOLD responses for decreasing values of GLA. In addition, words showed increasing linear BOLD responses for increasing GLA values in subcortical regions comprising the hippocampus, globus pallidus and caudate nucleus. We propose that these regions are involved in the matching of orthographic input onto representations in long-term memory. The results speak to a potential involvement of the basal ganglia in visual word recognition with globus pallidus and caudate nucleus activity potentially reflecting maintenance of orthographic input in working memory supporting the matching of the input onto stored representations by selection of appropriate lexical candidates and the inhibition of orthographically similar but non-matching candidates.

tDCS Facilitation of Picture Naming: Item-Specific, Task General, or Neither?

The aim of the present study was to clarify the conditions under which anodal tDCS applied to left hemisphere language sites may facilitate picture naming latencies in healthy young adults. We built upon previous studies by directly testing for item-specific and generalized effects of tDCS through manipulation of item-familiarization and through testing for both online and offline effects of stimulation, in the same paradigm. In addition, we tested for the robustness of these effects by comparing two left hemisphere sites critical for lexical retrieval. Twenty-eight healthy young adults completed two testing sessions receiving either anodal (1.5 mA, 20 min) or sham stimulation (1.5 mA, 30 s) in each session. Half of the participants received tDCS over the left inferior frontal region and the other half over the left posterior superior temporal region. All participants were asked to a name a set of pictures and their response latencies were compared at three time points (before, during, and after the end of stimulation). The stimulus set was constructed so that some items were presented at all time points, some before and after stimulation, and some during stimulation only. A parsimonious linear mixed effects model (LMM) revealed robust repetition priming effects as latencies were reliably faster for previously named items in all conditions. However, active tDCS did not produce any additional facilitation in relation to sham, and even led to slower performance in the IFG group when the stimulated items differed from those tested at baseline and post-test. Our findings add to the present debate about the efficacy of single-session tDCS for modulation of lexical retrieval in healthy young adults. We conclude that future research should take a more systematic, step-wise approach to the application of tDCS to the study of language and that more sensitive experimental paradigms, which include a training element, are more adapted to the study of cognitive processes in populations with optimal levels of cortical excitability.

The Fictive Brain: Neurocognitive Correlates of Engagement in Literature

Fiction is vital to our being. Many people enjoy engaging with fiction every day. Here we focus on literary reading as 1 instance of fiction consumption from a cognitive neuroscience perspective. The brain processes which play a role in the mental construction of fiction worlds and the related engagement with fictional characters, remain largely unknown. The authors discuss the neurocognitive poetics model ( Jacobs, 2015a ) of literary reading specifying the likely neuronal correlates of several key processes in literary reading, namely inference and situation model building, immersion, mental simulation and imagery, figurative language and style, and the issue of distinguishing fact from fiction. An overview of recent work on these key processes is followed by a discussion of methodological challenges in studying the brain bases of fiction processing.

Quantifying the Beauty of Words: A Neurocognitive Poetics Perspective

In this paper I would like to pave the ground for future studies in Computational Stylistics and (Neuro-)Cognitive Poetics by describing procedures for predicting the subjective beauty of words. A set of eight tentative word features is computed via Quantitative Narrative Analysis (QNA) and a novel metric for quantifying word beauty, the aesthetic potential is proposed. Application of machine learning algorithms fed with this QNA data shows that a classifier of the decision tree family excellently learns to split words into beautiful vs. ugly ones. The results shed light on surface and semantic features theoretically relevant for affective-aesthetic processes in literary reading and generate quantitative predictions for neuroaesthetic studies of verbal materials.

Neurofunctionally dissecting the reading system in children

The reading system can be broken down into four basic subcomponents in charge of prelexical, orthographic, phonological, and lexico-semantic processes. These processes need to jointly work together to become a fluent and efficient reader. Using functional magnetic resonance imaging (fMRI), we systematically analyzed differences in neural activation patterns of these four basic subcomponents in children (N=41, 9-13 years) using tasks specifically tapping each component (letter identification, orthographic decision, phonological decision, and semantic categorization). Regions of interest (ROI) were selected based on a meta-analysis of child reading and included the left ventral occipito-temporal cortex (vOT), left posterior parietal cortex (PPC), left inferior frontal gyrus (IFG), and bilateral supplementary motor area (SMA). Compared to a visual baseline task, enhanced activation in vOT and IFG was observed for all tasks with very little differences between tasks. Activity in the dorsal PPC system was confined to prelexical and phonological processing. Activity in the SMA was found in orthographic, phonological, and lexico-semantic tasks. Our results are consistent with the idea of an early engagement of the vOT accompanied by executive control functions in the frontal system, including the bilateral SMA.

Auditory attention enhances processing of positive and negative words in inferior and superior prefrontal cortex

Visually presented emotional words are processed preferentially and effects of emotional content are similar to those of explicit attention deployment in that both amplify visual processing. However, auditory processing of emotional words is less well characterized and interactions between emotional content and task-induced attention have not been fully understood. Here, we investigate auditory processing of emotional words, focussing on how auditory attention to positive and negative words impacts their cerebral processing. A Functional magnetic resonance imaging (fMRI) study manipulating word valence and attention allocation was performed. Participants heard negative, positive and neutral words to which they either listened passively or attended by counting negative or positive words, respectively. Regardless of valence, active processing compared to passive listening increased activity in primary auditory cortex, left intraparietal sulcus, and right superior frontal gyrus (SFG). The attended valence elicited stronger activity in left inferior frontal gyrus (IFG) and left SFG, in line with these regions' role in semantic retrieval and evaluative processing. No evidence for valence-specific attentional modulation in auditory regions or distinct valence-specific regional activations (i.e., negative > positive or positive > negative) was obtained. Thus, allocation of auditory attention to positive and negative words can substantially increase their processing in higher-order language and evaluative brain areas without modulating early stages of auditory processing. Inferior and superior frontal brain structures mediate interactions between emotional content, attention, and working memory when prosodically neutral speech is processed.

Electrophysiological correlates of the drift diffusion model in visual word recognition

This study was designed to explore the electrophysiological correlates of the diffusion models drift rate parameter in cognitive decision making. Eighty-two participants completed a lexical decision task while their stimulus-dependent event-related potentials (ERP) and theta frequency band power were measured. A mass univariate approach was applied to examine the timeline at which correlations were evident. Individual differences in drift rate parameter and condition-wise within-subject differences in drift rates for word emotionality and item repetition were found to be related to amplitude differences in the late positive complex (LPC). No relations to theta frequency band power changes were obtained. The drift rate parameter captures information accumulation of noisy evidence, while LPC amplitudes are discussed to reflect the strength of a memory trace. While these results point to a common underlying cognitive mechanism to explain drift rates and LPC modulation, they also provide a new angle on the timeline of visual word processing following word identification. Further confirmations of the results are needed to approve the LPC as neurophysiological marker of information accumulation. Hum Brain Mapp 38:5616-5627, 2017. © 2017 Wiley Periodicals, Inc.

Age-Dependent Positivity-Bias in Children's Processing of Emotion Terms

Emotions play an important role in human communication, and the daily-life interactions of young children often include situations that require the verbalization of emotional states with verbal means, e.g., with emotion terms. Through them, one can express own emotional states and those of others. Thus, the acquisition of emotion terms allows children to participate more intensively in social contexts - a basic requirement for learning new words and for elaborating socio-emotional skills. However, little is known about how children acquire and process this specific word category, which is positioned between concrete and abstract words. In particular, the influence of valence on emotion word processing during childhood has not been sufficiently investigated. Previous research points to an advantage of positive words over negative and neutral words in word processing. While previous studies found valence effects to be influenced by factors such as arousal, frequency, concreteness, and task, it is still unclear if and how valence effects are also modified by age. The present study compares the performance of children aged from 5 to 12 years and adults in two experimental tasks: lexical decision (word or pseudoword) and emotional categorization (positive or negative). Stimuli consisted of 48 German emotion terms (24 positive and 24 negative) matched for arousal, concreteness, age of acquisition, word class, word length, morphological complexity, frequency, and neighborhood density. Results from both tasks reveal two developmental trends: First, with increasing age children responded faster and more correctly, suggesting that emotion vocabulary gradually becomes more stable and differentiated during middle childhood. Second, the influence of valence varied with age: younger children (5- and 6-year-olds) showed significantly higher performance levels for positive emotion terms compared to negative emotion terms, whereas older children and adults did not. This age-related valence effect in emotion word processing will be discussed with respect to linguistic and methodological aspects.

Fast Brain Plasticity during Word Learning in Musically-Trained Children

Children learn new words every day and this ability requires auditory perception, phoneme discrimination, attention, associative learning and semantic memory. Based on previous results showing that some of these functions are enhanced by music training, we investigated learning of novel words through picture-word associations in musically-trained and control children (8-12 year-old) to determine whether music training would positively influence word learning. Results showed that musically-trained children outperformed controls in a learning paradigm that included picture-sound matching and semantic associations. Moreover, the differences between unexpected and expected learned words, as reflected by the N200 and N400 effects, were larger in children with music training compared to controls after only 3 min of learning the meaning of novel words. In line with previous results in adults, these findings clearly demonstrate a correlation between music training and better word learning. It is argued that these benefits reflect both bottom-up and top-down influences. The present learning paradigm might provide a useful dynamic diagnostic tool to determine which perceptive and cognitive functions are impaired in children with learning difficulties.

If You Don't Have Valence, Ask Your Neighbor: Evaluation of Neutral Words as a Function of Affective Semantic Associates

How do humans perform difficult forced-choice evaluations, e.g., of words that have been previously rated as being neutral? Here we tested the hypothesis that in this case, the valence of semantic associates is of significant influence. From corpus based co-occurrence statistics as a measure of association strength we computed individual neighborhoods for single neutral words comprised of the 10 words with the largest association strength. We then selected neutral words according to the valence of the associated words included in the neighborhoods, which were either mostly positive, mostly negative, mostly neutral or mixed positive and negative, and tested them using a valence decision task (VDT). The data showed that the valence of semantic neighbors can predict valence judgments to neutral words. However, all but the positive neighborhood items revealed a high tendency to elicit negative responses. For the positive and negative neighborhood categories responses congruent with the neighborhood's valence were faster than incongruent responses. We interpret this effect as a semantic network process that supports the evaluation of neutral words by assessing the valence of the associative semantic neighborhood. In this perspective, valence is considered a semantic super-feature, at least partially represented in associative activation patterns of semantic networks.

Drifting through Basic Subprocesses of Reading: A Hierarchical Diffusion Model Analysis of Age Effects on Visual Word Recognition

Reading is one of the most popular leisure activities and it is routinely performed by most individuals even in old age. Successful reading enables older people to master and actively participate in everyday life and maintain functional independence. Yet, reading comprises a multitude of subprocesses and it is undoubtedly one of the most complex accomplishments of the human brain. Not surprisingly, findings of age-related effects on word recognition and reading have been partly contradictory and are often confined to only one of four central reading subprocesses, i.e., sublexical, orthographic, phonological and lexico-semantic processing. The aim of the present study was therefore to systematically investigate the impact of age on each of these subprocesses. A total of 1,807 participants (young, N = 384; old, N = 1,423) performed four decision tasks specifically designed to tap one of the subprocesses. To account for the behavioral heterogeneity in older adults, this subsample was split into high and low performing readers. Data were analyzed using a hierarchical diffusion modeling approach, which provides more information than standard response time/accuracy analyses. Taking into account incorrect and correct response times, their distributions and accuracy data, hierarchical diffusion modeling allowed us to differentiate between age-related changes in decision threshold, non-decision time and the speed of information uptake. We observed longer non-decision times for older adults and a more conservative decision threshold. More importantly, high-performing older readers outperformed younger adults at the speed of information uptake in orthographic and lexico-semantic processing, whereas a general age-disadvantage was observed at the sublexical and phonological levels. Low-performing older readers were slowest in information uptake in all four subprocesses. Discussing these results in terms of computational models of word recognition, we propose age-related disadvantages for older readers to be caused by inefficiencies in temporal sampling and activation and/or inhibition processes.

On Elementary Affective Decisions: To Like Or Not to Like, That Is the Question

Perhaps the most ubiquitous and basic affective decision of daily life is deciding whether we like or dislike something/somebody, or, in terms of psychological emotion theories, whether the object/subject has positive or negative valence. Indeed, people constantly make such liking decisions within a glimpse and, importantly, often without expecting any obvious benefit or knowing the exact reasons for their judgment. In this paper, we review research on such elementary affective decisions (EADs) that entail no direct overt reward with a special focus on Neurocognitive Poetics and discuss methods and models for investigating the neuronal and cognitive-affective bases of EADs to verbal materials with differing degrees of complexity. In line with evolutionary and appraisal theories of (aesthetic) emotions and data from recent neurocognitive studies, the results of a decision tree modeling approach simulating EADs to single words suggest that a main driving force behind EADs is the extent to which such high-dimensional stimuli are associated with the “basic” emotions joy/happiness and disgust.

Mixing positive and negative valence: Affective-semantic integration of bivalent words

Single words have affective and aesthetic properties that influence their processing. Here we investigated the processing of a special case of word stimuli that are extremely difficult to evaluate, bivalent noun-noun-compounds (NNCs), i.e. novel words that mix a positive and negative noun, e.g. ‘Bombensex’ (bomb-sex). In a functional magnetic resonance imaging (fMRI) experiment we compared their processing with easier-to-evaluate non-bivalent NNCs in a valence decision task (VDT). Bivalent NNCs produced longer reaction times and elicited greater activation in the left inferior frontal gyrus (LIFG) than non-bivalent words, especially in contrast to words of negative valence. We attribute this effect to a LIFG-grounded process of semantic integration that requires greater effort for processing converse information, supporting the notion of a valence representation based on associations in semantic networks.

The Berlin Affective Word List for Children (kidBAWL): Exploring Processing of Affective Lexical Semantics in the Visual and Auditory Modalities

While research on affective word processing in adults witnesses increasing interest, the present paper looks at another group of participants that have been neglected so far: pupils (age range: 6-12 years). Introducing a variant of the Berlin Affective Wordlist (BAWL) especially adapted for children of that age group, the "kidBAWL," we examined to what extent pupils process affective lexical semantics similarly to adults. In three experiments using rating and valence decision tasks in both the visual and auditory modality, it was established that children show the two ubiquitous phenomena observed in adults with emotional word material: the asymmetric U-shaped function relating valence to arousal ratings, and the inversely U-shaped function relating response times to valence decision latencies. The results for both modalities show large structural similarities between pupil and adult data (taken from previous studies) indicating that in the present age range, the affective lexicon and the dynamic interplay between language and emotion is already well-developed. Differential effects show that younger children tend to choose less extreme ratings than older children and that rating latencies decrease with age. Overall, our study should help to develop more realistic models of word recognition and reading that include affective processes and offer a methodology for exploring the roots of pleasant literary experiences and ludic reading.

Slower Perception Followed by Faster Lexical Decision in Longer Words: A Diffusion Model Analysis

Effects of stimulus length on reaction times (RTs) in the lexical decision task are the topic of extensive research. While slower RTs are consistently found for longer pseudo-words, a finding coined the word length effect (WLE), some studies found no effects for words, and yet others reported faster RTs for longer words. Moreover, the WLE depends on the orthographic transparency of a language, with larger effects in more transparent orthographies. Here we investigate processes underlying the WLE in lexical decision in German-English bilinguals using a diffusion model (DM) analysis, which we compared to a linear regression approach. In the DM analysis, RT-accuracy distributions are characterized using parameters that reflect latent sub-processes, in particular evidence accumulation and decision-independent perceptual encoding, instead of typical parameters such as mean RT and accuracy. The regression approach showed a decrease in RTs with length for pseudo-words, but no length effect for words. However, DM analysis revealed that the null effect for words resulted from opposing effects of length on perceptual encoding and rate of evidence accumulation. Perceptual encoding times increased with length for words and pseudo-words, whereas the rate of evidence accumulation increased with length for real words but decreased for pseudo-words. A comparison between DM parameters in German and English suggested that orthographic transparency affects perceptual encoding, whereas effects of length on evidence accumulation are likely to reflect contextual information and the increase in available perceptual evidence with length. These opposing effects may account for the inconsistent findings on WLEs.

Preservation of Person-Specific Semantic Knowledge in Semantic Dementia: Does Direct Personal Experience Have a Specific Role?

Semantic dementia patients seem to have better knowledge of information linked to the self. More specifically, despite having severe semantic impairment, these patients show that they have more general information about the people they know personally by direct experience than they do about other individuals they know indirectly. However, the role of direct personal experience remains debated because of confounding factors such as frequency, recency of exposure, and affective relevance. We performed an exploratory study comparing the performance of five semantic dementia patients with that of 10 matched healthy controls on the recognition (familiarity judgment) and identification (biographic information recall) of personally familiar names vs. famous names. As expected, intergroup comparisons indicated a semantic breakdown in semantic dementia patients as compared with healthy controls. Moreover, unlike healthy controls, the semantic dementia patients recognized and identified personally familiar names better than they did famous names. This pattern of results suggests that direct personal experience indeed plays a specific role in the relative preservation of person-specific semantic meaning in semantic dementia. We discuss the role of direct personal experience on the preservation of semantic knowledge and the potential neurophysiological mechanisms underlying these processes.

The emotion potential of simple sentences: additive or interactive effects of nouns and adjectives?

The vast majority of studies on affective processes in reading focus on single words. The most robust finding is a processing advantage for positively valenced words, which has been replicated in the rare studies investigating effects of affective features of words during sentence or story comprehension. Here we were interested in how the different valences of words in a sentence influence its processing and supralexical affective evaluation. Using a sentence verification task we investigated how comprehension of simple declarative sentences containing a noun and an adjective depends on the valences of both words. The results are in line with the assumed general processing advantage for positive words. We also observed a clear interaction effect, as can be expected from the affective priming literature: sentences with emotionally congruent words (e.g., The grandpa is clever) were verified faster than sentences containing emotionally incongruent words (e.g., The grandpa is lonely). The priming effect was most prominent for sentences with positive words suggesting that both, early processing as well as later meaning integration and situation model construction, is modulated by affective processing. In a second rating task we investigated how the emotion potential of supralexical units depends on word valence. The simplest hypothesis predicts that the supralexical affective structure is a linear combination of the valences of the nouns and adjectives (Bestgen, 1994). Overall, our results do not support this: The observed clear interaction effect on ratings indicate that especially negative adjectives dominated supralexical evaluation, i.e., a sort of negativity bias in sentence evaluation. Future models of sentence processing thus should take interactive affective effects into account.

Many neighbors are not silent. fMRI evidence for global lexical activity in visual word recognition

Many neurocognitive studies investigated the neural correlates of visual word recognition, some of which manipulated the orthographic neighborhood density of words and nonwords believed to influence the activation of orthographically similar representations in a hypothetical mental lexicon. Previous neuroimaging research failed to find evidence for such global lexical activity associated with neighborhood density. Rather, effects were interpreted to reflect semantic or domain general processing. The present fMRI study revealed effects of lexicality, orthographic neighborhood density and a lexicality by orthographic neighborhood density interaction in a silent reading task. For the first time we found greater activity for words and nonwords with a high number of neighbors. We propose that this activity in the dorsomedial prefrontal cortex reflects activation of orthographically similar codes in verbal working memory thus providing evidence for global lexical activity as the basis of the neighborhood density effect. The interaction of lexicality by neighborhood density in the ventromedial prefrontal cortex showed lower activity in response to words with a high number compared to nonwords with a high number of neighbors. In the light of these results the facilitatory effect for words and inhibitory effect for nonwords with many neighbors observed in previous studies can be understood as being due to the operation of a fast-guess mechanism for words and a temporal deadline mechanism for nonwords as predicted by models of visual word recognition. Furthermore, we propose that the lexicality effect with higher activity for words compared to nonwords in inferior parietal and middle temporal cortex reflects the operation of an identification mechanism based on local lexico-semantic activity.

10 years of BAWLing into affective and aesthetic processes in reading: what are the echoes?

Reading is not only "cold" information processing, but involves affective and aesthetic processes that go far beyond what current models of word recognition, sentence processing, or text comprehension can explain. To investigate such "hot" reading processes, standardized instruments that quantify both psycholinguistic and emotional variables at the sublexical, lexical, inter-, and supralexical levels (e.g., phonological iconicity, word valence, arousal-span, or passage suspense) are necessary. One such instrument, the Berlin Affective Word List (BAWL) has been used in over 50 published studies demonstrating effects of lexical emotional variables on all relevant processing levels (experiential, behavioral, neuronal). In this paper, we first present new data from several BAWL studies. Together, these studies examine various views on affective effects in reading arising from dimensional (e.g., valence) and discrete emotion features (e.g., happiness), or embodied cognition features like smelling. Second, we extend our investigation of the complex issue of affective word processing to words characterized by a mixture of affects. These words entail positive and negative valence, and/or features making them beautiful or ugly. Finally, we discuss tentative neurocognitive models of affective word processing in the light of the present results, raising new issues for future studies.

Neurocognitive poetics: methods and models for investigating the neuronal and cognitive-affective bases of literature reception

A long tradition of research including classical rhetoric, esthetics and poetics theory, formalism and structuralism, as well as current perspectives in (neuro)cognitive poetics has investigated structural and functional aspects of literature reception. Despite a wealth of literature published in specialized journals like Poetics, however, still little is known about how the brain processes and creates literary and poetic texts. Still, such stimulus material might be suited better than other genres for demonstrating the complexities with which our brain constructs the world in and around us, because it unifies thought and language, music and imagery in a clear, manageable way, most often with play, pleasure, and emotion (Schrott and Jacobs, 2011). In this paper, I discuss methods and models for investigating the neuronal and cognitive-affective bases of literary reading together with pertinent results from studies on poetics, text processing, emotion, or neuroaesthetics, and outline current challenges and future perspectives.

The neural bases of the pseudohomophone effect: Phonological constraints on lexico-semantic access in reading

We investigated phonological processing in normal readers to answer the question to what extent phonological recoding is active during silent reading and if or how it guides lexico-semantic access. We addressed this issue by looking at pseudohomophone and baseword frequency effects in lexical decisions with event-related functional magnetic resonance imaging (fMRI). The results revealed greater activation in response to pseudohomophones than for well-controlled pseudowords in the left inferior/superior frontal and middle temporal cortex, left insula, and left superior parietal lobule. Furthermore, we observed a baseword frequency effect for pseudohomophones (e.g., FEAL) but not for pseudowords (e.g., FEEP). This baseword frequency effect was qualified by activation differences in bilateral angular and left supramarginal, and bilateral middle temporal gyri for pseudohomophones with low- compared to high-frequency basewords. We propose that lexical decisions to pseudohomophones involves phonology-driven lexico-semantic activation of their basewords and that this is converging neuroimaging evidence for automatically activated phonological representations during silent reading in experienced readers.

Take a stand on understanding: electrophysiological evidence for stem access in German complex verbs

The lexical representation of complex words in Indo-European languages is generally assumed to depend on semantic compositionality. This study investigated whether semantically compositional and noncompositional derivations are accessed via their constituent units or as whole words. In an overt visual priming experiment (300 ms stimulus onset asynchrony, SOA), event-related potentials (ERPs) were recorded for verbs (e.g., ziehen, “pull”) that were preceded by purely semantically related verbs (e.g., zerren, “drag”), by morphologically related and semantically compositional verbs (e.g., zuziehen, “pull together”), by morphologically related and semantically noncompositional verbs (e.g., erziehen, “educate”), by orthographically similar verbs (e.g., zielen, “aim”), or by unrelated verbs (e.g., tarnen, “mask”). Compared to the unrelated condition, which evoked an N400 effect with the largest amplitude at centro-parietal recording sites, the N400 was reduced in all other conditions. The rank order of N400 amplitudes turned out as follows: morphologically related and semantically compositional ≈ morphologically related and semantically noncompositional < purely semantically related < orthographically similar < unrelated. Surprisingly, morphologically related primes produced similar N400 modulations—irrespective of their semantic compositionality. The control conditions with orthographic similarity confirmed that these morphological effects were not the result of a simple form overlap between primes and targets. Our findings suggest that the lexical representation of German complex verbs refers to their base form, regardless of meaning compositionality. Theories of the lexical representation of German words need to incorporate this aspect of language processing in German.

Inborn and experience-dependent models of categorical brain organization. A position paper

The present review aims to summarize the debate in contemporary neuroscience between inborn and experience-dependent models of conceptual representations that goes back to the description of category-specific semantic disorders for biological and artifact categories. Experience-dependent models suggest that categorical disorders are the by-product of the differential weighting of different sources of knowledge in the representation of biological and artifact categories. These models maintain that semantic disorders are not really category-specific, because they do not respect the boundaries between different categories. They also argue that the brain structures which are disrupted in a given type of category-specific semantic disorder should correspond to the areas of convergence of the sensory-motor information which play a major role in the construction of that category. Furthermore, they provide a simple interpretation of gender-related categorical effects and are supported by studies assessing the importance of prior experience in the cortical representation of objects On the other hand, inborn models maintain that category-specific semantic disorders reflect the disruption of innate brain networks, which are shaped by natural selection to allow rapid identification of objects that are very relevant for survival. From the empirical point of view, these models are mainly supported by observations of blind subjects, which suggest that visual experience is not necessary for the emergence of category-specificity in the ventral stream of visual processing. The weight of the data supporting experience-dependent and inborn models is thoroughly discussed, stressing the fact observations made in blind subjects are still the subject of intense debate. It is concluded that at the present state of knowledge it is not possible to choose between experience-dependent and inborn models of conceptual representations.