The Impact of Inserting an Additional Mental Process

Updating of information in working memory: Time course and consequences

Working memory updating is the process that replaces outdated content in working memory by new content. This requires removing outdated information and encoding new information. It is still unclear whether removal and encoding run sequentially or simultaneously. We explored this question in two experiments investigating the time course of removal and encoding and their consequences for the functioning of working memory. The updating task we used involved three phases: the initial encoding, the processing, and the retrieval phase. Across four conditions, we manipulated whether the processing phase involved encoding, removal, neither, or both (i.e., updating). In Experiment 1, processing time was self-paced, and we measured processing times in each condition. In Experiment 2, we measured accuracy as a function of available processing time. After the processing, participants were asked to recall the final item for each position in the retrieval phase. In combination, the results of the two experiments show that the time required for updating was shorter than the sum of encoding and removal time. Moreover, it was nearly the same as the time taken for either the encoding or removal process, indicating that encoding and removal are concurrent processes during updating. Additionally, we analyzed the proportion of correct responses and of different error types with a memory measurement model to investigate the effects of encoding and removal for information held in working memory. The analysis revealed that removal involves unbinding the outdated information from its context. However, despite the weakened bindings of information to its initial context, the outdated information still remains activated in working memory. Other information held in working memory benefitted little from removal of outdated information.

Word Type and Frequency Effects on Lexical Decisions Are Process-dependent and Start Early

When encountering letter strings, we rapidly determine whether they are words. The speed of such lexical decisions (LDs) is affected by word frequency. Apart from influencing late, decision-related, processing stages, frequency has also been shown to affect very early stages, and even the processing of nonwords. We developed a detailed account of the different frequency effects involved in LDs by (1) dividing LDs into processing stages using a combination of hidden semi-Markov models and multivariate pattern analysis applied to EEG data and (2) using generalized additive mixed models to investigate how the effect of continuous word and nonword frequency differs between these stages. We discovered six stages shared between word types, with the fifth stage consisting of two substages for pseudowords only. In the earliest stages, visual processing was completed faster for frequent words, but took longer for word-like nonwords. Later stages involved an orthographic familiarity assessment followed by an elaborate decision process, both affected differently by frequency. We therefore conclude that frequency indeed affects all processes involved in LDs and that the magnitude and direction of these effects differ both by process and word type.

Testing two-step models of negative quantification using a novel machine learning analysis of EEG

The sentences "More than half of the students passed the exam" and "Fewer than half of the students failed the exam" describe the same set of situations, and yet the former results in shorter reaction times in verification tasks. The two-step model explains this result by postulating that negative quantifiers contain hidden negation, which involves an extra processing stage. To test this theory, we applied a novel EEG analysis technique focused on detecting cognitive stages (HsMM-MVPA) to data from a picture-sentence verification task. We estimated the number of processing stages during reading and verification of quantified sentences (e.g. "Fewer than half of the dots are blue") that followed the presentation of pictures containing coloured geometric shapes. We did not find evidence for an extra step during the verification of sentences with fewer than half. We provide an alternative interpretation of our results in line with an expectation-based pragmatic account.

Cognitive processing stages in mental rotation - How can cognitive modelling inform HsMM-EEG models?

The aspiration for insight into human cognitive processing has traditionally driven research in cognitive science. With methods such as the Hidden semi-Markov Model-Electroencephalography (HsMM-EEG) method, new approaches have been developed that help to understand the temporal structure of cognition by identifying temporally discrete processing stages. However, it remains challenging to assign concrete functional contributions by specific processing stages to the overall cognitive process. In this paper, we address this challenge by linking HsMM-EEG3 with cognitive modelling, with the aim of further validating the HsMM-EEG3 method and demonstrating the potential of cognitive models to facilitate functional interpretation of processing stages. For this purpose, we applied HsMM-EEG3 to data from a mental rotation task and developed an ACT-R cognitive model that is able to closely replicate human performance in this task. Applying HsMM-EEG3 to the mental rotation experiment data revealed a strong likelihood for 6 distinct stages of cognitive processing during trials, with an additional stage for non-rotated conditions. The cognitive model predicted intra-trial mental activity patterns that project well onto the processing stages, while explaining the additional stage as a marker of non-spatial shortcut use. Thereby, this combined methodology provided substantially more information than either method by itself and suggests conclusions for cognitive processing in general.

Thalamic bursts modulate cortical synchrony locally to switch between states of global functional connectivity in a cognitive task

Performing a cognitive task requires going through a sequence of functionally diverse stages. Although it is typically assumed that these stages are characterized by distinct states of cortical synchrony that are triggered by sub-cortical events, little reported evidence supports this hypothesis. To test this hypothesis, we first identified cognitive stages in single-trial MEG data of an associative recognition task, showing with a novel method that each stage begins with local modulations of synchrony followed by a state of directed functional connectivity. Second, we developed the first whole-brain model that can simulate cortical synchrony throughout a task. The model suggests that the observed synchrony is caused by thalamocortical bursts at the onset of each stage, targeted at cortical synapses and interacting with the structural anatomical connectivity. These findings confirm that cognitive stages are defined by distinct states of cortical synchrony and explains the network-level mechanisms necessary for reaching stage-dependent synchrony states.

Discovering the brain stages of lexical decision: Behavioral effects originate from a single neural decision process

Lexical decision (LD) - judging whether a sequence of letters constitutes a word - has been widely investigated. In a typical lexical decision task (LDT), participants are asked to respond whether a sequence of letters is an actual word or a nonword. Although behavioral differences between types of words/nonwords have been robustly detected in LDT, there is an ongoing discussion about the exact cognitive processes that underlie the word identification process in this task. To obtain data-driven evidence on the underlying processes, we recorded electroencephalographic (EEG) data and applied a novel machine-learning method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA). In the current study, participants performed an LDT in which we varied the frequency of words (high, low frequency) and "wordlikeness" of non-words (pseudowords, random non-words). The results revealed that models with six processing stages accounted best for the data in all conditions. While most stages were shared, Stage 5 differed between conditions. Together, these results indicate that the differences in word frequency and lexicality effects are driven by a single cognitive processing stage. Based on its latency and topology, we interpret this stage as a Decision process during which participants discriminate between words and nonwords using activated lexical information.

A beginning quantitative taxonomy of cognitive activation systems and application to continuous flow processes

Much progress has been made in the investigation of perceptual, cognitive, and action mechanisms under the assumption that when one subprocess precedes another, the first one starts and finishes before the other begins. We call such processes "Dondersian" after the Dutch physiologist who first formulated this concept. Serial systems obey this precept (e.g., Townsend, 1974). However, most dynamic systems in nature do not: instead, each subprocess communicates its state to its immediate successors continuously. Although the mathematics for physical systems has received extensive treatment over the last three centuries, applications to human cognition have been exiguous. Therefore, the pioneering papers by Charles Eriksen and colleagues on continuous flow dynamics (e.g., Eriksen & Schulz, Perception & Psychophysics, 25, 249-263, 1979; Coles et al.,, Journal of Experimental Psychology: Human Perception and Performance, 11(5), 529, 1985) must be viewed as truly revolutionary. Surprisingly, there has been almost no advancement on this front since. With the goal of bringing this theme back into the scientific consciousness and extending and deepening our understanding of such systems, we develop a taxonomy that emphasizes the fundamental characteristics of continuous flow dynamics. Subsequently, we complexify the treated systems in such a way as to illustrate the popular cascade model (Ashby, Psychological Review, 89, 599-607, 1982; McClelland, Psychological Review, 86, 287-330, 1979) and use it to simulate the classic findings of Eriksen and colleagues (Eriksen & Hoffman, Perception & Psychophysics, 12(2), 201-204, 1972).

EEG-based Identification of Evidence Accumulation Stages in Decision-Making

Dating back to the 19th century, the discovery of processing stages has been of great interest to researchers in cognitive science. The goal of this paper is to demonstrate the validity of a recently developed method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA), for discovering stages directly from EEG data, in contrast to classical reaction-time-based methods. To test the validity of stages discovered with the HsMM-MVPA method, we applied it to two relatively simple tasks where the interpretation of processing stages is straightforward. In these visual discrimination EEG data experiments, perceptual processing and decision difficulty were manipulated. The HsMM-MVPA revealed that participants progressed through five cognitive processing stages while performing these tasks. The brain activation of one of those stages was dependent on perceptual processing, whereas the brain activation and the duration of two other stages were dependent on decision difficulty. In addition, evidence accumulation models (EAMs) were used to assess to what extent the results of HsMM-MVPA are comparable to standard reaction-time-based methods. Consistent with the HsMM-MVPA results, EAMs showed that nondecision time varied with perceptual difficulty and drift rate varied with decision difficulty. Moreover, nondecision and decision time of the EAMs correlated highly with the first two and last three stages of the HsMM-MVPA, respectively, indicating that the HsMM-MVPA gives a more detailed description of stages discovered with this more classical method. The results demonstrate that cognitive stages can be robustly inferred with the HsMM-MVPA.