A multisensory perspective of working memory

Multisensory integration processing during olfactory-visual stimulation-An fMRI graph theoretical network analysis

In this study, we aimed to understand how whole-brain neural networks compute sensory information integration based on the olfactory and visual system. Task-related functional magnetic resonance imaging (fMRI) data was obtained during unimodal and bimodal sensory stimulation. Based on the identification of multisensory integration processing (MIP) specific hub-like network nodes analyzed with network-based statistics using region-of-interest based connectivity matrices, we conclude the following brain areas to be important for processing the presented bimodal sensory information: right precuneus connected contralaterally to the supramarginal gyrus for memory-related imagery and phonology retrieval, and the left middle occipital gyrus connected ipsilaterally to the inferior frontal gyrus via the inferior fronto-occipital fasciculus including functional aspects of working memory. Applied graph theory for quantification of the resulting complex network topologies indicates a significantly increased global efficiency and clustering coefficient in networks including aspects of MIP reflecting a simultaneous better integration and segregation. Graph theoretical analysis of positive and negative network correlations allowing for inferences about excitatory and inhibitory network architectures revealed-not significant, but very consistent-that MIP-specific neural networks are dominated by inhibitory relationships between brain regions involved in stimulus processing.

Reconceptualizing delirium as a disorder of complex system failure

Delirium is conceptually elusive and falls outside of conventional biomedical models. Positivist theoretical paradigms of single linear causality are therefore insufficient to provide mechanistic enlightenment. Delirium does, however, share parallels with features of failure within a complex system. Lessons from complex system theory provide important potential healthcare dividends with respect to delirium. The brain is complex and exhibits emergence, a feature of consciousness, which is crucially impacted in delirium. Volatility, non-linear relationships and multiple point failures are cardinal features of complex system failure, thence delirium. An alternative emphasis away from end of chain analysis and oversimplification of cause and an attempt to avoid introduction of new forms of failure in a responsive healthcare environment are lessons from complex system theory. Insights from complex systems provide potentially important mechanistic underpinnings and new lines of research enquiry for delirium. Not least, a fuller understanding of delirium from a complex system viewpoint may help transform management and outcomes in one of the biggest challenges of acute healthcare.

Extended high-frequency pure tone hearing thresholds and core executive functions

OBJECTIVE

To examine the relationship between extended high-frequency pure tone hearing thresholds (frequencies 10 to 14 kHz) and working memory capacity (WMC), inhibitory control, and cognitive flexibility.

DESIGN

Experimental study measuring correlations between different high-frequency hearing threshold measures and cognitive measures.

STUDY

Pure tone audiometry was assessed in the extended high-frequency (frequencies 10, 12.5 and 14 kHz). Subjects were also tested regarding WMC, inhibitory control (response inhibition), and cognitive flexibility (information updating and shifting ability).

SAMPLE

Forty-three subjects between 20 and 29 years old with normal hearing (≤ 20 dB HL) in the frequency range between 0.125 to 8 kHz.

RESULTS

No significant correlations were seen between high-frequency hearing thresholds defined as average best and worst ear high-frequency hearing thresholds and the cognitive measures. Differences between the best and the worst ear showed significant negative correlations with inhibitory control and global executive function (combination score for WMC, inhibitory control and cognitive flexibility).

CONCLUSIONS

The present study suggests that global executive functions, more specifically response inhibition, and hearing threshold asymmetry in the extended high-frequency range are interrelated.

An oscillatory neural network model that demonstrates the benefits of multisensory learning

Since the world consists of objects that stimulate multiple senses, it is advantageous for a vertebrate to integrate all the sensory information available. However, the precise mechanisms governing the temporal dynamics of multisensory processing are not well understood. We develop a computational modeling approach to investigate these mechanisms. We present an oscillatory neural network model for multisensory learning based on sparse spatio-temporal encoding. Recently published results in cognitive science show that multisensory integration produces greater and more efficient learning. We apply our computational model to qualitatively replicate these results. We vary learning protocols and system dynamics, and measure the rate at which our model learns to distinguish superposed presentations of multisensory objects. We show that the use of multiple channels accelerates learning and recall by up to 80%. When a sensory channel becomes disabled, the performance degradation is less than that experienced during the presentation of non-congruent stimuli. This research furthers our understanding of fundamental brain processes, paving the way for multiple advances including the building of machines with more human-like capabilities.

Exploring the role of task performance and learning style on prefrontal hemodynamics during a working memory task

Existing literature outlines the quality and location of activation in the prefrontal cortex (PFC) during working memory (WM) tasks. However, the effects of individual differences on the underlying neural process of WM tasks are still unclear. In this functional near infrared spectroscopy study, we administered a visual and auditory n-back task to examine activation in the PFC while considering the influences of task performance, and preferred learning strategy (VARK score). While controlling for age, results indicated that high performance (HP) subjects (accuracy > 90%) showed task dependent lower activation compared to normal performance subjects in PFC region Specifically HP groups showed lower activation in left dorsolateral PFC (DLPFC) region during performance of auditory task whereas during visual task they showed lower activation in the right DLPFC. After accounting for learning style, we found a correlation between visual and aural VARK score and level of activation in the PFC. Subjects with higher visual VARK scores displayed lower activation during auditory task in left DLPFC, while those with higher visual scores exhibited higher activation during visual task in bilateral DLPFC. During performance of auditory task, HP subjects had higher visual VARK scores compared to NP subjects indicating an effect of learning style on the task performance and activation. The results of this study show that learning style and task performance can influence PFC activation, with applications toward neurological implications of learning style and populations with deficits in auditory or visual processing.

Neural basis of the crossmodal correspondence between auditory pitch and visuospatial elevation

Crossmodal correspondences refer to associations between otherwise unrelated stimulus features in different sensory modalities. For example, high and low auditory pitches are associated with high and low visuospatial elevation, respectively. The neural mechanisms underlying crossmodal correspondences are currently unknown. Here, we used functional magnetic resonance imaging (fMRI) to investigate the neural basis of the pitch-elevation correspondence. Pitch-elevation congruency effects were observed bilaterally in the inferior frontal and insular cortex, the right frontal eye field and right inferior parietal cortex. Independent functional localizers failed to provide strong evidence for any of three proposed mechanisms for crossmodal correspondences: semantic mediation, magnitude estimation, and multisensory integration. Instead, pitch-elevation congruency effects overlapped with areas selective for visually presented non-word strings relative to sentences, and with regions sensitive to audiovisual asynchrony. Taken together with the prior literature, the observed congruency effects are most consistent with mediation by multisensory attention.

Audiovisual Integration Varies With Target and Environment Richness in Immersive Virtual Reality

We are continually bombarded by information arriving to each of our senses; however, the brain seems to effortlessly integrate this separate information into a unified percept. Although multisensory integration has been researched extensively using simple computer tasks and stimuli, much less is known about how multisensory integration functions in real-world contexts. Additionally, several recent studies have demonstrated that multisensory integration varies tremendously across naturalistic stimuli. Virtual reality can be used to study multisensory integration in realistic settings because it combines realism with precise control over the environment and stimulus presentation. In the current study, we investigated whether multisensory integration as measured by the redundant signals effects (RSE) is observable in naturalistic environments using virtual reality and whether it differs as a function of target and/or environment cue-richness. Participants detected auditory, visual, and audiovisual targets which varied in cue-richness within three distinct virtual worlds that also varied in cue-richness. We demonstrated integrative effects in each environment-by-target pairing and further showed a modest effect on multisensory integration as a function of target cue-richness but only in the cue-rich environment. Our study is the first to definitively show that minimal and more naturalistic tasks elicit comparable redundant signals effects. Our results also suggest that multisensory integration may function differently depending on the features of the environment. The results of this study have important implications in the design of virtual multisensory environments that are currently being used for training, educational, and entertainment purposes.

The Effects of Exercise on Memory Function Among Young to Middle-Aged Adults: Systematic Review and Recommendations for Future Research

OBJECTIVE

To systematically summarize the experimental effects of exercise on cognitive-related memory function among young to middle-aged adults, which has yet to be done in the literature.

DATA SOURCE

PubMed.

STUDY INCLUSION AND EXCLUSION CRITERIA

Studies were included if they were published in the English language, indexed in PubMed, employed an experimental study design (eg, traditional parallel group randomized controlled trial: either acute intervention or chronic/training intervention study), and conducted among human adults. Studies were excluded if nonhumans (ie, animal models) were studied, if children/adolescents (<18 years) or older adults (>50 years) were evaluated, and if select chronic diseases (eg, diabetes and dementia) were present.

DATA EXTRACTION

A systematic review approach was employed.

DATA SYNTHESIS

An extraction table was created synthesizing the key results, and recommendations for future research are emphasized.

RESULTS

Among the 17 evaluated studies, 2 were published before the year 2000 (ie, 1998 and 1999), 2 were published in 2007, and the remaining 13 were published in the years 2011 and beyond. This highlights the emergence of this research topic within this age-group (young to middle-aged adults). Among the 17 evaluated studies, 14 were conducted among healthy samples, with 3 conducted among those with a diagnosis of depression. Among the 17 studies, 4 employed a chronic training protocol, with 13 utilizing an acute exercise protocol. Among the 3 experimental studies in the depressed population, all demonstrated a favorable effect of exercise on memory function. Among the 14 trials in the nondepressed population, 10 (71%) demonstrated a favorable effect of exercise on some aspect of memory function.

CONCLUSION

Acute and chronic exercise appears to play a pronounced effect on memory function among young to middle-aged adults. Implications and recommendations for future research are outlined in this systematic review.

The challenge of forgetting: Neurobiological mechanisms of auditory directed forgetting

Directed forgetting (DF) is considered an adaptive mechanism to cope with unwanted memories. Understanding it is crucial to develop treatments for disorders in which thought control is an issue. With an item-method DF paradigm in an auditory form, the underlying neurocognitive processes that support auditory DF were investigated. Subjects were asked to perform multi-modal encoding of word-stimuli before knowing whether to remember or forget each word. Using functional magnetic resonance imaging, we found that DF is subserved by a right frontal-parietal-cingulate network. Both qualitative and quantitative analyses of the activation of this network show converging evidence suggesting that DF is a complex process in which active inhibition, attentional switching, and working memory are needed to manipulate both unwanted and preferred items. These results indicate that DF is a complex inhibitory mechanism which requires the crucial involvement of brain areas outside prefrontal regions to operate over attentional and working memory processes. Hum Brain Mapp 39:249-263, 2018. © 2017 Wiley Periodicals, Inc.

Semantic congruent audiovisual integration during the encoding stage of working memory: an ERP and sLORETA study

Although multisensory integration is an inherent component of functional brain organization, multisensory integration during working memory (WM) has attracted little attention. The present study investigated the neural properties underlying the multisensory integration of WM by comparing semantically related bimodal stimulus presentations with unimodal stimulus presentations and analysing the results using the standardized low-resolution brain electromagnetic tomography (sLORETA) source location approach. The results showed that the memory retrieval reaction times during congruent audiovisual conditions were faster than those during unisensory conditions. Moreover, our findings indicated that the event-related potential (ERP) for simultaneous audiovisual stimuli differed from the ERP for the sum of unisensory constituents during the encoding stage and occurred within a 236-530 ms timeframe over the frontal and parietal-occipital electrodes. The sLORETA images revealed a distributed network of brain areas that participate in the multisensory integration of WM. These results suggested that information inputs from different WM subsystems yielded nonlinear multisensory interactions and became integrated during the encoding stage. The multicomponent model of WM indicates that the central executive could play a critical role in the integration of information from different slave systems.

Voice over: Audio-visual congruency and content recall in the gallery setting

Experimental research has shown that pairs of stimuli which are congruent and assumed to ‘go together’ are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and narrative style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues.

Painful engrams: Oscillatory correlates of working memory for phasic nociceptive laser stimuli

Research suggests that working memory (WM) is impaired in chronic pain. Yet, information on how potentially noxious stimuli are maintained in memory is limited in patients as well as in healthy people. We recorded electroencephalography (EEG) in healthy volunteers during a modified delayed match-to-sample task where maintenance in memory of relevant attributes of nociceptive laser stimuli was essential for subsequent cued-discrimination. Participants performed in high and low load conditions (i.e. three vs. two stimuli to keep in WM). Modulation of EEG oscillations in the beta band during the retention interval and in the alpha band during the pre-retention interval reflected performance in the WM task. Importantly, both a non-verbal and a verbal neuropsychological WM test predicted oscillatory modulations. Moreover, these two neuropsychological tests and self-reported personality measures predicted the performance in the nociceptive WM task. Results demonstrate (i) that beta and alpha EEG oscillations can represent WM for nociceptive stimuli; (ii) the association between neuropsychological measures of WM and the brain representation of phasic nociceptive painful stimuli; and (iii) that personality factors can predict memory for nociceptive stimuli at the behavioural level. Altogether, our findings offer a promising approach for investigating cortical correlates of nociceptive memory in clinical pain conditions.

The variability of multisensory processes of natural stimuli in human and non-human primates in a detection task

Background

Behavioral studies in both human and animals generally converge to the dogma that multisensory integration improves reaction times (RTs) in comparison to unimodal stimulation. These multisensory effects depend on diverse conditions among which the most studied were the spatial and temporal congruences. Further, most of the studies are using relatively simple stimuli while in everyday life, we are confronted to a large variety of complex stimulations constantly changing our attentional focus over time, a modality switch that can impact on stimuli detection. In the present study, we examined the potential sources of the variability in reaction times and multisensory gains with respect to the intrinsic features of a large set of natural stimuli.

Methodology/Principle findings

Rhesus macaque monkeys and human subjects performed a simple audio-visual stimulus detection task in which a large collection of unimodal and bimodal natural stimuli with semantic specificities was presented at different saliencies. Although we were able to reproduce the well-established redundant signal effect, we failed to reveal a systematic violation of the race model which is considered to demonstrate multisensory integration. In both monkeys and human species, our study revealed a large range of multisensory gains, with negative and positive values. While modality switch has clear effects on reaction times, one of the main causes of the variability of multisensory gains appeared to be linked to the intrinsic physical parameters of the stimuli.

Conclusion/Significance

Based on the variability of multisensory benefits, our results suggest that the neuronal mechanisms responsible of the redundant effect (interactions vs. integration) are highly dependent on the stimulus complexity suggesting different implications of uni- and multisensory brain regions. Further, in a simple detection task, the semantic values of individual stimuli tend to have no significant impact on task performances, an effect which is probably present in more cognitive tasks.

Visuospatial Bootstrapping

Visuospatial bootstrapping is the name given to a phenomenon whereby performance on visually presented verbal serial-recall tasks is better when stimuli are presented in a spatial array rather than a single location. However, the display used has to be a familiar one. This phenomenon implies communication between cognitive systems involved in storing short-term memory for verbal and visual information, alongside connections to and from knowledge held in long-term memory. Bootstrapping is a robust, replicable phenomenon that should be incorporated in theories of working memory and its interaction with long-term memory. This article provides an overview of bootstrapping, contextualizes it within research on links between long-term knowledge and short-term memory, and addresses how it can help inform current working memory theory.

Visual Distractors Disrupt Audiovisual Integration Regardless of Stimulus Complexity

The intricate relationship between multisensory integration and attention has been extensively researched in the multisensory field; however, the necessity of attention for the binding of multisensory stimuli remains contested. In the current study, we investigated whether diverting attention from well-known multisensory tasks would disrupt integration and whether the complexity of the stimulus and task modulated this interaction. A secondary objective of this study was to investigate individual differences in the interaction of attention and multisensory integration. Participants completed a simple audiovisual speeded detection task and McGurk task under various perceptual load conditions: no load (multisensory task while visual distractors present), low load (multisensory task while detecting the presence of a yellow letter in the visual distractors), and high load (multisensory task while detecting the presence of a number in the visual distractors). Consistent with prior studies, we found that increased perceptual load led to decreased reports of the McGurk illusion, thus confirming the necessity of attention for the integration of speech stimuli. Although increased perceptual load led to longer response times for all stimuli in the speeded detection task, participants responded faster on multisensory trials than unisensory trials. However, the increase in multisensory response times violated the race model for no and low perceptual load conditions only. Additionally, a geometric measure of Miller’s inequality showed a decrease in multisensory integration for the speeded detection task with increasing perceptual load. Surprisingly, we found diverging changes in multisensory integration with increasing load for participants who did not show integration for the no load condition: no changes in integration for the McGurk task with increasing load but increases in integration for the detection task. The results of this study indicate that attention plays a crucial role in multisensory integration for both highly complex and simple multisensory tasks and that attention may interact differently with multisensory processing in individuals who do not strongly integrate multisensory information.

Sequential memory: Binding dynamics

Temporal order memories are critical for everyday animal and human functioning. Experiments and our own experience show that the binding or association of various features of an event together and the maintaining of multimodality events in sequential order are the key components of any sequential memories-episodic, semantic, working, etc. We study a robustness of binding sequential dynamics based on our previously introduced model in the form of generalized Lotka-Volterra equations. In the phase space of the model, there exists a multi-dimensional binding heteroclinic network consisting of saddle equilibrium points and heteroclinic trajectories joining them. We prove here the robustness of the binding sequential dynamics, i.e., the feasibility phenomenon for coupled heteroclinic networks: for each collection of successive heteroclinic trajectories inside the unified networks, there is an open set of initial points such that the trajectory going through each of them follows the prescribed collection staying in a small neighborhood of it. We show also that the symbolic complexity function of the system restricted to this neighborhood is a polynomial of degree L - 1, where L is the number of modalities.

A Neurophysiological and Neuropsychological Consideration of Mindful Movement: Clinical and Research Implications

In this article, we present ideas related to three key aspects of mindfulness training: the regulation of attention via noradrenaline, the importance of working memory and its various components (particularly the central executive and episodic buffer), and the relationship of both of these to mind-wandering. These same aspects of mindfulness training are also involved in the preparation and execution of movement and implicated in the pathophysiology of psychosis. We argue that by moving in a mindful way, there may be an additive effect of training as the two elements of the practice (mindfulness and movement) independently, and perhaps synergistically, engage common underlying systems (the default mode network). We discuss how working with mindful movement may be one route to mindfulness training for individuals who would struggle to sit still to complete the more commonly taught mindfulness practices. Drawing on our clinical experience working with individuals with severe and enduring mental health conditions, we show the real world application of these ideas and how they can be used to help those who are suffering and for whom current treatments are still far from adequate.