A multisensory perspective of working memory

Beyond Routine Maintenance: Current Trends in Working Memory Research

Working memory (WM) is an evolving concept. Our understanding of the neural functions that support WM develops iteratively alongside the approaches used to study it, and both can be profoundly shaped by available tools and prevailing theoretical paradigms. Here, the organizers of the 2024 Working Memory Symposium-inspired by this year's meeting-highlight current trends and looming questions in WM research. This review is organized into sections describing (1) ongoing efforts to characterize WM function across sensory modalities, (2) the growing appreciation that WM representations are malleable to context and future actions, (3) the enduring problem of how multiple WM items and features are structured and integrated, and (4) new insights about whether WM shares function with other cognitive processes that have conventionally been considered distinct. This review aims to chronicle where the field is headed and calls attention to issues that are paramount for future research.

Visual nudging of navigation strategies improves frequency discrimination during auditory-guided locomotion

Perception in natural environments requires integrating multisensory inputs while navigating our surroundings. During locomotion, sensory cues such as vision and audition change coherently, providing crucial environmental information. This integration may affect perceptual thresholds due to sensory interference. Vision often dominates in multimodal contexts, overshadowing auditory information and potentially degrading audition. While traditional laboratory experiments offer controlled insights into sensory integration, they often fail to replicate the dynamic, multisensory interactions of real-world behavior. We used a naturalistic paradigm in which participants navigate an arena searching for a target guided by position-dependent auditory cues. Previous findings showed that frequency discrimination thresholds during self-motion matched those in stationary paradigms, even though participants often relied on visually dominated navigation instead of auditory feedback. This suggested that vision might affect auditory perceptual thresholds in naturalistic settings. Here, we manipulated visual input to examine its effect on frequency discrimination and search strategy selection. By degrading visual input, we nudged participants' attention toward audition, leveraging subtle sensory adjustments to promote adaptive use of auditory cues without restricting their freedom of choice. Thus, this approach explores how attentional shifts influence multisensory integration during self-motion. Our results show that frequency discrimination thresholds improved by restricting visual input, suggesting that reducing visual interference can increase auditory sensitivity. This is consistent with adaptive behavioral theories, suggesting that individuals can dynamically adjust their perceptual strategies to leverage the most reliable sensory inputs. These findings contribute to a better understanding of multisensory integration, highlighting the flexibility of sensory systems in complex environments.

Characterization of language abilities and semantic networks in very preterm children at school-age

It has been widely assessed that very preterm children (<32 weeks gestational age) present language and memory impairments compared with full-term children. However, differences in their underlying semantic memory structure have not been studied yet. Nevertheless, the way concepts are learned and organized across development relates to children's capacities in retrieving and using information later. Therefore, the semantic memory organization could underlie several cognitive deficits existing in very preterm children. Computational mathematical models offer the possibility to characterize semantic networks through three coefficients calculated on spoken language: average shortest path length (i.e., distance between concepts), clustering (i.e., local interconnectivity), and modularity (i.e., compartmentalization into small sub-networks). Here we assessed these coefficients in 38 very preterm schoolchildren (aged 8-10 years) compared with 38 full-term schoolchildren (aged 7-10 years) based on a verbal fluency task. Using semantic network analysis, very preterm children showed a longer distance between concepts and a lower interconnectivity at a local level than full-term children. In addition, we found a trend for a higher modularity at a global in very preterm children compared with full-term children. These findings provide preliminary evidence that very preterm children demonstrate subtle impairments in the organization of their semantic network, encouraging the adaptation of the support and education they receive.

Comparing auditory and visual aspects of multisensory working memory using bimodally matched feature patterns

Working memory (WM) reflects the transient maintenance of information in the absence of external input, which can be attained via multiple senses separately or simultaneously. Pertaining to WM, the prevailing literature suggests the dominance of vision over other sensory systems. However, this imbalance may be stemming from challenges in finding comparable stimuli across modalities. Here, we addressed this problem by using a balanced multisensory retro-cue WM design, which employed combinations of auditory (ripple sounds) and visuospatial (Gabor patches) patterns, adjusted relative to each participant's discrimination ability. In three separate experiments, the participant was asked to determine whether the (retro-cued) auditory and/or visual items maintained in WM matched or mismatched the subsequent probe stimulus. In Experiment 1, all stimuli were audiovisual, and the probes were either fully mismatching, only partially mismatching, or fully matching the memorized item. Experiment 2 was otherwise the same as Experiment 1, but the probes were unimodal. In Experiment 3, the participant was cued to maintain only the auditory or visual aspect of an audiovisual item pair. In Experiments 1 and 3, the participant's matching performance was significantly more accurate for the auditory than visual attributes of probes. When the perceptual and task demands are bimodally equated, auditory attributes can be matched to multisensory items in WM at least as accurately as, if not more precisely than, their visual counterparts.

Hanging on the telephone: Maintaining visuospatial bootstrapping over time in working memory

Visuospatial bootstrapping (VSB) refers to the phenomenon in which performance on a verbal working memory task can be enhanced by presenting the verbal material within a familiar visuospatial configuration. This effect is part of a broader literature concerning how working memory is influenced by use of multimodal codes and contributions from long-term memory. The present study aimed to establish whether the VSB effect extends over a brief (5-s) delay period, and to explore the possible mechanisms operating during retention. The VSB effect, as indicated by a verbal recall advantage for digit sequences presented within a familiar visuospatial configuration (modelled on the T-9 keypad) relative to a single-location display, was observed across four experiments. The presence and size of this effect changed with the type of concurrent task activity applied during the delay. Articulatory suppression (Experiment 1) increased the visuospatial display advantage, while spatial tapping (Experiment 2) and a visuospatial judgment task (Experiment 3) both removed it. Finally, manipulation of the attentional demands placed by a verbal task also reduced (but did not abolish) this effect (Experiment 4). This pattern of findings demonstrates how provision of familiar visuospatial information at encoding can continue to support verbal working memory over time, with varying demands on modality-specific and general processing resources.

Working memory multicomponent model outcomes in individuals with traumatic brain injury: Critical review and meta-analysis

OBJECTIVE

Traumatic Brain Injury (TBI) often leads to cognitive impairments, particularly regarding working memory (WM). This meta-analysis aims to examine the impact of TBI on WM, taking into account moderating factors which has received little attention in previous research, such as severity of injury, the different domains of Baddeley's multi-component model, and the interaction between these two factors, as well as the interaction with other domains of executive functions.

METHOD

Following Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines, a systematic review and meta-analysis searched Google Scholar, PubMed, and PsycNET for studies with objective WM measures. Multiple meta-analyses were performed to compare the effects of TBI severity on different WM components. Twenty-four English, peer-reviewed articles, mostly cross-sectional were included.

RESULTS

TBI significantly impairs general WM and all Baddeley's model components, most notably the Central Executive (d' = 0.74). Severity categories, mild-moderate and moderate-severe, were identified. Impairment was found across severities, with "moderate-severe" demonstrating the largest effect size (d' = 0.81). Individuals with moderate-severe TBI showed greater impairments in the Central Executive and Episodic Buffer compared to those with mild-moderate injury, whereas no such differences were found for the Phonological Loop and Visuospatial Sketchpad.

CONCLUSIONS

These findings enhance our understanding of WM deficits in varying severities of TBI, highlighting the importance of assessing and treating WM in clinical practice and intervention planning.

Multisensory working memory capture of attention

During visual search, representations in working memory (WM) can guide the deployment of attention toward memory-matching visual input. Although previous studies have demonstrated that multisensory interactions facilitate WM and visual search, it remains unclear whether multisensory interaction influences attentional capture by WM. To address this issue, the present study adopted a dual-task paradigm, pairing a visual search task with a WM task, in which the memory modality was manipulated to be either visual or audiovisual. The results revealed that memory-driven attentional capture was observed under the visual and the audiovisual condition. Additionally, the capture effects and response time (RT) costs under the audiovisual condition were weaker than those under the visual condition, even on the trials with the earliest RTs. Furthermore, RT benefits under the audiovisual condition were comparable with those under the visual condition. These findings suggest that multisensory interactions can enhance cognitive control, leading to robust strategic effects and improved search performance. In this process, cognitive control tends to suppress the attentional capture by WM-matching distractors rather than enhance the attentional capture by WM-matching targets. The present study offers new insights into the influence of multisensory interactions on attentional capture by WM.

Multi-timescale neural dynamics for multisensory integration

Carrying out any everyday task, be it driving in traffic, conversing with friends or playing basketball, requires rapid selection, integration and segregation of stimuli from different sensory modalities. At present, even the most advanced artificial intelligence-based systems are unable to replicate the multisensory processes that the human brain routinely performs, but how neural circuits in the brain carry out these processes is still not well understood. In this Perspective, we discuss recent findings that shed fresh light on the oscillatory neural mechanisms that mediate multisensory integration (MI), including power modulations, phase resetting, phase-amplitude coupling and dynamic functional connectivity. We then consider studies that also suggest multi-timescale dynamics in intrinsic ongoing neural activity and during stimulus-driven bottom-up and cognitive top-down neural network processing in the context of MI. We propose a new concept of MI that emphasizes the critical role of neural dynamics at multiple timescales within and across brain networks, enabling the simultaneous integration, segregation, hierarchical structuring and selection of information in different time windows. To highlight predictions from our multi-timescale concept of MI, real-world scenarios in which multi-timescale processes may coordinate MI in a flexible and adaptive manner are considered.

The audiovisual competition effect induced by temporal asynchronous encoding weakened the visual dominance in working memory retrieval

Converging evidence suggests a facilitation effect of multisensory interactions on memory performance, reflected in higher accuracy or faster response time under a bimodal encoding condition than a unimodal condition. However, relatively little attention has been given to the effect of multisensory competition on memory. The present study adopted an adaptive staircase test to measure the point of subjective simultaneity (PSS), combined with a delayed matched-to-sample (DMS) task to probe the effect of audiovisual competition during the encoding stage on subsequent unisensory retrieval. The results showed that there was a robust visual dominance effect and multisensory interference effect in WM retrieval, regardless of the subjective synchronous or subjective asynchronous audiovisual presentation. However, a weakened visual dominance effect was observed when the auditory stimulus was presented before the visual stimulus in the encoding period, particularly in the semantically incongruent case. These findings revealed that the prior-entry of sensory information in the early perceptual stage could affect the processing in the late cognitive stage to some extent, and supported the evidence that there is a persistent advantage for visuospatial sketchpad in multisensory WM.

The impact of premature birth on auditory-visual processes in very preterm schoolchildren

Interactions between stimuli from different sensory modalities and their integration are central to daily life, contributing to improved perception. Being born prematurely and the subsequent hospitalization can have an impact not only on sensory processes, but also on the manner in which information from different senses is combined-i.e., multisensory processes. Very preterm (VPT) children (<32 weeks gestational age) present impaired multisensory processes in early childhood persisting at least through the age of five. However, it remains largely unknown whether and how these consequences persist into later childhood. Here, we evaluated the integrity of auditory-visual multisensory processes in VPT schoolchildren. VPT children (N = 28; aged 8-10 years) received a standardized cognitive assessment and performed a simple detection task at their routine follow-up appointment. The simple detection task involved pressing a button as quickly as possible upon presentation of an auditory, visual, or simultaneous audio-visual stimulus. Compared to full-term (FT) children (N = 23; aged 6-11 years), reaction times of VPT children were generally slower and more variable, regardless of sensory modality. Nonetheless, both groups exhibited multisensory facilitation on mean reaction times and inter-quartile ranges. There was no evidence that standardized cognitive or clinical measures correlated with multisensory gains of VPT children. However, while gains in FT children exceeded predictions based on probability summation and thus forcibly invoked integrative processes, this was not the case for VPT children. Our findings provide evidence of atypical multisensory profiles in VPT children persisting into school-age. These results could help in targeting supportive interventions for this vulnerable population.

Semantically congruent bimodal presentation modulates cognitive control over attentional guidance by working memory

Although previous studies have well established that audiovisual enhancement has a promoting effect on working memory and selective attention, there remains an open question about the influence of audiovisual enhancement on attentional guidance by working memory. To address this issue, the present study adopted a dual-task paradigm that combines a working memory task and a visual search task, in which the content of working memory was presented in audiovisual or visual modalities. Given the importance of search speed in memory-driven attentional suppression, we divided participants into two groups based on their reaction time (RT) in neutral trials and examined whether audiovisual enhancement in attentional suppression was modulated by search speed. The results showed that the slow search group exhibited a robust memory-driven attentional suppression effect, and the suppression effect started earlier and its magnitude was greater in the audiovisual condition than in the visual-only condition. However, among the faster search group, the suppression effect only occurred in the trials with longer RTs in the visual-only condition, and its temporal dynamics were selectively improved in the audiovisual condition. Furthermore, audiovisual enhancement of memory-driven attention evolved over time. These findings suggest that semantically congruent bimodal presentation can progressively facilitate the strength and temporal dynamics of memory-driven attentional suppression, and that search speed plays an important role in this process. This may be due to a synergistic effect between multisensory working memory representation and top-down suppression mechanism. The present study demonstrates the flexible role of audiovisual enhancement on cognitive control over memory-driven attention.

The effects of errorless psychomotor training in the Y balance lower limb reaching task

This study investigated the training effect of errorless psychomotor training, a motor training method with perceptual, attentional, and psychological manipulation, in a balance-related, lower limb reaching task (Y balance reaching task) on dynamic balance by young adults. Thirty-nine participants (Mean age = 27.03 years, SD = 2.64 years) were trained with different psychomotor training methods in the Y balance reaching task. Results illustrate that errorless psychomotor training significantly improved the participants' dynamic balance and proprioceptive abilities. Additionally, gaze fixation duration on target during reaching decreased after errorless psychomotor training, suggesting that errorless psychomotor training could decrease visual information demand and be concurrently compensated by up-weighting on proprioception. This multisensory reweighting and cross-modal attention could contribute to the improvement of dynamic balance ability in sports.

Temporal integration characteristics of an image defined by binocular disparity cues

We can correctly recognize the content of an image by presenting all of the elements within a limited time, such as in a slit view or a divided painting image. It is important to clarify how temporally divided information is integrated and perceived to understand the temporal properties of the information-processing mechanism of visual systems. Previous studies related to this topic have often used two-dimensional pictorial stimuli; however, few have considered the temporal integration of binocular disparity for the recognition of objects defined with disparity. In this study, we examined image recognition properties based on the temporal integration of binocular disparity, by comparing that based on the temporal integration of luminance. The effect of element onset asynchrony (the time lag among presented elements) was somewhat similar between disparity and luminance with respect to randomly divided elements. On the other hand, under slit-vision conditions, the tolerance range of spatiotemporal integration for luminance stimuli was much wider than that for disparity stimuli. These results indicate that the temporal integration mechanism in localized areas is common to disparity and luminance, but that for global motion shows differences between the two mechanisms. Thus, we conclude that global motion has little contribution to the temporal integration of binocular disparity information for image recognition.

Comparing auditory and visual aspects of multisensory working memory using bimodally matched feature patterns

Working memory (WM) reflects the transient maintenance of information in the absence of external input, which can be attained via multiple senses separately or simultaneously. Pertaining to WM, the prevailing literature suggests the dominance of vision over other sensory systems. However, this imbalance may be stemming from challenges in finding comparable stimuli across modalities. Here, we addressed this problem by using a balanced multisensory retro-cue WM design, which employed combinations of auditory (ripple sounds) and visuospatial (Gabor patches) patterns, adjusted relative to each participant's discrimination ability. In three separate experiments, the participant was asked to determine whether the (retro-cued) auditory and/or visual items maintained in WM matched or mismatched the subsequent probe stimulus. In Experiment 1, all stimuli were audiovisual, and the probes were either fully mismatching, only partially mismatching, or fully matching the memorized item. Experiment 2 was otherwise same as Experiment 1, but the probes were unimodal. In Experiment 3, the participant was cued to maintain only the auditory or visual aspect of an audiovisual item pair. In two of the three experiments, the participant matching performance was significantly more accurate for the auditory than visual attributes of probes. When the perceptual and task demands are bimodally equated, auditory attributes can be matched to multisensory items in WM at least as accurately as, if not more precisely than, their visual counterparts.

The effect of unisensory and multisensory information on lexical decision and free recall in young and older adults

Studies using simple low-level stimuli show that multisensory stimuli lead to greater improvements in processing speed for older adults than young adults. However, there is insufficient evidence to explain how these benefits influence performance for more complex processes such as judgement and memory tasks. This study examined how presenting stimuli in multiple sensory modalities (audio-visual) instead of one (audio-only or visual-only) may help older adults to improve their memory and cognitive processing compared to young adults. Young and older adults completed lexical decision (real word vs. pseudoword judgement) and word recall tasks, either independently, or in combination (dual-task), with and without perceptual noise. Older adults were better able to remember words when encoding independently. In contrast, young adults were better able to remember words when encoding in combination with lexical decisions. Both young and older adults had better word recall in the audio-visual condition compared with the audio-only condition. The findings indicate significant age differences when dealing with multiple tasks during encoding. Crucially, there is no greater multisensory benefit for older adults compared to young adults in more complex processes, rather multisensory stimuli can be useful in enhancing cognitive performance for both young and older adults.

Visuospatial attention revamps cortical processing of sound amid audiovisual uncertainty

Selective attentional biases arising from one sensory modality manifest in others. The effects of visuospatial attention, important in visual object perception, are unclear in the auditory domain during audiovisual (AV) scene processing. We investigate temporal and spatial factors that underlie such transfer neurally. Auditory encoding of random tone pips in AV scenes was addressed via a temporal response function model (TRF) of participants' electroencephalogram (N = 30). The spatially uninformative pips were associated with spatially distributed visual contrast reversals ("flips"), through asynchronous probabilistic AV temporal onset distributions. Participants deployed visuospatial selection on these AV stimuli to perform a task. A late (~300 ms) cross-modal influence over the neural representation of pips was found in the original and a replication study (N = 21). Transfer depended on selected visual input being (i) presented during or shortly after a related sound, in relatively limited temporal distributions (<165 ms); (ii) positioned across limited (1:4) visual foreground to background ratios. Neural encoding of auditory input, as a function of visual input, was largest at visual foreground quadrant sectors and lowest at locations opposite to the target. The results indicate that ongoing neural representations of sounds incorporate visuospatial attributes for auditory stream segregation, as cross-modal transfer conveys information that specifies the identity of multisensory signals. A potential mechanism is by enhancing or recalibrating the tuning properties of the auditory populations that represent them as objects. The results account for the dynamic evolution under visual attention of multisensory integration, specifying critical latencies at which relevant cortical networks operate.

Processing symmetry between visual and auditory spatial representations in updating working memory

Updating spatial representations in visual and auditory working memory relies on common processes, and the modalities should compete for attentional resources. If competition occurs, one type of spatial information is presumably weighted over the other, irrespective of sensory modality. This study used incompatible spatial information conveyed from two different cue modalities to examine relative dominance in memory updating. Participants mentally manoeuvred a designated target in a matrix according to visual or auditory stimuli that were presented simultaneously, to identify a terminal location. Prior to the navigation task, the relative perceptual saliences of the visual cues were manipulated to be equal, superior, or inferior to the auditory cues. The results demonstrate that visual and auditory information competed for attentional resources, such that visual/auditory guidance was impaired by incongruent cues delivered from the other modality. Although visual bias was generally observed in working-memory navigation, stimuli of relatively high salience interfered with or facilitated other stimuli regardless of modality, demonstrating the processing symmetry of spatial updating in visual and auditory spatial working memory. Furthermore, this processing symmetry can be identified during the encoding of sensory inputs into working-memory representations. The results imply that auditory spatial updating is comparable to visual spatial updating in that salient stimuli receive a high priority when selecting inputs and are used when tracking spatial representations.

A Novel Neurorehabilitation Approach for Neural Plasticity Overstimulation and Reorganization in Patients with Neurological Disorders

Neurological disorders are those that are associated with impairments in the nervous system. These impairments affect the patient’s activities of daily living. Recently, many advanced modalities have been used in the rehabilitation field to treat various neurological impairments. However, many of these modalities are available only in clinics, and some are expensive. Most patients with neurological disorders have difficulty reaching clinics. This review was designed to establish a new neurorehabilitation approach based on the scientific way to improve patients’ functional recovery following neurological disorders in clinics or at home. The human brain is a network, an intricate, integrated system that coordinates operations among billions of units. In fact, grey matter contains most of the neuronal cell bodies. It includes the brain and the spinal cord areas involved in muscle control, sensory perception, memory, emotions, decision-making, and self-control. Consequently, patients’ functional ability results from complex interactions among various brain and spinal cord areas and neuromuscular systems. While white matter fibers connect numerous brain areas, stimulating or improving non-motor symptoms, such as motivation, cognitive, and sensory symptoms besides motor symptoms may enhance functional recovery in patients with neurological disorders. The basic principles of the current treatment approach are established based on brain connectivity. Using motor, sensory, motivation, and cognitive (MSMC) interventions during rehabilitation may promote neural plasticity and maximize functional recovery in patients with neurological disorders. Experimental studies are strongly needed to verify our theories and hypothesis.

Evaluating the dual-task decrement within a simulated environment: Word recall and visual search

Simulated environments have become better able to replicate the real world and can be used for a variety of purposes, such as testing new technology without any of the costs or risks associated with working in the real world. Because of this, it is now possible to gain a better understanding of cognitive demands when working in operational environments, where individuals are often required to multitask. Multitasking often results in performance decrements, where adding more tasks can cause a decrease in performance in each of the individual tasks. However, little research investigated multitasking performance in simulated environments. In the current study we examined how multitasking affects performance in simulated environments. Forty-eight participants performed a dual visual search and word memory task where participants were navigated through a simulated environment while being presented with words. Performance was then compared to single-task performance (visual search and word memory alone). Results showed that participants experienced significant dual-task interference when comparing the dual-tasks to the single-tasks and subjective measures confirmed these findings. These results could provide useful insight for the design of technology in operational environments, but also serve as an evaluation of MRT in simulated environments.

Long-term memory representations for audio-visual scenes

In this study, we investigated the nature of long-term memory representations for naturalistic audio-visual scenes. Whereas previous research has shown that audio-visual scenes are recognized more accurately than their unimodal counterparts, it remains unclear whether this benefit stems from audio-visually integrated long-term memory representations or a summation of independent retrieval cues. We tested two predictions for audio-visually integrated memory representations. First, we used a modeling approach to test whether recognition performance for audio-visual scenes is more accurate than would be expected from independent retrieval cues. This analysis shows that audio-visual integration is not necessary to explain the benefit of audio-visual scenes relative to purely auditory or purely visual scenes. Second, we report a series of experiments investigating the occurrence of study-test congruency effects for unimodal and audio-visual scenes. Most importantly, visually encoded information was immune to additional auditory information presented during testing, whereas auditory encoded information was susceptible to additional visual information presented during testing. This renders a true integration of visual and auditory information in long-term memory representations unlikely. In sum, our results instead provide evidence for visual dominance in long-term memory. Whereas associative auditory information is capable of enhancing memory performance, the long-term memory representations appear to be primarily visual.

Cognitive Neurorehabilitation in Epilepsy Patients via Virtual Reality Environments: Systematic Review

OBJECTIVE

Epilepsy patients could possibly benefit from the remuneration observed in the use of virtual reality (VR) and virtual environments (VEs), especially in cognitive difficulties associated with visuospatial navigation (memory, attention, and processing speed).

AIM

Research questions under consideration in the present systematic review are associated to VEs' efficiency as a cognitive rehabilitation practice in epilepsy and the particular VR methods indicated for epilepsy patients. To meet criteria, studies included participants suffering from any form of epilepsy and a methodological design with a structured rehabilitation program/model. Data were collected online, using academic databases.

RESULTS

Fourteen studies were included in the literature review and 6 in the statistical analysis. ROBINS-I protocol was implemented to assess the risk of bias. An inverse variance analysis (random effects) of pooled estimates of differences was implemented, in the form of continuous data. Despite the heterogeneity of the studies, all of them agree on the beneficial aspects of VR and VEs in cognitive rehabilitation in relation to visuospatial memory, attention, and information processing speed.

CONCLUSION

We suggest that patients suffering from epilepsy may benefit from the use of VR cognitive rehabilitation interventions, concerning visuospatial memory, attention, and information processing speed. However, further investigation is needed in order to gain a better understanding of the mechanisms involved in cognitive rehabilitation via VEs and establish efficient and dynamic rehabilitation protocols.

Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device

Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes' identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception.

A structured multimodal teaching approach enhancing musculoskeletal physical examination skills among undergraduate medical students

Current evidence indicates that undergraduate medical students display deficits in musculoskeletal physical examination skills (MPES). While various instructional methods are recommended for teaching clinical skills, effective methods for teaching MPES have not been established. This study compared the effectiveness of a multimodal teaching approach incorporating video-based learning, interactive small-group teaching, hands-on practicing, peer-assisted learning, formative assessment, and constructive feedback with traditional bedside teaching in developing undergraduate orthopedic MPES. Participants were 151 fifth-year medical students divided into two groups. One group received multimodal teaching, and the other received traditional bedside teaching. In both groups, the participants learned how to physically examine the knee and shoulder. The primary outcome was objective structured clinical examination (OSCE) scores, while the secondary outcomes included teaching sessions' total durations, facilitator's demonstration time, participants' practice time, and proportion of students with passing checklist scores and global ratings-based assessments for the two teaching approaches. The multimodal teaching group had significantly higher OSCE scores (checklist scores, global ratings, and passing rates; p = 0.02, 0.02, 0.01, respectively) than the comparison group. Individual OSCE component assessments showed significant improvements in the special musculoskeletal physical examination test. The overall duration and amount of participants' hands-on time were significantly longer for the multimodal than for the traditional bedside teaching group (p = 0.01 and 0.01, respectively), and the facilitator's demonstration time was significantly shorter (p = 0.01). The multimodal learner-centered teaching approach evaluated in this study was effective for teaching MPES. It appeared to maximize learner engagement through enhancing interactions and providing increased time to engage in hands-on practice. This teaching approach improved MPES levels, maximized teaching efficiency for scenarios with limited instruction time and resources, and enhanced competency of undergraduate medical students in performing special musculoskeletal physical examinations compared to traditional bedside teaching.

With No Attention Specifically Directed to It, Rhythmic Sound Does Not Automatically Facilitate Visual Task Performance

In a century where humans and machines—powered by artificial intelligence or not—increasingly work together, it is of interest to understand human processing of multi-sensory stimuli in relation to attention and working memory. This paper explores whether and when supporting visual information with rhythmic auditory stimuli can optimize multi-sensory information processing. In turn, this can make the interaction between humans or between machines and humans more engaging, rewarding and activating. For this purpose a novel working memory paradigm was developed where participants are presented with a series of five target digits randomly interchanged with five distractor digits. Their goal is to remember the target digits and recall them orally. Depending on the condition support is provided by audio and/or rhythm. It is expected that the sound will lead to a better performance. It is also expected that this effect of sound is different in case of rhythmic and non-rhythmic sound. Last but not least, some variability is expected across participants. To make correct conclusions, the data of the experiment was statistically analyzed in a classic way, but also predictive models were developed in order to predict outcomes based on a range of input variables related to the experiment and the participant. The effect of auditory support could be confirmed, but no difference was observed between rhythmic and non-rhythmic sounds. Overall performance was indeed affected by individual differences, such as visual dominance or perceived task difficulty. Surprisingly a music education did not significantly affect the performance and even tended toward a negative effect. To better understand the underlying processes of attention, also brain activation data, e.g., by means of electroencephalography (EEG), should be recorded. This approach can be subject to a future work.

The Spatiotemporal Neural Dynamics of Intersensory Attention Capture of Salient Stimuli: A Large-Scale Auditory-Visual Modeling Study

The spatiotemporal dynamics of the neural mechanisms underlying endogenous (top-down) and exogenous (bottom-up) attention, and how attention is controlled or allocated in intersensory perception are not fully understood. We investigated these issues using a biologically realistic large-scale neural network model of visual-auditory object processing of short-term memory. We modeled and incorporated into our visual-auditory object-processing model the temporally changing neuronal mechanisms for the control of endogenous and exogenous attention. The model successfully performed various bimodal working memory tasks, and produced simulated behavioral and neural results that are consistent with experimental findings. Simulated fMRI data were generated that constitute predictions that human experiments could test. Furthermore, in our visual-auditory bimodality simulations, we found that increased working memory load in one modality would reduce the distraction from the other modality, and a possible network mediating this effect is proposed based on our model.

Effect of Audiovisual Cross-Modal Conflict during Working Memory Tasks: A Near-Infrared Spectroscopy Study

Cognitive conflict effects are well characterized within unimodality. However, little is known about cross-modal conflicts and their neural bases. This study characterizes the two types of visual and auditory cross-modal conflicts through working memory tasks and brain activities. The participants consisted of 31 healthy, right-handed, young male adults. The Paced Auditory Serial Addition Test (PASAT) and the Paced Visual Serial Addition Test (PVSAT) were performed under distractor and no distractor conditions. Distractor conditions comprised two conditions in which either the PASAT or PVSAT was the target task, and the other was used as a distractor stimulus. Additionally, oxygenated hemoglobin (Oxy-Hb) concentration changes in the frontoparietal regions were measured during tasks. The results showed significantly lower PASAT performance under distractor conditions than under no distractor conditions, but not in the PVSAT. Oxy-Hb changes in the bilateral ventrolateral prefrontal cortex (VLPFC) and inferior parietal cortex (IPC) significantly increased in the PASAT with distractor compared with no distractor conditions, but not in the PVSAT. Furthermore, there were significant positive correlations between Δtask performance accuracy and ΔOxy-Hb in the bilateral IPC only in the PASAT. Visual cross-modal conflict significantly impairs auditory task performance, and bilateral VLPFC and IPC are key regions in inhibiting visual cross-modal distractors.

EXPRESS: The Rubber Hand Illusion: Top-Down Attention Modulates Embodiment

The Rubber Hand Illusion (RHI) creates distortions of body ownership through multimodal integration of somatosensory and visual inputs. This illusion largely rests on bottom-up (automatic multisensory and perceptual integration) mechanisms. However, the relative contribution from top-down factors, such as controlled processes involving attentional regulation, remains unclear. Following previous work that highlights the putative influence of higher-order cognition in the RHI, we aimed to further examine how modulations of working memory load and task instructions—two conditions engaging top-down cognitive processes—influence the experience of the RHI, as indexed by a number of psychometric dimensions. Relying on exploratory factor analysis for assessing this phenomenology within the RHI, our results confirm the influence of higher-order, top-down mental processes. Whereas task instruction strongly modulated embodiment of the rubber hand, cognitive load altered the affective dimension of the RHI. Our findings corroborate that top-down processes shape the phenomenology of the RHI and herald new ways to improve experimental control over the RHI.

Is the prefrontal cortex organized by supramodal or modality-specific sensory demands during adolescence?

Attention is inherently biased towards the visual modality during most multisensory scenarios in adults, but the developmental trajectory towards visual dominance has not been fully elucidated. More recent evidence in primates and adult humans suggests a modality-specific stratification of the prefrontal cortex. The current study therefore used functional magnetic resonance imaging (fMRI) to investigate the neuronal correlates of proactive (following cues) and reactive (following probes) cognitive control for simultaneous audio-visual stimulation in 67 healthy adolescents (13-18 years old). Behavioral results were only partially supportive of visual dominance in adolescents, with both reduced response times and accuracy during attend-visual relative to attend-auditory trials. Differential activation of medial and lateral prefrontal cortex for processing incongruent relative to congruent stimuli (reactive control) was also only observed during attend-visual trials. There was no evidence of modality-specific prefrontal cortex stratification during the active processing of multisensory stimuli or during separate functional connectivity analyses. Attention-related modulations were also greater within visual relative to auditory cortex, but were less robust than observed in previous adult studies. Collectively, current results suggest a continued transition towards visual dominance in adolescence, as well as limited modality-specific specialization of prefrontal cortex and attentional modulations of unisensory cortex.

Multisensory Facilitation of Working Memory Training

Research suggests that memorization of multisensory stimuli benefits performance compared to memorization of unisensory stimuli; however, little is known about multisensory facilitation in the context of working memory (WM) training and transfer. To investigate this, 240 adults were randomly assigned to an N-back training task that consisted of visual-only stimuli, alternating visual and auditory blocks, or audio-visual (multisensory) stimuli, or to a passive control group. Participants in the active groups completed 13 sessions of N-back training (6.7 hours in total) and all groups completed a battery of WM tasks: untrained N-back tasks, Corsi Blocks, Sequencing, and Symmetry Span. The Multisensory group showed similar training N-level gain compared to the Visual Only group, and both of these groups outperformed the Alternating group on the training task. As expected, all three active groups significantly improved on untrained visual N-back tasks compared to the Control group. In contrast, the Multisensory group showed significantly greater gains on the Symmetry Span task and to a certain extent on the Sequencing task compared to other groups. These results tentatively suggest that incorporating multisensory objects in a WM training protocol can benefit performance on the training task and potentially facilitate transfer to complex WM span tasks.

The role of vision and proprioception in self-motion encoding: An immersive virtual reality study

Past research on the advantages of multisensory input for remembering spatial information has mainly focused on memory for objects or surrounding environments. Less is known about the role of cue combination in memory for own body location in space. In a previous study, we investigated participants' accuracy in reproducing a rotation angle in a self-rotation task. Here, we focus on the memory aspect of the task. Participants had to rotate themselves back to a specified starting position in three different sensory conditions: a blind condition, a condition with disrupted proprioception, and a condition where both vision and proprioception were reliably available. To investigate the difference between encoding and storage phases of remembering proprioceptive information, rotation amplitude and recall delay were manipulated. The task was completed in a real testing room and in immersive virtual reality (IVR) simulations of the same environment. We found that proprioceptive accuracy is lower when vision is not available and that performance is generally less accurate in IVR. In reality conditions, the degree of rotation affected accuracy only in the blind condition, whereas in IVR, it caused more errors in both the blind condition and to a lesser degree when proprioception was disrupted. These results indicate an improvement in encoding own body location when vision and proprioception are optimally integrated. No reliable effect of delay was found.

Do you hear what you see? Utilizing phonocardiography to enhance proficiency in cardiac auscultation

INTRODUCTION

Cardiac auscultation skills have proven difficult to train and maintain. The authors investigated whether using phonocardiograms as visual adjuncts to audio cases improved first-year medical students' cardiac auscultation performance.

METHODS

The authors randomized 135 first-year medical students using an email referral link in 2018 and 2019 to train using audio-only cases (audio group) or audio with phonocardiogram tracings (combined group). Training included 7 cases with normal and abnormal auscultation findings. The assessment included feature identification and diagnostic accuracy using 14 audio-only cases, 7 presented during training, and 7 alternate versions of the same diagnoses. The assessment-administered immediately after training and repeated 7 days later-prompted participants to identify the key features and diagnoses for 14 audio-only cases. Key feature scores and diagnostic accuracy were compared between groups using repeated measures ANOVA.

RESULTS

Mean key feature scores were statistically significantly higher in the combined group (70%, 95% CI 67-75%) compared to the audio group (61%, 95% CI 56-66%) (F(1,116) = 6.144, p = 0.015, d_s = 0.45). Similarly, mean diagnostic accuracy in the combined group (68%, 95% CI 62-73%) was significantly higher than the audio group, although with small effect size (59%, 95% CI 54-65%) (F(1,116) = 4.548, p = 0.035, d_s = 0.40). Time on task for the assessment and prior auscultation experience did not significantly impact performance on either measure.

DISCUSSION

The addition of phonocardiograms to supplement cardiac auscultation training improves diagnostic accuracy and heart sound feature identification amongst novice students compared to training with audio alone.

Theta Oscillations and Source Connectivity During Complex Audiovisual Object Encoding in Working Memory

Working memory is a limited capacity memory system that involves the short-term storage and processing of information. Neuroscientific studies of working memory have mostly focused on the essential roles of neural oscillations during item encoding from single sensory modalities (e.g., visual and auditory). However, the characteristics of neural oscillations during multisensory encoding in working memory are rarely studied. Our study investigated the oscillation characteristics of neural signals in scalp electrodes and mapped functional brain connectivity while participants encoded complex audiovisual objects in a working memory task. Experimental results showed that theta oscillations (4–8 Hz) were prominent and topographically distributed across multiple cortical regions, including prefrontal (e.g., superior frontal gyrus), parietal (e.g., precuneus), temporal (e.g., inferior temporal gyrus), and occipital (e.g., cuneus) cortices. Furthermore, neural connectivity at the theta oscillation frequency was significant in these cortical regions during audiovisual object encoding compared with single modality object encoding. These results suggest that local oscillations and interregional connectivity via theta activity play an important role during audiovisual object encoding and may contribute to the formation of working memory traces from multisensory items.

Commonalities of visual and auditory working memory in a spatial-updating task

Although visual and auditory inputs are initially processed in separate perception systems, studies have built on the idea that to maintain spatial information these modalities share a component of working memory. The present study used working memory navigation tasks to examine functional similarities and dissimilarities in the performance of updating tasks. Participants mentally updated the spatial location of a target in a virtual array in response to sequential pictorial and sonant directional cues before identifying the target's final location. We predicted that if working memory representations are modality-specific, mixed-modality cues would demonstrate a cost of modality switching relative to unimodal cues. The results indicate that updating performance using visual unimodal cues positively correlated with that using auditory unimodal cues. Task performance using unimodal cues was comparable to that using mixed modality cues. The results of a subsequent experiment involving updating of target traces were consistent with those of the preceding experiments and support the view of modality-nonspecific memory.

A Functional MRI Paradigm for Efficient Mapping of Memory Encoding Across Sensory Conditions

We introduce a new and time-efficient memory-encoding paradigm for functional magnetic resonance imaging (fMRI). This paradigm is optimized for mapping multiple contrasts using a mixed design, using auditory (environmental/vocal) and visual (scene/face) stimuli. We demonstrate that the paradigm evokes robust neuronal activity in typical sensory and memory networks. We were able to detect auditory and visual sensory-specific encoding activities in auditory and visual cortices. Also, we detected stimulus-selective activation in environmental-, voice-, scene-, and face-selective brain regions (parahippocampal place and fusiform face area). A subsequent recognition task allowed the detection of sensory-specific encoding success activity (ESA) in both auditory and visual cortices, as well as sensory-unspecific positive ESA in the hippocampus. Further, sensory-unspecific negative ESA was observed in the precuneus. Among others, the parallel mixed design enabled sustained and transient activity comparison in contrast to rest blocks. Sustained and transient activations showed great overlap in most sensory brain regions, whereas several regions, typically associated with the default-mode network, showed transient rather than sustained deactivation. We also show that the use of a parallel mixed model had relatively little influence on positive or negative ESA. Together, these results demonstrate a feasible, versatile, and brief memory-encoding task, which includes multiple sensory stimuli to guarantee a comprehensive measurement. This task is especially suitable for large-scale clinical or population studies, which aim to test task-evoked sensory-specific and sensory-unspecific memory-encoding performance as well as broad sensory activity across the life span within a very limited time frame.

Sensory and Behavioral Components of Neocortical Signal Flow in Discrimination Tasks with Short-Term Memory

In the neocortex, each sensory modality engages distinct sensory areas that route information to association areas. Where signal flow converges for maintaining information in short-term memory and how behavior may influence signal routing remain open questions. Using wide-field calcium imaging, we compared cortex-wide neuronal activity in layer 2/3 for mice trained in auditory and tactile tasks with delayed response. In both tasks, mice were either active or passive during stimulus presentation, moving their body or sitting quietly. Irrespective of behavioral strategy, auditory and tactile stimulation activated distinct subdivisions of the posterior parietal cortex, anterior area A and rostrolateral area RL, which held stimulus-related information necessary for the respective tasks. In the delay period, in contrast, behavioral strategy rather than sensory modality determined short-term memory location, with activity converging frontomedially in active trials and posterolaterally in passive trials. Our results suggest behavior-dependent routing of sensory-driven cortical signals flow from modality-specific posterior parietal cortex (PPC) subdivisions to higher association areas.

Predicting Stimulus Modality and Working Memory Load During Visual- and Audiovisual-Acquired Equivalence Learning

Scholars have extensively studied the electroencephalography (EEG) correlates of associative working memory (WM) load. However, the effect of stimulus modality on EEG patterns within this process is less understood. To fill this research gap, the present study re-analyzed EEG datasets recorded during visual and audiovisual equivalence learning tasks from earlier studies. The number of associations required to be maintained (WM load) in WM was increased using the staircase method during the acquisition phase of the tasks. The support vector machine algorithm was employed to predict WM load and stimulus modality using the power, phase connectivity, and cross-frequency coupling (CFC) values obtained during time segments with different WM loads in the visual and audiovisual tasks. A high accuracy (>90%) in predicting stimulus modality based on power spectral density and from the theta-beta CFC was observed. However, accuracy in predicting WM load was higher (≥75% accuracy) than that in predicting stimulus modality (which was at chance level) using theta and alpha phase connectivity. Under low WM load conditions, this connectivity was highest between the frontal and parieto-occipital channels. The results validated our findings from earlier studies that dissociated stimulus modality based on power-spectra and CFC during equivalence learning. Furthermore, the results emphasized the importance of alpha and theta frontoparietal connectivity in WM load.

Rapid changes in brain activity during learning of grapheme-phoneme associations in adults

Learning to associate written letters with speech sounds is crucial for the initial phase of acquiring reading skills. However, little is known about the cortical reorganization for supporting letter-speech sound learning, particularly the brain dynamics during the learning of grapheme-phoneme associations. In the present study, we trained 30 Finnish participants (mean age: 24.33 years, SD: 3.50 years) to associate novel foreign letters with familiar Finnish speech sounds on two consecutive days (first day ​~ ​50 ​min; second day ​~ ​25 ​min), while neural activity was measured using magnetoencephalography (MEG). Two sets of audiovisual stimuli were used for the training in which the grapheme-phoneme association in one set (Learnable) could be learned based on the different learning cues provided, but not in the other set (Control). The learning progress was tracked at a trial-by-trial basis and used to segment different learning stages for the MEG source analysis. The learning-related changes were examined by comparing the brain responses to Learnable and Control uni/multi-sensory stimuli, as well as the brain responses to learning cues at different learning stages over the two days. We found dynamic changes in brain responses related to multi-sensory processing when grapheme-phoneme associations were learned. Further, changes were observed in the brain responses to the novel letters during the learning process. We also found that some of these learning effects were observed only after memory consolidation the following day. Overall, the learning process modulated the activity in a large network of brain regions, including the superior temporal cortex and the dorsal (parietal) pathway. Most interestingly, middle- and inferior-temporal regions were engaged during multi-sensory memory encoding after the cross-modal relationship was extracted from the learning cues. Our findings highlight the brain dynamics and plasticity related to the learning of letter-speech sound associations and provide a more refined model of grapheme-phoneme learning in reading acquisition.

Active Breaks: A Pilot and Feasibility Study to Evaluate the Effectiveness of Physical Activity Levels in a School Based Intervention in an Italian Primary School

Background: The school gives access to children, regardless of age, ethnicity, gender and socio-economic class and can be identified as the key environment in which to promote children’s physical activity (PA). The guidelines of the European Union recommend accumulating at least 10-min bouts of PA to reach the daily 60 min. Active breaks (ABs) led by teachers inside the classroom represent a good strategy to promote PA. The aim of this pilot and feasibility study was to evaluate the feasibility and effectiveness in terms of PA level of an AB programme in children aged 8–9 years attending primary school. Methods: A pre-post quasi-experimental pilot and feasibility study was performed in two primary school classes, one of which was assigned to a 14-week AB intervention (AB group) and the other to the control group (CG). At baseline and at follow-up, children were monitored for sedentary and motor activity during an entire week using ActiGraph Accelerometer (ActiLife6 wGT3X-BT). The satisfaction of children and teachers was assessed by self-administered questionnaires. Results: In the pre-post comparison, AB group (n = 16) showed a reduction in the minutes spent in weekly sedentary activity (−168.7 min, p > 0.05), an increase in the number of step counts (+14,026.9, p < 0.05) and in time spent in moderate to vigorous PA (MVPA): weekly MVPA: +64.4 min, daily MVPA: +8.05 min, percentage of MVPA: +0.70%. On the contrary, CG showed a worsening in all variables. ANCOVA analysis, after adjusting for baseline values, showed significant differences between the AB group and CG for time spent in MVPA, percentage of MVPA and step counts. The satisfaction of children and teachers was good. Teachers were able to adapt the AB protocol to the needs of the school curriculum, thus confirming the feasibility of the AB programme. Conclusions: This pilot and feasibility study showed the feasibility and effectiveness of the AB protocol and represented the basis for a future controlled trial.

Are two cues always better than one? The role of multiple intra-sensory cues compared to multi-cross-sensory cues in children's incidental category learning

Simultaneous presentation of multisensory cues has been found to facilitate children's learning to a greater extent than unisensory cues (e.g., Broadbent, White, Mareschal, & Kirkham, 2017). Current research into children's multisensory learning, however, does not address whether these findings are due to having multiple cross-sensory cues that enhance stimuli perception or a matter of having multiple cues, regardless of modality, that are informative to category membership. The current study examined the role of multiple cross-sensory cues (e.g., audio-visual) compared to multiple intra-sensory cues (e.g., two visual cues) on children's incidental category learning. On a computerized incidental category learning task, children aged six to ten years (N = 454) were allocated to either a visual-only (V: unisensory), auditory-only (A: unisensory), audio-visual (AV: multisensory), Visual-Visual (VV: multi-cue) or Auditory-Auditory (AA: multi-cue) condition. In children over eight years of age, the availability of two informative cues, regardless of whether they had been presented across two different modalities or within the same modality, was found to be more beneficial to incidental learning than with unisensory cues. In six-year-olds, however, the presence of multiple auditory cues (AA) did not facilitate learning to the same extent as multiple visual cues (VV) or when cues were presented across two different modalities (AV). The findings suggest that multiple sensory cues presented across or within modalities may have differential effects on children's incidental learning across middle childhood, depending on the sensory domain in which they are presented. Implications for the use of multi-cross-sensory and multiple-intra-sensory cues for children's learning across this age range are discussed.

Proprioceptive accuracy in Immersive Virtual Reality: A developmental perspective

Proprioceptive development relies on a variety of sensory inputs, among which vision is hugely dominant. Focusing on the developmental trajectory underpinning the integration of vision and proprioception, the present research explores how this integration is involved in interactions with Immersive Virtual Reality (IVR) by examining how proprioceptive accuracy is affected by Age, Perception, and Environment. Individuals from 4 to 43 years old completed a self-turning task which asked them to manually return to a previous location with different sensory modalities available in both IVR and reality. Results were interpreted from an exploratory perspective using Bayesian model comparison analysis, which allows the phenomena to be described using probabilistic statements rather than simplified reject/not-reject decisions. The most plausible model showed that 4-8-year-old children can generally be expected to make more proprioceptive errors than older children and adults. Across age groups, proprioceptive accuracy is higher when vision is available, and is disrupted in the visual environment provided by the IVR headset. We can conclude that proprioceptive accuracy mostly develops during the first eight years of life and that it relies largely on vision. Moreover, our findings indicate that this proprioceptive accuracy can be disrupted by the use of an IVR headset.

The neural basis of complex audiovisual objects maintenances in working memory

Working memory research has primarily concentrated on studying our senses separately; the neural basis of maintaining information from multiple sensory modalities in working memory has been not well elucidated. It is debated whether multisensory information is maintained in the form of modality-specific representations or amodal representations. The present study investigated what brain regions were engaged in both types of complex audiovisual objects maintenances (semantically congruent and incongruent) using functional magnetic resonance imaging and conjunction analysis, and examined in which form to maintain multisensory objects information in working memory. The conjunction analysis showed that there was common brain regions activation involving left parietal cortex (e.g., left angular gyrus, supramarginal gyrus, and precuneus) while maintaining semantically congruent audiovisual object, whereas the common brain regions activation including the bilateral angular, left superior parietal lobule, and left middle temporal gyrus was found during maintaining semantically incongruent audiovisual objects. Importantly, the shared conjoint brain regions activation consists of bilateral angular gyrus and left middle frontal gyrus was observed while maintaining both types of semantically congruent and incongruent complex audiovisual objects. These brain regions may play different role while maintaining these complex multisensory objects, such as supramodel storage per se and intentional attention. The findings of the present studymight support the amodal view that working memory has a central storage system to maintain multisensory information from different sensory inputs.

Classic psychedelics: the special role of the visual system

Here, we briefly overview the various aspects of classic serotonergic hallucinogens reported by a number of studies. One of the key hypotheses of our paper is that the visual effects of psychedelics might play a key role in resetting fears. Namely, we especially focus on visual processes because they are among the most prominent features of hallucinogen-induced hallucinations. We hypothesize that our brain has an ancient visual-based (preverbal) intrinsic cognitive process that, during the transient inhibition of top-down convergent and abstract thinking (mediated by the prefrontal cortex) by psychedelics, can neutralize emotional fears of unconscious and conscious life experiences from the past. In these processes, the decreased functional integrity of the self-referencing processes of the default mode network, the modified multisensory integration (linked to bodily self-consciousness and self-awareness), and the modified amygdala activity may also play key roles. Moreover, the emotional reset (elimination of stress-related emotions) by psychedelics may induce psychological changes and overwrite the stress-related neuroepigenetic information of past unconscious and conscious emotional fears.

A possible key role of vision in the development of schizophrenia

Based on a brief overview of the various aspects of schizophrenia reported by numerous studies, here we hypothesize that schizophrenia may originate (and in part be performed) from visual areas. In other words, it seems that a normal visual system or at least an evanescent visual perception may be an essential prerequisite for the development of schizophrenia as well as of various types of hallucinations. Our study focuses on auditory and visual hallucinations, as they are the most prominent features of schizophrenic hallucinations (and also the most studied types of hallucinations). Here, we evaluate the possible key role of the visual system in the development of schizophrenia.

Power-spectra and cross-frequency coupling changes in visual and Audio-visual acquired equivalence learning

The three phases of the applied acquired equivalence learning test, i.e. acquisition, retrieval and generalization, investigate the capabilities of humans in associative learning, working memory load and rule-transfer, respectively. Earlier findings denoted the role of different subcortical structures and cortical regions in the visual test. However, there is a lack of information about how multimodal cues modify the EEG-patterns during acquired equivalence learning. To test this we have recorded EEG from 18 healthy volunteers and analyzed the power spectra and the strength of cross-frequency coupling, comparing a unimodal visual-guided and a bimodal, audio-visual-guided paradigm. We found that the changes in the power of the different frequency band oscillations were more critical during the visual paradigm and they showed less synchronized activation compared to the audio-visual paradigm. These findings indicate that multimodal cues require less prominent, but more synchronized cortical contribution, which might be a possible biomarker of forming multimodal associations.

A Multisensory Multilevel Health Education Model for Diverse Communities

Owing to their enormous capacity to improve health and save lives, effective health promotion frameworks have been at the forefront of public health research and practice. A multilevel focus, as exemplified by the Socio-Ecological Model (SEM), is one common denominator among these frameworks. The SEM highlights important social and ecological influences on health behavior by delineating the different levels of influence. These include public policy, organizational, community, interpersonal, and intrapersonal levels, which, when considered during the development of health promotion campaigns, especially those that focus on health education, strengthen the potential influence of that campaign on targeted behaviors. However, the SEM lacks a complimenting framework for understanding the role of conventional and unconventional approaches to health education; that is, how to design a health education intervention that considers both the context, such as the social and ecological levels of influence, and the best approaches for developing and delivering the health education in a manner that optimizes their effectiveness in today's modern and increasingly diverse world. Addressing this gap, the current article presents an integrative Multisensory Multilevel Health Education Model (MMHEM), which incorporates three key domains-(1) Art (innovativeness/creativity), (2) Culture (cultural tailoring), and (3) Science (evidence-based), while promoting the importance of considering the socio-ecological levels of influence on targeted behaviors. Using a successful health education intervention, called the Hip Hop Stroke, we deconstruct the Multisensory Multilevel Health Education Model and discuss its potential role as a guide for developing public health education interventions.

Analyzing the Impact of Soft, Stimulating and Depressing Songs on Attention Among Undergraduate Students: A Cross-Sectional Pilot Study in Bangladesh

Music is strongly linked to attention and giving attention can boost intelligence. The purpose of this study was to scrutinize the impact of soft, stimulating, and depressing songs on the attention of students. The study was performed on 280 undergraduate students. Students were divided into 4 groups (i.e., control, soft, stimulating, and depressing) and subjected to 3 songs, soft (That’s My Name), stimulating (Rain Over Me) and depressing (Broken Angel) songs. The Uddin’s Numeral Finding (NF) and Typo Revealing (TR) tests were used to analyze the attention of the students. In the NF, 75.54% attention was exerted by students subjected to stimulating song followed by soft song’s group (i.e., 74.32%) with respect to control group. Amid all groups, the lowest percentage, 70.77% of attention was reported for students subjected to the depressing song. For TR test, stimulating song’s group exerted highest, 45.97% attention, soft song’s group exerted 45.27%, control group exerted 42.70%, and lowest (i.e., 41.54%) attention was exerted by depressing song’s group. In NF test, concerning sex, amid male and female, male exerted higher (77.04%) attention than female but for TR test female exerted higher (i.e., 48.15%) attention for students subjected to stimulating song. Regarding the age of the study in case of NF test for stimulating song’s group, 18–20 years age students exerted highest, 82.07% attention but for TR test highest, 48.75% attention was reported for 23–25 years age students. For NF test, regarding the age of the study 1st-year student exerted highest, 92.44% attention but for TR test highest, 57.33% attention was reported for 3rd-year students for stimulating song’s group. Concerning residential status in both NF and TR tests, for students lived with family subjected to stimulating song exerted highest, 77.93% and 48.6% attention, respectively with respect to students lived without family and remaining groups. This study suggested that song influences the neuronal circuits linked to alert and cognitive functions and the stimulating song has the acme power of increasing attention while depressing song reduces the attention. Therefore, the exciting song can be an operative intervention for enhancing attention, cognitive functions, and treatment of associated neuropsychological disorders.

Multisensory processing in event-based prospective memory

Failures in prospective memory (PM) - that is, the failure to remember intended future actions - can have adverse consequences. It is therefore important to study those processes that may help to minimize such cognitive failures. Although multisensory integration has been shown to enhance a wide variety of behaviors, including perception, learning, and memory, its effect on prospective memory, in particular, is largely unknown. In the present study, we investigated the effects of multisensory processing on two simultaneously-performed memory tasks: An ongoing 2- or 3-back working memory (WM) task (20% target ratio), and a PM task in which the participants had to respond to a rare predefined letter (8% target ratio). For PM trials, multisensory enhancement was observed for congruent multisensory signals; however, this effect did not generalize to the ongoing WM task. Participants were less likely to make errors for PM than for WM trials, thus suggesting that they may have biased their attention toward the PM task. Multisensory advantages on memory tasks, such as PM and WM, may be dependent on how attention resources are allocated across dual tasks.