Temporal synchrony effects of optic flow and vestibular inputs on multisensory heading perception

Multisensory mechanisms of gait and balance in Parkinson's disease: an integrative review

Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have on our aging population. Posture and gait control does not happen automatically, as previously believed, but rather requires continuous involvement of central nervous mechanisms. To effectively exert control over the body, the brain must integrate multiple streams of sensory information, including visual, vestibular, and somatosensory signals. The mechanisms which underpin the integration of these multisensory signals are the principal topic of the present work. Existing multisensory integration theories focus on how failure of cognitive processes thought to be involved in multisensory integration leads to falls in older adults. Insufficient emphasis, however, has been placed on specific contributions of individual sensory modalities to multisensory integration processes and cross-modal interactions that occur between the sensory modalities in relation to gait and balance. In the present work, we review the contributions of somatosensory, visual, and vestibular modalities, along with their multisensory intersections to gait and balance in older adults and patients with Parkinson's disease. We also review evidence of vestibular contributions to multisensory temporal binding windows, previously shown to be highly pertinent to fall risk in older adults. Lastly, we relate multisensory vestibular mechanisms to potential neural substrates, both at the level of neurobiology (concerning positron emission tomography imaging) and at the level of electrophysiology (concerning electroencephalography). We hope that this integrative review, drawing influence across multiple subdisciplines of neuroscience, paves the way for novel research directions and therapeutic neuromodulatory approaches, to improve the lives of older adults and patients with neurodegenerative diseases.

Nanomaterials for Flexible Neuromorphics

The quest to imbue machines with intelligence akin to that of humans, through the development of adaptable neuromorphic devices and the creation of artificial neural systems, has long stood as a pivotal goal in both scientific inquiry and industrial advancement. Recent advancements in flexible neuromorphic electronics primarily rely on nanomaterials and polymers owing to their inherent uniformity, superior mechanical and electrical capabilities, and versatile functionalities. However, this field is still in its nascent stage, necessitating continuous efforts in materials innovation and device/system design. Therefore, it is imperative to conduct an extensive and comprehensive analysis to summarize current progress. This review highlights the advancements and applications of flexible neuromorphics, involving inorganic nanomaterials (zero-/one-/two-dimensional, and heterostructure), carbon-based nanomaterials such as carbon nanotubes (CNTs) and graphene, and polymers. Additionally, a comprehensive comparison and summary of the structural compositions, design strategies, key performance, and significant applications of these devices are provided. Furthermore, the challenges and future directions pertaining to materials/devices/systems associated with flexible neuromorphics are also addressed. The aim of this review is to shed light on the rapidly growing field of flexible neuromorphics, attract experts from diverse disciplines (e.g., electronics, materials science, neurobiology), and foster further innovation for its accelerated development.

From Multisensory Integration to Multisensory Decision-Making

Organisms live in a dynamic environment in which sensory information from multiple sources is ever changing. A conceptually complex task for the organisms is to accumulate evidence across sensory modalities and over time, a process known as multisensory decision-making. This is a new concept, in terms of that previous researches have been largely conducted in parallel disciplines. That is, much efforts have been put either in sensory integration across modalities using activity summed over a duration of time, or in decision-making with only one sensory modality that evolves over time. Recently, a few studies with neurophysiological measurements emerge to study how different sensory modality information is processed, accumulated, and integrated over time in decision-related areas such as the parietal or frontal lobes in mammals. In this review, we summarize and comment on these studies that combine the long-existed two parallel fields of multisensory integration and decision-making. We show how the new findings provide insight into our understanding about neural mechanisms mediating multisensory information processing in a more complete way.

Simulated vestibular spatial disorientation mouse model under coupled rotation revealing potential involvement of Slc17a6

Spatial disorientation (SD) is the main contributor to flight safety risks, but research progress in animals has been limited, impeding a deeper understanding of the underlying mechanisms of SD. This study proposed a method for constructing and evaluating a vestibular SD mouse model, which adopted coupled rotational stimulation with visual occlusion. Physiological parameters were measured alongside behavioral indices to assess the model, and neuronal changes were observed through immunofluorescent staining. The evaluation of the model involved observing decreased colonic temperature and increased arterial blood pressure in mice exposed to SD, along with notable impairments in motor and cognitive function. Our investigation unveiled that vestibular SD stimulation elicited neuronal activation in spatially associated cerebral areas, such as the hippocampus. Furthermore, transcriptomic sequencing and bioinformatics analysis revealed the potential involvement of Slc17a6 in the mechanism of SD. These findings lay a foundation for further investigation into the molecular mechanisms underlying SD.

Visuo-vestibular heading perception: a model system to study multi-sensory decision making

Integrating noisy signals across time as well as sensory modalities, a process named multi-sensory decision making (MSDM), is an essential strategy for making more accurate and sensitive decisions in complex environments. Although this field is just emerging, recent extraordinary works from different perspectives, including computational theory, psychophysical behaviour and neurophysiology, begin to shed new light onto MSDM. In the current review, we focus on MSDM by using a model system of visuo-vestibular heading. Combining well-controlled behavioural paradigms on virtual-reality systems, single-unit recordings, causal manipulations and computational theory based on spiking activity, recent progress reveals that vestibular signals contain complex temporal dynamics in many brain regions, including unisensory, multi-sensory and sensory-motor association areas. This challenges the brain for cue integration across time and sensory modality such as optic flow which mainly contains a motion velocity signal. In addition, new evidence from the higher-level decision-related areas, mostly in the posterior and frontal/prefrontal regions, helps revise our conventional thought on how signals from different sensory modalities may be processed, converged, and moment-by-moment accumulated through neural circuits for forming a unified, optimal perceptual decision. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Self-motion perception and sequential decision-making: where are we heading?

To navigate and guide adaptive behaviour in a dynamic environment, animals must accurately estimate their own motion relative to the external world. This is a fundamentally multisensory process involving integration of visual, vestibular and kinesthetic inputs. Ideal observer models, paired with careful neurophysiological investigation, helped to reveal how visual and vestibular signals are combined to support perception of linear self-motion direction, or heading. Recent work has extended these findings by emphasizing the dimension of time, both with regard to stimulus dynamics and the trade-off between speed and accuracy. Both time and certainty-i.e. the degree of confidence in a multisensory decision-are essential to the ecological goals of the system: terminating a decision process is necessary for timely action, and predicting one's accuracy is critical for making multiple decisions in a sequence, as in navigation. Here, we summarize a leading model for multisensory decision-making, then show how the model can be extended to study confidence in heading discrimination. Lastly, we preview ongoing efforts to bridge self-motion perception and navigation per se, including closed-loop virtual reality and active self-motion. The design of unconstrained, ethologically inspired tasks, accompanied by large-scale neural recordings, raise promise for a deeper understanding of spatial perception and decision-making in the behaving animal. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Temporal and spatial properties of vestibular signals for perception of self-motion

It is well recognized that the vestibular system is involved in numerous important cognitive functions, including self-motion perception, spatial orientation, locomotion, and vector-based navigation, in addition to basic reflexes, such as oculomotor or body postural control. Consistent with this rationale, vestibular signals exist broadly in the brain, including several regions of the cerebral cortex, potentially allowing tight coordination with other sensory systems to improve the accuracy and precision of perception or action during self-motion. Recent neurophysiological studies in animal models based on single-cell resolution indicate that vestibular signals exhibit complex spatiotemporal dynamics, producing challenges in identifying their exact functions and how they are integrated with other modality signals. For example, vestibular and optic flow could provide congruent and incongruent signals regarding spatial tuning functions, reference frames, and temporal dynamics. Comprehensive studies, including behavioral tasks, neural recording across sensory and sensory-motor association areas, and causal link manipulations, have provided some insights into the neural mechanisms underlying multisensory self-motion perception.

Segregation and integration of sensory features by flexible temporal characteristics of independent neural representations

Segregation and integration are two fundamental yet competing computations in cognition. For example, in serial speech processing, stable perception necessitates the sequential establishment of perceptual representations to remove irrelevant features for achieving invariance. Whereas multiple features need to combine to create a coherent percept. How to simultaneously achieve seemingly contradicted computations of segregation and integration in a serial process is unclear. To investigate their neural mechanisms, we used loudness and lexical tones as a research model and employed a novel multilevel oddball paradigm with Electroencephalogram (EEG) recordings to explore the dynamics of mismatch negativity (MMN) responses to their deviants. When two types of deviants were presented separately, distinct topographies of MMNs to loudness and tones were observed at different latencies (loudness earlier), supporting the sequential dynamics of independent representations for two features. When they changed simultaneously, the latency of responses to tones became shorter and aligned with that to loudness, while the topographies remained independent, yielding the combined MMN as a linear additive of single MMNs of loudness and tones. These results suggest that neural dynamics can be temporally synchronized to distinct sensory features and balance the computational demands of segregation and integration, grounding for invariance and feature binding in serial processing.

Mouse frontal cortex mediates additive multisensory decisions

The brain can combine auditory and visual information to localize objects. However, the cortical substrates underlying audiovisual integration remain uncertain. Here, we show that mouse frontal cortex combines auditory and visual evidence; that this combination is additive, mirroring behavior; and that it evolves with learning. We trained mice in an audiovisual localization task. Inactivating frontal cortex impaired responses to either sensory modality, while inactivating visual or parietal cortex affected only visual stimuli. Recordings from >14,000 neurons indicated that after task learning, activity in the anterior part of frontal area MOs (secondary motor cortex) additively encodes visual and auditory signals, consistent with the mice's behavioral strategy. An accumulator model applied to these sensory representations reproduced the observed choices and reaction times. These results suggest that frontal cortex adapts through learning to combine evidence across sensory cortices, providing a signal that is transformed into a binary decision by a downstream accumulator.

Cholinergic system correlates of postural control changes in Parkinson's disease freezers

Postural instability and freezing of gait are the most debilitating dopamine-refractory motor impairments in advanced stages of Parkinson's disease because of increased risk of falls and poorer quality of life. Recent findings suggest an inability to efficaciously utilize vestibular information during static posturography among people with Parkinson's disease who exhibit freezing of gait, with associated changes in cholinergic system integrity as assessed by vesicular acetylcholine transporter PET. There is a lack of adequate understanding of how postural control varies as a function of available sensory information in patients with Parkinson's disease with freezing of gait. The goal of this cross-sectional study was to examine cerebral cholinergic system changes that associate with inter-sensory postural control processing features as assessed by dynamic computerized posturography and acetylcholinesterase PET. Seventy-five participants with Parkinson's disease, 16 of whom exhibited freezing of gait, underwent computerized posturography on the NeuroCom© Equitest sensory organization test platform, striatal dopamine, and acetylcholinesterase PET scanning. Findings demonstrated that patients with Parkinson's disease with freezing of gait have greater difficulty maintaining balance in the absence of reliable proprioceptive cues as compared to those without freezing of gait [β = 0.28 (0.021, 0.54), P = 0.034], an effect that was independent of disease severity [β = 0.16 (0.062, 0.26), P < 0.01] and age [β = 0.092 (-0.005, 0.19), P = 0.062]. Exploratory voxel-based analysis revealed an association between postural control and right hemispheric cholinergic network related to visual-vestibular integration and self-motion perception. High anti-cholinergic burden predicted postural control impairment in a manner dependent on right hemispheric cortical cholinergic integrity [β = 0.34 (0.065, 0.61), P < 0.01]. Our findings advance the perspective that cortical cholinergic system might play a role in supporting postural control after nigro-striatal dopaminergic losses in Parkinson's disease. Failure of cortex-dependent visual-vestibular integration may impair detection of postural instability in absence of reliable proprioceptive cues. Better understanding of how the cholinergic system plays a role in this process may augur novel treatments and therapeutic interventions to ameliorate debilitating symptoms in patients with advanced Parkinson's disease.

PtNPs/PEDOT:PSS-Modified Microelectrode Arrays for Detection of the Discharge of Head Direction Cells in the Retrosplenial Cortex of Rats under Dissociation between Visual and Vestibular Inputs

The electrophysiological activities of head direction (HD) cells under visual and vestibular input dissociation are important to understanding the formation of the sense of direction in animals. In this paper, we fabricated a PtNPs/PEDOT:PSS-modified MEA to detect changes in the discharge of HD cells under dissociated sensory conditions. The electrode shape was customized for the retrosplenial cortex (RSC) and was conducive to the sequential detection of neurons at different depths in vivo when combined with a microdriver. The recording sites of the electrode were modified with PtNPs/PEDOT:PSS to form a three-dimensional convex structure, leading to closer contact with neurons and improving the detection performance and signal-to-noise ratio of the MEA. We designed a rotating cylindrical arena to separate the visual and vestibular information of the rats and detected the changes in the directional tuning of the HD cells in the RSC. The results showed that after visual and vestibular sensory dissociation, HD cells used visual information to establish newly discharged directions which differed from the original direction. However, with the longer time required to process inconsistent sensory information, the function of the HD system gradually degraded. After recovery, the HD cells reverted to their newly established direction rather than the original direction. The research based on our MEAs revealed how HD cells process dissociated sensory information and contributes to the study of the spatial cognitive navigation mechanism.

F = ma. Is the macaque brain Newtonian?

Intuitive Physics, the ability to anticipate how the physical events involving mass objects unfold in time and space, is a central component of intelligent systems. Intuitive physics is a promising tool for gaining insight into mechanisms that generalize across species because both humans and non-human primates are subject to the same physical constraints when engaging with the environment. Physical reasoning abilities are widely present within the animal kingdom, but monkeys, with acute 3D vision and a high level of dexterity, appreciate and manipulate the physical world in much the same way humans do.

Mammalian-brain-inspired neuromorphic motion-cognition nerve achieves cross-modal perceptual enhancement

Perceptual enhancement of neural and behavioral response due to combinations of multisensory stimuli are found in many animal species across different sensory modalities. By mimicking the multisensory integration of ocular-vestibular cues for enhanced spatial perception in macaques, a bioinspired motion-cognition nerve based on a flexible multisensory neuromorphic device is demonstrated. A fast, scalable and solution-processed fabrication strategy is developed to prepare a nanoparticle-doped two-dimensional (2D)-nanoflake thin film, exhibiting superior electrostatic gating capability and charge-carrier mobility. The multi-input neuromorphic device fabricated using this thin film shows history-dependent plasticity, stable linear modulation, and spatiotemporal integration capability. These characteristics ensure parallel, efficient processing of bimodal motion signals encoded as spikes and assigned with different perceptual weights. Motion-cognition function is realized by classifying the motion types using mean firing rates of encoded spikes and postsynaptic current of the device. Demonstrations of recognition of human activity types and drone flight modes reveal that the motion-cognition performance match the bio-plausible principles of perceptual enhancement by multisensory integration. Our system can be potentially applied in sensory robotics and smart wearables.

Cortical Mechanisms of Multisensory Linear Self-motion Perception

Accurate self-motion perception, which is critical for organisms to survive, is a process involving multiple sensory cues. The two most powerful cues are visual (optic flow) and vestibular (inertial motion). Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction, often in a statistically Bayesian-optimal way. In the last decade, single-unit recordings in awake, behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution, giving insight into possible neural mechanisms underlying multisensory self-motion perception. Here, we review these findings, along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities. We show that, in light of new data, conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged. We propose that different temporal component signals may mediate different functions, a possibility that requires future studies.