Cross-modal representations in early visual and auditory cortices revealed by multi-voxel pattern analysis

Whisker stimulation with different frequencies reveals non-uniform modulation of functional magnetic resonance imaging signal across sensory systems in awake rats

Primary sensory systems are classically considered to be separate units, however there is current evidence that there are notable interactions between them. We examined the cross-sensory interplay by applying a quiet and motion-tolerant zero echo time functional magnetic resonance imaging (fMRI) technique to elucidate the evoked brain-wide responses to whisker pad stimulation in awake and anesthetized rats. Specifically, characterized the brain-wide responses in core and non-core regions to whisker pad stimulation by the varying stimulation-frequency, and determined whether isoflurane-medetomidine anesthesia, traditionally used in preclinical imaging, confounded investigations related to sensory integration. We demonstrated that unilateral whisker pad stimulation not only elicited robust activity along the whisker-mediated tactile system, but also in auditory, visual, high-order, and cerebellar regions, indicative of brain-wide cross-sensory and associative activity. By inspecting the response profiles to different stimulation frequencies and temporal signal characteristics, we observed that the non-core regions responded to stimulation in a very different way compared to the primary sensory system, likely reflecting different encoding modes between the primary sensory, cross-sensory, and integrative processing. Lastly, while the activity evoked in low-order sensory structures could be reliably detected under anesthesia, the activity in high-order processing and the complex differences between primary, cross-sensory, and associative systems were visible only in the awake state. We conclude that our study reveals novel aspects of the cross-sensory interplay of whisker-mediated tactile system, and importantly, that these would be difficult to observe in anesthetized rats.

Representational similarity analysis of self- versus other-processing: Effect of trait aggressiveness

In this study, using the self/other adjective judgment task, we aimed to explore how people perceive themselves in comparison to various other people, including friends, strangers, and those they dislike. Next, using representational similarity analysis, we sought to elucidate how these perceptual similarities and differences are represented in brain activity and how aggressiveness is related to these representations. Behavioral ratings show that, on average, people tend to consider themselves more like their friends than neutral strangers, and least like people they dislike. This pattern of similarity is positively correlated with neural representation in social and cognitive circuits of the brain and negatively correlated with neural representation in emotional centers that may represent emotional arousal associated with various social objects. Aggressiveness seems to predispose a person to a pattern of behavior that is the opposite of the average pattern, that is, a tendency to think of oneself as less like one's friends and more like one's enemies. This corresponds to an increase in the similarity of the behavioral representation with the representation in the emotional centers and a decrease in its similarity with the representation in the social and cognitive centers. This can be seen as evidence that in individuals prone to aggression, behavior in the social environment may depend to a greater extent on the representation of social objects in the emotional rather than social and cognitive brain circuits.

Spoken language processing activates the primary visual cortex

Primary visual cortex (V1) is generally thought of as a low-level sensory area that primarily processes basic visual features. Although there is evidence for multisensory effects on its activity, these are typically found for the processing of simple sounds and their properties, for example spatially or temporally-congruent simple sounds. However, in congenitally blind individuals, V1 is involved in language processing, with no evidence of major changes in anatomical connectivity that could explain this seemingly drastic functional change. This is at odds with current accounts of neural plasticity, which emphasize the role of connectivity and conserved function in determining a neural tissue's role even after atypical early experiences. To reconcile what appears to be unprecedented functional reorganization with known accounts of plasticity limitations, we tested whether V1's multisensory roles include responses to spoken language in sighted individuals. Using fMRI, we found that V1 in normally sighted individuals was indeed activated by comprehensible spoken sentences as compared to an incomprehensible reversed speech control condition, and more strongly so in the left compared to the right hemisphere. Activation in V1 for language was also significant and comparable for abstract and concrete words, suggesting it was not driven by visual imagery. Last, this activation did not stem from increased attention to the auditory onset of words, nor was it correlated with attentional arousal ratings, making general attention accounts an unlikely explanation. Together these findings suggest that V1 responds to spoken language even in sighted individuals, reflecting the binding of multisensory high-level signals, potentially to predict visual input. This capability might be the basis for the strong V1 language activation observed in people born blind, re-affirming the notion that plasticity is guided by pre-existing connectivity and abilities in the typically developed brain.

Decoding sounds depicting hand-object interactions in primary somatosensory cortex

Neurons, even in the earliest sensory regions of cortex, are subject to a great deal of contextual influences from both within and across modality connections. Recent work has shown that primary sensory areas can respond to and, in some cases, discriminate stimuli that are not of their target modality: for example, primary somatosensory cortex (SI) discriminates visual images of graspable objects. In the present work, we investigated whether SI would discriminate sounds depicting hand-object interactions (e.g. bouncing a ball). In a rapid event-related functional magnetic resonance imaging experiment, participants listened attentively to sounds from 3 categories: hand-object interactions, and control categories of pure tones and animal vocalizations, while performing a one-back repetition detection task. Multivoxel pattern analysis revealed significant decoding of hand-object interaction sounds within SI, but not for either control category. Crucially, in the hand-sensitive voxels defined from an independent tactile localizer, decoding accuracies were significantly higher for hand-object interactions compared to pure tones in left SI. Our findings indicate that simply hearing sounds depicting familiar hand-object interactions elicit different patterns of activity in SI, despite the complete absence of tactile stimulation. These results highlight the rich contextual information that can be transmitted across sensory modalities even to primary sensory areas.

Confidence modulates the decodability of scene prediction during partially-observable maze exploration in humans

Prediction ability often involves some degree of uncertainty-a key determinant of confidence. Here, we sought to assess whether predictions are decodable in partially-observable environments where one's state is uncertain, and whether this information is sensitive to confidence produced by such uncertainty. We used functional magnetic resonance imaging-based, partially-observable maze navigation tasks in which subjects predicted upcoming scenes and reported their confidence regarding these predictions. Using a multi-voxel pattern analysis, we successfully decoded both scene predictions and subjective confidence from activities in the localized parietal and prefrontal regions. We also assessed confidence in their beliefs about where they were in the maze. Importantly, prediction decodability varied according to subjective scene confidence in the superior parietal lobule and state confidence estimated by the behavioral model in the inferior parietal lobule. These results demonstrate that prediction in uncertain environments depends on the prefrontal-parietal network within which prediction and confidence interact.

Neural interactions in occipitotemporal cortex during basic human movement perception by dynamic causal modeling

Action recognition is an essential component of our daily life. The occipitotemporal cortex (OTC) is an important area in human movement perception. The previous studies have revealed that three vital regions including the extrastriate body area (EBA), human middle temporal complex (hMT+), and posterior superior temporal sulcus (pSTS) in OTC play an important role in motion perception. The aim of the current study is to explore the neural interactions between these three regions during basic human movement perception. Functional magnetic resonance imaging data were acquired when participants viewed dynamic videos depicting basic human movements. By the dynamic causal modeling analysis, a model space consisting of 576 models was constructed and evaluated to select the optimal model given the data. The information of the visual movement was found to enter the system through hMT+. We speculated that hMT+ would be the region to show sensitivity to the presence of motion and it subsequently influence and be influenced by the other two regions. Our results also revealed the manner in which the three regions interact during action recognition. Furthermore, We found significantly enhanced modulated connectivity from hMT+ to both EBA and pSTS, as well as from EBA to both hMT+ and pSTS. We inferred that there may be multiple routes for human action perception. One responsible route for processing motion signals is through hMT+ to pSTS, and the other projects information to pSTS may be via the form-processing route. In addition, pSTS may integrate and mediate visual signals and possibly convey them to distributed areas to maintain high-order cognitive tasks.