Size Isn't All that Matters: Noticing Differences in Size and Temporal Order

Overlapping but Divergent Neural Correlates Underpinning Audiovisual Synchrony and Temporal Order Judgments

Multisensory processing is a core perceptual capability, and the need to understand its neural bases provides a fundamental problem in the study of brain function. Both synchrony and temporal order judgments are commonly used to investigate synchrony perception between different sensory cues and multisensory perception in general. However, extensive behavioral evidence indicates that these tasks do not measure identical perceptual processes. Here we used functional magnetic resonance imaging to investigate how behavioral differences between the tasks are instantiated as neural differences. As these neural differences could manifest at either the sustained (task/state-related) and/or transient (event-related) levels of processing, a mixed block/event-related design was used to investigate the neural response of both time-scales. Clear differences in both sustained and transient BOLD responses were observed between the two tasks, consistent with behavioral differences indeed arising from overlapping but divergent neural mechanisms. Temporal order judgments, but not synchrony judgments, required transient activation in several left hemisphere regions, which may reflect increased task demands caused by an extra stage of processing. Our results highlight that multisensory integration mechanisms can be task dependent, which, in particular, has implications for the study of atypical temporal processing in clinical populations.

Investigating structure and function in the healthy human brain: validity of acute versus chronic lesion-symptom mapping

Modern voxel-based lesion-symptom mapping (VLSM) analyses techniques provide powerful tools to examine the relationship between structure and function of the healthy human brain. However, there is still uncertainty on the type of and the appropriate time point of imaging and of behavioral testing for such analyses. Here we tested the validity of the three most common combinations of structural imaging data and behavioral scores used in VLSM analyses. Given the established knowledge about the neural substrate of the primary motor system in humans, we asked the mundane question of where the motor system is represented in the normal human brain, analyzing individual arm motor function of 60 unselected stroke patients. Only the combination of acute behavioral scores and acute structural imaging precisely identified the principal brain area for the emergence of hemiparesis after stroke, i.e., the corticospinal tract (CST). In contrast, VLSM analyses based on chronic behavior-in combination with either chronic or acute imaging-required the exclusion of patients who had recovered from an initial paresis to reveal valid anatomical results. Thus, if the primary research aim of a VLSM lesion analysis is to uncover the neural substrates of a certain function in the healthy human brain and if no longitudinal designs with repeated evaluations are planned, the combination of acute imaging and behavior represents the ideal dataset.

Human brain basis of musical rhythm perception: common and distinct neural substrates for meter, tempo, and pattern

Rhythm as the time structure of music is composed of distinct temporal components such as pattern, meter, and tempo. Each feature requires different computational processes: meter involves representing repeating cycles of strong and weak beats; pattern involves representing intervals at each local time point which vary in length across segments and are linked hierarchically; and tempo requires representing frequency rates of underlying pulse structures. We explored whether distinct rhythmic elements engage different neural mechanisms by recording brain activity of adult musicians and non-musicians with positron emission tomography (PET) as they made covert same-different discriminations of (a) pairs of rhythmic, monotonic tone sequences representing changes in pattern, tempo, and meter, and (b) pairs of isochronous melodies. Common to pattern, meter, and tempo tasks were focal activities in right, or bilateral, areas of frontal, cingulate, parietal, prefrontal, temporal, and cerebellar cortices. Meter processing alone activated areas in right prefrontal and inferior frontal cortex associated with more cognitive and abstract representations. Pattern processing alone recruited right cortical areas involved in different kinds of auditory processing. Tempo processing alone engaged mechanisms subserving somatosensory and premotor information (e.g., posterior insula, postcentral gyrus). Melody produced activity different from the rhythm conditions (e.g., right anterior insula and various cerebellar areas). These exploratory findings suggest the outlines of some distinct neural components underlying the components of rhythmic structure.

Common spatial organization of number and emotional expression: a mental magnitude line

Converging behavioral and neural evidence suggests that numerical representations are mentally organized in left-to-right orientation. Here we show that this format of spatial organization extends to emotional expression. In Experiment 1, right-side responses became increasingly faster as number (represented by Arabic numerals) or happiness (depicted in facial stimuli) increased, for judgments completely unrelated to magnitude. Additional experiments suggest that magnitude (i.e., more/less relations), not valence (i.e., positive/negative), underlies left-to-right orientation of emotional expression (Experiment 2), and that this orientation accommodates to the context-relevant emotion (e.g., happier faces are more rightward when judged on happiness, but more leftward when judged on angriness; Experiment 3). These findings show that people automatically extract magnitude from a variety of stimuli, representing such information in common left-to-right format, perhaps reflecting a mental magnitude line. We suggest that number is but one dimension in a hyper-general representational system uniting disparate dimensions of magnitude and likely subserved by common neural mechanisms in posterior parietal cortex.