Information-based functional brain mapping

Category-Selective Representation of Relationships in the Visual Cortex

Understanding social interaction requires processing social agents and their relationships. The latest results show that much of this process is visually solved: visual areas can represent multiple people encoding emergent information about their interaction that is not explained by the response to the individuals alone. A neural signature of this process is an increased response in visual areas, to face-to-face (seemingly interacting) people, relative to people presented as unrelated (back-to-back). This effect highlighted a network of visual areas for representing relational information. How is this network organized? Using functional MRI, we measured the brain activity of healthy female and male humans (N = 42), in response to images of two faces or two (head-blurred) bodies, facing toward or away from each other. Taking the facing > non-facing effect as a signature of relation perception, we found that relations between faces and between bodies were coded in distinct areas, mirroring the categorical representation of faces and bodies in the visual cortex. Additional analyses suggest the existence of a third network encoding relations between (nonsocial) objects. Finally, a separate occipitotemporal network showed the generalization of relational information across body, face, and nonsocial object dyads (multivariate pattern classification analysis), revealing shared properties of relations across categories. In sum, beyond single entities, the visual cortex encodes the relations that bind multiple entities into relationships; it does so in a category-selective fashion, thus respecting a general organizing principle of representation in high-level vision. Visual areas encoding visual relational information can reveal the processing of emergent properties of social (and nonsocial) interaction, which trigger inferential processes.

A neurocomputational account of the link between social perception and social action

People selectively help others based on perceptions of their merit or need. Here, we develop a neurocomputational account of how these social perceptions translate into social choice. Using a novel fMRI social perception task, we show that both merit and need perceptions recruited the brain's social inference network. A behavioral computational model identified two non-exclusive mechanisms underlying variance in social perceptions: a consistent tendency to perceive others as meritorious/needy (bias) and a propensity to sample and integrate normative evidence distinguishing high from low merit/need in other people (sensitivity). Variance in people's merit (but not need) bias and sensitivity independently predicted distinct aspects of altruism in a social choice task completed months later. An individual's merit bias predicted context-independent variance in people's overall other-regard during altruistic choice, biasing people towards prosocial actions. An individual's merit sensitivity predicted context-sensitive discrimination in generosity towards high and low merit recipients by influencing other-regard and self-regard during altruistic decision-making. This context-sensitive perception-action link was associated with activation in the right temporoparietal junction. Together, these findings point towards stable, biologically based individual differences in perceptual processes related to abstract social concepts like merit, and suggest that these differences may have important behavioral implications for an individual's tendency toward favoritism or discrimination in social settings.

Tomatoes Are Red: The Perception of Achromatic Objects Elicits Retrieval of Associated Color Knowledge

When preparing to name an object, semantic knowledge about the object and its attributes is activated, including perceptual properties. It is unclear, however, whether semantic attribute activation contributes to lexical access or is a consequence of activating a concept irrespective of whether that concept is to be named or not. In this study, we measured neural responses using fMRI while participants named objects that are typically green or red, presented in black line drawings. Furthermore, participants underwent two other tasks with the same objects, color naming and semantic judgment, to see if the activation pattern we observe during picture naming is (a) similar to that of a task that requires accessing the color attribute and (b) distinct from that of a task that requires accessing the concept but not its name or color. We used representational similarity analysis to detect brain areas that show similar patterns within the same color category, but show different patterns across the two color categories. In all three tasks, activation in the bilateral fusiform gyri ("Human V4") correlated with a representational model encoding the red-green distinction weighted by the importance of color feature for the different objects. This result suggests that when seeing objects whose color attribute is highly diagnostic, color knowledge about the objects is retrieved irrespective of whether the color or the object itself have to be named.

Representational similarity analysis of self- versus other-processing: Effect of trait aggressiveness

In this study, using the self/other adjective judgment task, we aimed to explore how people perceive themselves in comparison to various other people, including friends, strangers, and those they dislike. Next, using representational similarity analysis, we sought to elucidate how these perceptual similarities and differences are represented in brain activity and how aggressiveness is related to these representations. Behavioral ratings show that, on average, people tend to consider themselves more like their friends than neutral strangers, and least like people they dislike. This pattern of similarity is positively correlated with neural representation in social and cognitive circuits of the brain and negatively correlated with neural representation in emotional centers that may represent emotional arousal associated with various social objects. Aggressiveness seems to predispose a person to a pattern of behavior that is the opposite of the average pattern, that is, a tendency to think of oneself as less like one's friends and more like one's enemies. This corresponds to an increase in the similarity of the behavioral representation with the representation in the emotional centers and a decrease in its similarity with the representation in the social and cognitive centers. This can be seen as evidence that in individuals prone to aggression, behavior in the social environment may depend to a greater extent on the representation of social objects in the emotional rather than social and cognitive brain circuits.

Familiarity Processing through Faces and Names: Insights from Multivoxel Pattern Analysis

The way our brain processes personal familiarity is still debatable. We used searchlight multivoxel pattern analysis (MVPA) to identify areas where local fMRI patterns could contribute to familiarity detection for both faces and name categories. Significantly, we identified cortical areas in frontal, temporal, cingulate, and insular areas, where it is possible to accurately cross-classify familiar stimuli from one category using a classifier trained with the stimulus from the other (i.e., abstract familiarity) based on local fMRI patterns. We also discovered several areas in the fusiform gyrus, frontal, and temporal regions-primarily lateralized to the right hemisphere-supporting the classification of familiar faces but failing to do so for names. Also, responses to familiar names (compared to unfamiliar names) consistently showed less activation strength than responses to familiar faces (compared to unfamiliar faces). The results evinced a set of abstract familiarity areas (independent of the stimulus type) and regions specifically related only to face familiarity, contributing to recognizing familiar individuals.

BackWards - Unveiling the brain's topographic organization of paraspinal sensory input

Cortical reorganization and its potential pathological significance are being increasingly studied in musculoskeletal disorders such as chronic low back pain (CLBP) patients. However, detailed sensory-topographic maps of the human back are lacking, and a baseline characterization of such representations, reflecting the somatosensory organization of the healthy back, is needed before exploring potential sensory map reorganization. To this end, a novel pneumatic vibrotactile stimulation method was used to stimulate paraspinal sensory afferents, while studying their cortical representations in unprecedented detail. In 41 young healthy participants, vibrotactile stimulations at 20 Hz and 80 Hz were applied bilaterally at nine locations along the thoracolumbar axis while functional magnetic resonance imaging (fMRI) was performed. Model-based whole-brain searchlight representational similarity analysis (RSA) was used to investigate the organizational structure of brain activity patterns evoked by thoracolumbar sensory inputs. A model based on segmental distances best explained the similarity structure of brain activity patterns that were located in different areas of sensorimotor cortices, including the primary somatosensory and motor cortices and parts of the superior parietal cortex, suggesting that these brain areas process sensory input from the back in a "dermatomal" manner. The current findings provide a sound basis for testing the "cortical map reorganization theory" and its pathological relevance in CLBP.

Recurrent connectivity supports higher-level visual and semantic object representations in the brain

Visual object recognition has been traditionally conceptualised as a predominantly feedforward process through the ventral visual pathway. While feedforward artificial neural networks (ANNs) can achieve human-level classification on some image-labelling tasks, it's unclear whether computational models of vision alone can accurately capture the evolving spatiotemporal neural dynamics. Here, we probe these dynamics using a combination of representational similarity and connectivity analyses of fMRI and MEG data recorded during the recognition of familiar, unambiguous objects. Modelling the visual and semantic properties of our stimuli using an artificial neural network as well as a semantic feature model, we find that unique aspects of the neural architecture and connectivity dynamics relate to visual and semantic object properties. Critically, we show that recurrent processing between the anterior and posterior ventral temporal cortex relates to higher-level visual properties prior to semantic object properties, in addition to semantic-related feedback from the frontal lobe to the ventral temporal lobe between 250 and 500 ms after stimulus onset. These results demonstrate the distinct contributions made by semantic object properties in explaining neural activity and connectivity, highlighting it as a core part of object recognition not fully accounted for by current biologically inspired neural networks.

Hydroxyurea maintains working memory function in pediatric sickle cell disease

Pediatric patients with sickle cell disease (SCD) have decreased oxygen-carrying capacity in the blood and reduced or restricted cerebral blood flow resulting in neurocognitive deficits and cerebral infarcts. The standard treatment for children with SCD is hydroxyurea; however, the treatment-related neurocognitive effects are unclear. A key area of impairment in SCD is working memory, which is implicated in other cognitive and academic skills. N-back tasks are commonly used to investigate neural correlates of working memory. We analyzed functional magnetic resonance imaging (fMRI) of patients with SCD while they performed n-back tasks by assessing the blood-oxygenation level-dependent (BOLD) signals during working memory processing. Twenty hydroxyurea-treated and 11 control pediatric patients with SCD (7-18 years old) performed 0-, 1-, and 2-back tasks at 2 time points, once before hydroxyurea treatment (baseline) and ~1 year after treatment (follow-up). Neurocognitive measures (e.g., verbal comprehension, processing speed, full-scale intelligence quotient, etc.) were assessed at both time points. Although no significant changes in behavior performance of n-back tasks and neurocognitive measures were observed in the treated group, we observed a treatment-by-time interaction in the right cuneus and angular gyrus for the 2- > 0-back contrast. Through searchlight-pattern classifications in the treated and control groups to identify changes in brain activation between time points during the 2-back task, we found more brain areas, especially the posterior region, with changes in the pattern and magnitude of BOLD signals in the control group compared to the treated group. In the control group, increases in 2-back BOLD signals were observed in the right crus I cerebellum, right inferior parietal lobe, right inferior temporal lobe, right angular gyrus, left cuneus and left middle frontal gyrus at 1-year follow-up. Moreover, BOLD signals elevated as the working memory load increased from 0- to 1-back but did not increase further from 1- to 2-back in the right inferior temporal lobe, right angular gyrus, and right superior frontal gyrus. These observations may result from increased cognitive effort during working memory processing with no hydroxyurea treatment. In contrast, we found fewer changes in the pattern and magnitude of BOLD signals across time points in the treated group. Furthermore, BOLD signals in the left crus I cerebellum, right angular gyrus, left cuneus and right superior frontal gyrus of the treated group increased continuously with increasing working memory load from 0- to 2-back, potentially related to a broader dynamic range in response to task difficulty and cognitive effort. Collectively, these findings suggest that hydroxyurea treatment helped maintain working memory function in SCD.

Neurocomputational Substrates Underlying the Effect of Identifiability on Third-Party Punishment

The identifiable target effect refers to the preference for helping identified victims and punishing identifiable perpetrators compared with equivalent but unidentifiable counterparts. The identifiable target effect is often attributed to the heightened moral emotions evoked by identified targets. However, the specific neurocognitive processes that mediate and/or modulate this effect remain largely unknown. Here, we combined a third-party punishment game with brain imaging and computational modeling to unravel the neurocomputational underpinnings of the identifiable transgressor effect. Human participants (males and females) acted as bystanders and punished identified or anonymous wrongdoers. Participants were more punitive toward identified wrongdoers than anonymous wrongdoers because they took a vicarious perspective of victims and adopted lower reference points of inequity (i.e., more stringent norms) in the identified context than in the unidentified context. Accordingly, there were larger activity of the ventral anterior insula, more distinct multivariate neural patterns in the dorsal anterior insula and dorsal anterior cingulate cortex, and lower strength between ventral anterior insula and dorsolateral PFC and between dorsal anterior insula and ventral striatum connectivity in response to identified transgressors than anonymous transgressors. These findings implicate the interplay of expectancy violations, emotions, and self-interest in the identifiability effect. Last, individual differences in the identifiability effect were associated with empathic concern/social dominance orientation, activity in the precuneus/cuneus and temporo-parietal junction, and intrinsic functional connectivity of the dorsolateral PFC. Together, our work is the first to uncover the neurocomputational processes mediating identifiable transgressor effect and to characterize psychophysiological profiles modulating the effect.SIGNIFICANCE STATEMENT The identifiable target effect, more help to identified victims or stronger punishment to identifiable perpetrators, is common in daily life. We examined the neurocomputational mechanisms mediating/modulating the identifiability effect on third-party punishment by bridging literature from economics and cognitive neuroscience. Our findings reveal that identifiable transgressor effect is mediated by lower reference points of inequity (i.e., more stringent norms), which might be associated with a stronger involvement of the emotion processes and a weaker engagement of the analytic/deliberate processes. Furthermore, personality traits, altered brain activity, and intrinsic functional connectivity contribute to the individual variance in the identifiability effect. Overall, our study advances the understanding of the identifiability effect by shedding light on its component processes and modulating factors.

The individualized neural tuning model: Precise and generalizable cartography of functional architecture in individual brains

Quantifying how brain functional architecture differs from person to person is a key challenge in human neuroscience. Current individualized models of brain functional organization are based on brain regions and networks, limiting their use in studying fine-grained vertex-level differences. In this work, we present the individualized neural tuning (INT) model, a fine-grained individualized model of brain functional organization. The INT model is designed to have vertex-level granularity, to capture both representational and topographic differences, and to model stimulus-general neural tuning. Through a series of analyses, we demonstrate that (a) our INT model provides a reliable individualized measure of fine-grained brain functional organization, (b) it accurately predicts individualized brain response patterns to new stimuli, and (c) for many benchmarks, it requires only 10-20 minutes of data for good performance. The high reliability, specificity, precision, and generalizability of our INT model affords new opportunities for building brain-based biomarkers based on naturalistic neuroimaging paradigms.

Encoding of continuous perceptual choices in human early visual cortex

Introduction

Research on the neural mechanisms of perceptual decision-making has typically focused on simple categorical choices, say between two alternative motion directions. Studies on such discrete alternatives have often suggested that choices are encoded either in a motor-based or in an abstract, categorical format in regions beyond sensory cortex.

Methods

In this study, we used motion stimuli that could vary anywhere between 0° and 360° to assess how the brain encodes choices for features that span the full sensory continuum. We employed a combination of neuroimaging and encoding models based on Gaussian process regression to assess how either stimuli or choices were encoded in brain responses.

Results

We found that single-voxel tuning patterns could be used to reconstruct the trial-by-trial physical direction of motion as well as the participants' continuous choices. Importantly, these continuous choice signals were primarily observed in early visual areas. The tuning properties in this region generalized between choice encoding and stimulus encoding, even for reports that reflected pure guessing.

Discussion

We found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.

Disrupted network interactions serve as a neural marker of dyslexia

Dyslexia, a frequent learning disorder, is characterized by severe impairments in reading and writing and hypoactivation in reading regions in the left hemisphere. Despite decades of research, it remains unclear to date if observed behavioural deficits are caused by aberrant network interactions during reading and whether differences in functional activation and connectivity are directly related to reading performance. Here we provide a comprehensive characterization of reading-related brain connectivity in adults with and without dyslexia. We find disrupted functional coupling between hypoactive reading regions, especially between the left temporo-parietal and occipito-temporal cortices, and an extensive functional disruption of the right cerebellum in adults with dyslexia. Network analyses suggest that individuals with dyslexia process written stimuli via a dorsal decoding route and show stronger reading-related interaction with the right cerebellum. Moreover, increased connectivity within networks is linked to worse reading performance in dyslexia. Collectively, our results provide strong evidence for aberrant task-related connectivity as a neural marker for dyslexia that directly impacts behavioural performance. The observed differences in activation and connectivity suggest that one effective way to alleviate reading problems in dyslexia is through modulating interactions within the reading network with neurostimulation methods.

Adaptive short-term plasticity in the typical reading network

The left temporo-parietal cortex (TPC) is crucial for phonological decoding, i.e., for learning and retaining sound-letter mappings, and appears hypoactive in dyslexia. Here, we tested the causal contribution of this area for reading in typical readers with transcranial magnetic stimulation (TMS) and explored the reading network's response with fMRI. By investigating the underlying neural correlates of stimulation-induced modulations of the reading network, we can help improve targeted interventions for individuals with dyslexia. 28 typical adult readers overtly read simple and complex words and pseudowords during fMRI after effective and sham TMS over the left TPC. To explore differences in functional activation and effective connectivity within the reading network, we performed univariate and multivariate analyses, as well as dynamic causal modeling. While TMS-induced effects on reading performance and brain activation showed large individual variability, multivariate analyses revealed a shift in activation in the left inferior frontal cortex for pseudoword reading after effective TMS. Furthermore, TMS increased effective connectivity from the left ventral occipito-temporal cortex to the left TPC. In the absence of effects on reading performance, the observed changes in task-related activity and the increase in functional coupling between the two core reading nodes suggest successful short-term compensatory reorganization in the reading network following TMS-induced disruption. This study is the first to explore neurophysiological changes induced by TMS to a core reading node in typical readers while performing an overt reading task. We provide evidence for remote stimulation effects and emphasize the relevance of functional interactions in the reading network.

A systematic review of neuroimaging approaches to mapping language in individuals

Although researchers often rely on group-level fMRI results to draw conclusions about the neurobiology of language, doing so without accounting for the complexities of individual brains may reduce the validity of our findings. Furthermore, understanding brain organization in individuals is critically important for both basic science and clinical translation. To assess the state of single-subject language localization in the functional neuroimaging literature, we carried out a systematic review of studies published through April 2020. Out of 977 papers identified through our search, 121 met our inclusion criteria for reporting single-subject fMRI results (fMRI studies of language in adults that report task-based single-subject statistics). Of these, 20 papers reported using a single-subject test-retest analysis to assess reliability. Thus, we found that a relatively modest number of papers reporting single-subject results quantified single-subject reliability. These varied substantially in acquisition parameters, task design, and reliability measures, creating significant challenges for making comparisons across studies. Future endeavors to optimize the localization of language networks in individuals will benefit from the standardization and broader reporting of reliability metrics for different tasks and acquisition parameters.

Spatiotemporal dynamics of self-generated imagery reveal a reverse cortical hierarchy from cue-induced imagery

Visual imagery allows for the construction of rich internal experience in our mental world. However, it has remained poorly understood how imagery experience derives volitionally as opposed to being cue driven. Here, using electroencephalography and functional magnetic resonance imaging, we systematically investigate the spatiotemporal dynamics of self-generated imagery by having participants volitionally imagining one of the orientations from a learned pool. We contrast self-generated imagery with cue-induced imagery, where participants imagined line orientations based on associative cues acquired previously. Our results reveal overlapping neural signatures of cue-induced and self-generated imagery. Yet, these neural signatures display substantially differential sensitivities to the two types of imagery: self-generated imagery is supported by an enhanced involvement of the anterior cortex in representing imagery contents. By contrast, cue-induced imagery is supported by enhanced imagery representations in the posterior visual cortex. These results jointly support a reverse cortical hierarchy in generating and maintaining imagery contents in self-generated versus externally cued imagery.

Generative replay underlies compositional inference in the hippocampal-prefrontal circuit

Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.

Disturbance of information in superior parietal lobe during dual-task interference in a simulated driving task

Performing a secondary task while driving causes a decline in driving performance. This phenomenon, called dual-task interference, can have lethal consequences. Previous fMRI studies have looked at the changes in the average brain activity to uncover the neural correlates of dual-task interference. From these results, it is unclear whether the overall modulations in brain activity result from general effects such as task difficulty, attentional modulations, and mental effort or whether it is caused by a change in the responses specific to each condition due to dual-task interference. To overcome this limitation, here, we used multi-voxel pattern analysis (MVPA) to interrogate the change in the information content in multiple brain regions during dual-task interference in simulated driving. Participants performed a lane-change task in a simulated driving environment, along with a tone discrimination task with either short or long onset time difference (Stimulus Onset Asynchrony, SOA) between the two tasks. Behavioral results indicated a robust dual-task effect on lane-change reaction time (RT). MVPA revealed regions that carry information about the driving lane-change direction (shift right/shift left), including the superior parietal lobe (SPL), visual, and motor regions. Comparison of decoding accuracies across SOA conditions in the SPL region revealed lower accuracy in the short compared to the long SOA condition. This change in accuracy was not observed in the visual and motor regions. These findings suggest that the dual-task interference in driving may be related to the disturbance of information processing in the SPL region.

Disruptions of salience network during uncertain anticipation of conflict control in anxiety

BACKGROUND

Anxiety has been characterized by disrupted processing of conflict control, while little is known about anticipatory processing of conflicts in anxiety. Anticipation is the key factor in both anxiety and cognitive control, especially under uncertain conditions. The current study therefore examined neurocomputational mechanisms of uncertain anticipation of conflict control in anxiety.

METHODS

Twenty-six participants with high-trait anxiety and twenty-nine low-trait anxiety participants completed a cue-flanker task with functional magnetic resonance imaging. The hierarchical drift diffusion model (HDDM) was used to measure the cognitive computations during the task. To identify the neurocomputational mechanism of anticipatory control in anxiety, mediation analysis and dynamic causal modelling (DCM) analysis were conducted to examine the relationship between functional connectivity of brain networks and the parameters of HDDM.

RESULTS

We found influences of regulatory signals from the dorsolateral prefrontal cortex to dorsal anterior cingulate cortex on decision threshold in low-trait anxiety (LTA), but not in high-trait anxiety (HTA), especially for the condition with uncertain cues. The results indicate deficient top-down anticipatory control of upcoming conflicts in anxious individuals. DCM and HDDM analyses revealed that lower decision threshold was associated with higher intrinsic connectivity of salience network (SN) in anxious individuals, suggesting that dysfunctional SN disrupts anticipation of conflict control under uncertainty in anxiety.

CONCLUSIONS

Our results suggest hyperfunction of the SN underlies the deficient information accumulation during uncertain anticipation of upcoming conflicts in anxiety. Our findings shed new light on the mechanisms of anticipation processing and the psychopathology of anxiety.

Higher subjective socioeconomic status is linked to increased charitable giving and mentalizing-related neural value coding

Socioeconomic status (SES), a concept related to an individual's economic and social position relative to others, can shape social interactions like altruistic behaviors. However, little is known about the exact neurocognitive mechanisms that link SES with altruism. Our study aimed to provide a comprehensive account of the sociocognitive and neural mechanisms through which SES affects charitable giving - an important variant of human altruism. To this end, participants completed a charitable donation task while their brain activity was measured using functional magnetic resonance imaging (fMRI). We also assessed participants' socio-cognitive ability to infer other people's mental states (i.e., mentalizing) - a driver of prosocial behavior - in an independent social task. Behaviorally, we found that both charitable giving and social cognition were status-dependent, as subjective SES positively predicted donations and mentalizing capacity. Moreover, the link between SES and charitable giving was mediated by individuals' mentalizing capacity. At the neural level, a multivariate pattern analysis of fMRI data revealed that higher subjective SES was associated with stronger value coding in the right temporoparietal junction (rTPJ). The strength of this value representation predicted charitable giving and was linked to mentalizing. Furthermore, we observed an increased negative functional coupling between rTPJ and left putamen with higher SES. Together, increased charitable giving in higher-status individuals could be explained by status-dependent recruitment of mentalizing-related value coding and altered functional connectivity in the brain. Our findings provide insights into the socio- and neurocognitive mechanisms explaining why and when higher SES leads to prosociality, which might ultimately inform targeted interventions to promote prosocial behavior in human societies.

A functional account of stimulation-based aerobic glycolysis and its role in interpreting BOLD signal intensity increases in neuroimaging experiments

In aerobic glycolysis, oxygen is abundant, and yet cells metabolize glucose without using it, decreasing their ATP per glucose yield by 15-fold. During task-based stimulation, aerobic glycolysis occurs in localized brain regions, presenting a puzzle: why produce ATP inefficiently when, all else being equal, evolution should favor the efficient use of metabolic resources? The answer is that all else is not equal. We propose that a tradeoff exists between efficient ATP production and the efficiency with which ATP is spent to transmit information. Aerobic glycolysis, despite yielding little ATP per glucose, may support neuronal signaling in thin (< 0.5 µm), information-efficient axons. We call this the efficiency tradeoff hypothesis. This tradeoff has potential implications for interpretations of task-related BOLD "activation" observed in fMRI. We hypothesize that BOLD "activation" may index local increases in aerobic glycolysis, which support signaling in thin axons carrying "bottom-up" information, or "prediction error"-i.e., the BIAPEM (BOLD increases approximate prediction error metabolism) hypothesis. Finally, we explore implications of our hypotheses for human brain evolution, social behavior, and mental disorders.

Reading Braille by Touch Recruits Posterior Parietal Cortex

Blind readers use a tactile reading system consisting of raised dot arrays: braille/⠃⠗⠇. How do human brains implement reading by touch? The current study looked for signatures of reading-specific orthographic processes in braille, separate from low-level somatosensory responses and semantic processes. Of specific interest were responses in posterior parietal cortices (PPCs), because of their role in high-level tactile perception. Congenitally blind, proficient braille readers read real words and pseudowords by touch while undergoing fMRI. We leveraged the system of contractions in English braille, where one braille cell can represent multiple English print letters (e.g., "ing" ⠬, "one" ⠐⠕), making it possible to separate physical and orthographic word length. All words in the study consisted of four braille cells, but their corresponding Roman letter spellings varied from four to seven letters (e.g., "con-c-er-t" ⠒⠉⠻⠞. contracted: four cells; uncontracted: seven letters). We found that the bilateral supramarginal gyrus in the PPC increased its activity as the uncontracted word length increased. By contrast, in the hand region of primary somatosensory cortex (S1), activity increased as a function of a low-level somatosensory feature: dot-number per word. The PPC also showed greater response to pseudowords than real words and distinguished between real and pseudowords in multivariate-pattern analysis. Parieto-occipital, early visual and ventral occipito-temporal, as well as prefrontal cortices also showed sensitivity to the real-versus-pseudoword distinction. We conclude that PPC is involved in orthographic processing for braille, that is, braille character and word recognition, possibly because of braille's tactile modality.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Conceptual representations in the default, control and attention networks are task-dependent and cross-modal

Conceptual knowledge is central to human cognition. Neuroimaging studies suggest that conceptual processing involves modality-specific and multimodal brain regions in a task-dependent fashion. However, it remains unclear (1) to what extent conceptual feature representations are also modulated by the task, (2) whether conceptual representations in multimodal regions are indeed cross-modal, and (3) how the conceptual system relates to the large-scale functional brain networks. To address these issues, we conducted multivariate pattern analyses on fMRI data. 40 participants performed three tasks-lexical decision, sound judgment, and action judgment-on written words. We found that (1) conceptual feature representations are strongly modulated by the task, (2) conceptual representations in several multimodal regions are cross-modal, and (3) conceptual feature retrieval involves the default, frontoparietal control, and dorsal attention networks. Conceptual representations in these large-scale networks are task-dependent and cross-modal. Our findings support theories that assume conceptual processing to rely on a flexible, multi-level architecture.

Changes in functional brain activity patterns associated with computer programming learning in novices

BACKGROUND

Computer programming, the process of designing, writing, and testing executable computer code, is an essential skill in numerous fields. A description of the neural structures engaged and modified during programming skill acquisition could help improve training programs and provide clues to the neural substrates underlying the acquisition of related skills.

METHODS

Fourteen female university students without prior computer programing experience were examined by functional magnetic resonance imaging (fMRI) during the early and late stages of a 5-month 'Computer Processing' course. Brain regions involved in task performance and learning were identified by comparing responses to programming and control tasks during the early and late stages.

RESULTS

The accuracy of performing a programming task was significantly improved during the late stage. Various regions of the frontal, temporal, parietal, and occipital cortex as well as several subcortical structures (caudate nuclei and cerebellum) were activated during programming tasks. Brain activity in the right inferior frontal gyrus was greater during the late stage and significantly correlated with improved task performance. Although the left inferior frontal gyrus was also highly active during the programming task, there were no learning-induced changes in activity or a significant correlation between activity and improved task performances.

CONCLUSION

Computer programming learning among novices induces functional neuroplasticity within the right inferior frontal gyrus but not the left inferior gyrus (Broca's area).

Multivariate fMRI responses in superior temporal cortex predict visual contributions to, and individual differences in, the intelligibility of noisy speech

Humans have the unique ability to decode the rapid stream of language elements that constitute speech, even when it is contaminated by noise. Two reliable observations about noisy speech perception are that seeing the face of the talker improves intelligibility and the existence of individual differences in the ability to perceive noisy speech. We introduce a multivariate BOLD fMRI measure that explains both observations. In two independent fMRI studies, clear and noisy speech was presented in visual, auditory and audiovisual formats to thirty-seven participants who rated intelligibility. An event-related design was used to sort noisy speech trials by their intelligibility. Individual-differences multidimensional scaling was applied to fMRI response patterns in superior temporal cortex and the dissimilarity between responses to clear speech and noisy (but intelligible) speech was measured. Neural dissimilarity was less for audiovisual speech than auditory-only speech, corresponding to the greater intelligibility of noisy audiovisual speech. Dissimilarity was less in participants with better noisy speech perception, corresponding to individual differences. These relationships held for both single word and entire sentence stimuli, suggesting that they were driven by intelligibility rather than the specific stimuli tested. A neural measure of perceptual intelligibility may aid in the development of strategies for helping those with impaired speech perception.

Causal inference in cognitive neuroscience

Causal inference is a key step in many research endeavors in cognitive science and neuroscience, and particularly cognitive neuroscience. Statistical knowledge is sufficient for prediction and diagnosis, but causal knowledge is required for action and intervention. Most statistics courses and textbooks emphasize the difficulty of causal inference, focusing on the maxim that "correlation does not mean causation": there can be multiple causal possibilities, often many of them, consistent with given observed statistics. This paper focuses instead on the conceptual issues and assumptions that confront causal and other kinds of inference, primarily focusing on cognitive neuroscience. We connect inference methods with goals and challenges, and provide concrete guidance about how to select appropriate tools for the scientific task. This article is categorized under: Psychology > Theory and Methods Philosophy > Foundations of Cognitive Science.

The PyMVPA BIDS-App: a robust multivariate pattern analysis pipeline for fMRI data

With the advent of multivariate pattern analysis (MVPA) as an important analytic approach to fMRI, new insights into the functional organization of the brain have emerged. Several software packages have been developed to perform MVPA analysis, but deploying them comes with the cost of adjusting data to individual idiosyncrasies associated with each package. Here we describe PyMVPA BIDS-App, a fast and robust pipeline based on the data organization of the BIDS standard that performs multivariate analyses using powerful functionality of PyMVPA. The app runs flexibly with blocked and event-related fMRI experimental designs, is capable of performing classification as well as representational similarity analysis, and works both within regions of interest or on the whole brain through searchlights. In addition, the app accepts as input both volumetric and surface-based data. Inspections into the intermediate stages of the analyses are available and the readability of final results are facilitated through visualizations. The PyMVPA BIDS-App is designed to be accessible to novice users, while also offering more control to experts through command-line arguments in a highly reproducible environment.

Statistical inference on representational geometries

Neuroscience has recently made much progress, expanding the complexity of both neural activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium-imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io).

Neural representations of anxiety in adolescents with anorexia nervosa: a multivariate approach

Anorexia nervosa (AN) is characterized by low body weight, fear of gaining weight, and distorted body image. Anxiety may play a role in the formation and course of the illness, especially related to situations involving food, eating, weight, and body image. To understand distributed patterns and consistency of neural responses related to anxiety, we enrolled 25 female adolescents with AN and 22 non-clinical female adolescents with mild anxiety who underwent two fMRI sessions in which they saw personalized anxiety-provoking word stimuli and neutral words. Consistency in brain response patterns across trials was determined using a multivariate representational similarity analysis (RSA) approach within anxiety circuits and in a whole-brain voxel-wise searchlight analysis. In the AN group there was higher representational similarity for anxiety-provoking compared with neutral stimuli predominantly in prefrontal regions including the frontal pole, medial prefrontal cortex, dorsolateral prefrontal cortex, and medial orbitofrontal cortex, although no significant group differences. Severity of anxiety correlated with consistency of brain responses within anxiety circuits and in cortical and subcortical regions including the frontal pole, middle frontal gyrus, orbitofrontal cortex, thalamus, lateral occipital cortex, middle temporal gyrus, and cerebellum. Higher consistency of activation in those with more severe anxiety symptoms suggests the possibility of a greater degree of conditioned brain responses evoked by personally-relevant emotional stimuli. Anxiety elicited by disorder-related stimuli may activate stereotyped, previously-learned neural responses within- and outside of classical anxiety circuits. Results have implications for understanding consistent and automatic responding to environmental stimuli that may play a role in maintenance of AN.

Objects sharpen visual scene representations: evidence from MEG decoding

Real-world scenes consist of objects, defined by local information, and scene background, defined by global information. Although objects and scenes are processed in separate pathways in visual cortex, their processing interacts. Specifically, previous studies have shown that scene context makes blurry objects look sharper, an effect that can be observed as a sharpening of object representations in visual cortex from around 300 ms after stimulus onset. Here, we use MEG to show that objects can also sharpen scene representations, with the same temporal profile. Photographs of indoor (closed) and outdoor (open) scenes were blurred such that they were difficult to categorize on their own but easily disambiguated by the inclusion of an object. Classifiers were trained to distinguish MEG response patterns to intact indoor and outdoor scenes, presented in an independent run, and tested on degraded scenes in the main experiment. Results revealed better decoding of scenes with objects than scenes alone and objects alone from 300 ms after stimulus onset. This effect was strongest over left posterior sensors. These findings show that the influence of objects on scene representations occurs at similar latencies as the influence of scenes on object representations, in line with a common predictive processing mechanism.

Atypical cognitive training-induced learning and brain plasticity and their relation to insistence on sameness in children with autism

Children with autism spectrum disorders (ASDs) often display atypical learning styles; however, little is known regarding learning-related brain plasticity and its relation to clinical phenotypic features. Here, we investigate cognitive learning and neural plasticity using functional brain imaging and a novel numerical problem-solving training protocol. Children with ASD showed comparable learning relative to typically developing children but were less likely to shift from rule-based to memory-based strategy. While learning gains in typically developing children were associated with greater plasticity of neural representations in the medial temporal lobe and intraparietal sulcus, learning in children with ASD was associated with more stable neural representations. Crucially, the relation between learning and plasticity of neural representations was moderated by insistence on sameness, a core phenotypic feature of ASD. Our study uncovers atypical cognitive and neural mechanisms underlying learning in children with ASD, and informs pedagogical strategies for nurturing cognitive abilities in childhood autism.

Dynamic spatial coding in parietal cortex mediates tactile-motor transformation

Movements towards touch on the body require integrating tactile location and body posture information. Tactile processing and movement planning both rely on posterior parietal cortex (PPC) but their interplay is not understood. Here, human participants received tactile stimuli on their crossed and uncrossed feet, dissociating stimulus location relative to anatomy versus external space. Participants pointed to the touch or the equivalent location on the other foot, which dissociates sensory and motor locations. Multi-voxel pattern analysis of concurrently recorded fMRI signals revealed that tactile location was coded anatomically in anterior PPC but spatially in posterior PPC during sensory processing. After movement instructions were specified, PPC exclusively represented the movement goal in space, in regions associated with visuo-motor planning and with regional overlap for sensory, rule-related, and movement coding. Thus, PPC flexibly updates its spatial codes to accommodate rule-based transformation of sensory input to generate movement to environment and own body alike.

Multi-modal representation of the size of space in the human brain

To estimate the size of an indoor space, we must analyze the visual boundaries that limit the spatial extent and acoustic cues from reflected interior surfaces. We used fMRI to examine how the brain processes geometric size of indoor scenes when various types of sensory cues are presented individually or together. Specifically, we asked whether the size of space is represented in a modality-specific way or in an integrative way that combines multimodal cues. In a block-design study, images or sounds that depict small and large sized indoor spaces were presented. Visual stimuli were real-world pictures of empty spaces that were small or large. Auditory stimuli were sounds convolved with different reverberation. By using a multi-voxel pattern classifier, we asked whether the two sizes of space can be classified in visual, auditory, and visual-auditory combined conditions. We identified both sensory specific and multimodal representations of the size of space. To further investigate the nature of the multimodal region, we specifically examined whether it contained multimodal information in a coexistent or integrated form. We found that AG and the right IFG pars opercularis had modality-integrated representation, displaying sensitivity to the match in the spatial size information conveyed through image and sound. Background functional connectivity analysis further demonstrated that the connection between sensory specific regions and modality-integrated regions increase in the multimodal condition compared to single modality conditions. Our results suggest that the spatial size perception relies on both sensory specific and multimodal representations, as well as their interplay during multimodal perception.

Neural signatures of social inferences predict the number of real-life social contacts and autism severity

We regularly infer other people's thoughts and feelings from observing their actions, but how this ability contributes to successful social behavior and interactions remains unknown. We show that neural activation patterns during social inferences obtained in the laboratory predict the number of social contacts in the real world, as measured by the social network index, in three neurotypical samples (total n = 126) and one sample of autistic adults (n = 23). We also show that brain patterns during social inference generalize across individuals in these groups. Cross-validated associations between brain activations and social inference localize selectively to the right posterior superior temporal sulcus and were specific for social, but not nonsocial, inference. Activation within this same brain region also predicts autism-like trait scores from questionnaires and autism symptom severity. Thus, neural activations produced while thinking about other people's mental states predict variance in multiple indices of social functioning in the real world.

Similar object shape representation encoded in the inferolateral occipitotemporal cortex of sighted and early blind people

We can sense an object's shape by vision or touch. Previous studies suggested that the inferolateral occipitotemporal cortex (ILOTC) implements supramodal shape representations as it responds more to seeing or touching objects than shapeless textures. However, such activation in the anterior portion of the ventral visual pathway could be due to the conceptual representation of an object or visual imagery triggered by touching an object. We addressed these possibilities by directly comparing shape and conceptual representations of objects in early blind (who lack visual experience/imagery) and sighted participants. We found that bilateral ILOTC in both groups showed stronger activation during a shape verification task than during a conceptual verification task made on the names of the same manmade objects. Moreover, the distributed activity in the ILOTC encoded shape similarity but not conceptual association among objects. Besides the ILOTC, we also found shape representation in both groups' bilateral ventral premotor cortices and intraparietal sulcus (IPS), a frontoparietal circuit relating to object grasping and haptic processing. In contrast, the conceptual verification task activated both groups' left perisylvian brain network relating to language processing and, interestingly, the cuneus in early blind participants only. The ILOTC had stronger functional connectivity to the frontoparietal circuit than to the left perisylvian network, forming a modular structure specialized in shape representation. Our results conclusively support that the ILOTC selectively implements shape representation independently of visual experience, and this unique functionality likely comes from its privileged connection to the frontoparietal haptic circuit.

Neural Evidence for Boundary Updating as the Source of the Repulsive Bias in Classification

Binary classification, an act of sorting items into two classes by setting a boundary, is biased by recent history. One common form of such bias is repulsive bias, a tendency to sort an item into the class opposite to its preceding items. Sensory-adaptation and boundary-updating are considered as two contending sources of the repulsive bias, yet no neural support has been provided for either source. Here, we explored human brains of both men and women, using functional magnetic resonance imaging (fMRI), to find such support by relating the brain signals of sensory-adaptation and boundary-updating to human classification behavior. We found that the stimulus-encoding signal in the early visual cortex adapted to previous stimuli, yet its adaptation-related changes were dissociated from current choices. Contrastingly, the boundary-representing signals in the inferior-parietal and superior-temporal cortices shifted to previous stimuli and covaried with current choices. Our exploration points to boundary-updating, rather than sensory-adaptation, as the origin of the repulsive bias in binary classification.SIGNIFICANCE STATEMENT Many animal and human studies on perceptual decision-making have reported an intriguing history effect called "repulsive bias," a tendency to classify an item as the opposite class of its previous item. Regarding the origin of repulsive bias, two contending ideas have been proposed: "bias in stimulus representation because of sensory adaptation" versus "bias in class-boundary setting because of belief updating." By conducting model-based neuroimaging experiments, we verified their predictions about which brain signal should contribute to the trial-to-trial variability in choice behavior. We found that the brain signal of class boundary, but not stimulus representation, contributed to the choice variability associated with repulsive bias. Our study provides the first neural evidence supporting the boundary-based hypothesis of repulsive bias.

Connectopic mapping techniques do not reflect functional gradients in the brain

Functional gradients, in which response properties change gradually across a brain region, have been proposed as a key organising principle of the brain. Recent studies using both resting-state and natural viewing paradigms have indicated that these gradients may be reconstructed from functional connectivity patterns via "connectopic mapping" analyses. However, local connectivity patterns may be confounded by spatial autocorrelations artificially introduced during data analysis, for instance by spatial smoothing or interpolation between coordinate spaces. Here, we investigate whether such confounds can produce illusory connectopic gradients. We generated datasets comprising random white noise in subjects' functional volume spaces, then optionally applied spatial smoothing and/or interpolated the data to a different volume or surface space. Both smoothing and interpolation induced spatial autocorrelations sufficient for connectopic mapping to produce both volume- and surface-based local gradients in numerous brain regions. Furthermore, these gradients appeared highly similar to those obtained from real natural viewing data, although gradients generated from real and random data were statistically different in certain scenarios. We also reconstructed global gradients across the whole-brain - while these appeared less susceptible to artificial spatial autocorrelations, the ability to reproduce previously reported gradients was closely linked to specific features of the analysis pipeline. These results indicate that previously reported gradients identified by connectopic mapping techniques may be confounded by artificial spatial autocorrelations introduced during the analysis, and in some cases may reproduce poorly across different analysis pipelines. These findings imply that connectopic gradients need to be interpreted with caution.

The effect of context and reason on the neural correlates of intentions

Many studies have identified networks in parietal and prefrontal cortex that are involved in intentional action. Yet, our understanding of the way these networks are involved in intentions is still very limited. In this study, we investigate two characteristics of these processes: context- and reason-dependence of the neural states associated with intentions. We ask whether these states depend on the context a person is in and the reasons they have for choosing an action. We used a combination of functional magnetic resonance imaging (fMRI) and multivariate decoding to directly assess the context- and reason-dependency of the neural states underlying intentions. We show that action intentions can be decoded from fMRI data based on a classifier trained in the same context and with the same reason, in line with previous decoding studies. Furthermore, we found that intentions can be decoded across different reasons for choosing an action. However, decoding across different contexts was not successful. We found anecdotal to moderate evidence against context-invariant information in all regions of interest and for all conditions but one. These results suggest that the neural states associated with intentions are modulated by the context of the action.

Benchmarking explanation methods for mental state decoding with deep learning models

Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., experiencing anger or joy) and brain activity by identifying those spatial and temporal features of brain activity that allow to accurately identify (i.e., decode) these states. Once a DL model has been trained to accurately decode a set of mental states, neuroimaging researchers often make use of methods from explainable artificial intelligence research to understand the model's learned mappings between mental states and brain activity. Here, we benchmark prominent explanation methods in a mental state decoding analysis of multiple functional Magnetic Resonance Imaging (fMRI) datasets. Our findings demonstrate a gradient between two key characteristics of an explanation in mental state decoding, namely, its faithfulness and its alignment with other empirical evidence on the mapping between brain activity and decoded mental state: explanation methods with high explanation faithfulness, which capture the model's decision process well, generally provide explanations that align less well with other empirical evidence than the explanations of methods with less faithfulness. Based on our findings, we provide guidance for neuroimaging researchers on how to choose an explanation method to gain insight into the mental state decoding decisions of DL models.

Temporal lobe perceptual predictions for speech are instantiated in motor cortex and reconciled by inferior frontal cortex

Humans use predictions to improve speech perception, especially in noisy environments. Here we use 7-T functional MRI (fMRI) to decode brain representations of written phonological predictions and degraded speech signals in healthy humans and people with selective frontal neurodegeneration (non-fluent variant primary progressive aphasia [nfvPPA]). Multivariate analyses of item-specific patterns of neural activation indicate dissimilar representations of verified and violated predictions in left inferior frontal gyrus, suggestive of processing by distinct neural populations. In contrast, precentral gyrus represents a combination of phonological information and weighted prediction error. In the presence of intact temporal cortex, frontal neurodegeneration results in inflexible predictions. This manifests neurally as a failure to suppress incorrect predictions in anterior superior temporal gyrus and reduced stability of phonological representations in precentral gyrus. We propose a tripartite speech perception network in which inferior frontal gyrus supports prediction reconciliation in echoic memory, and precentral gyrus invokes a motor model to instantiate and refine perceptual predictions for speech.

Dual-stream cortical pathways mediate sensory prediction

Predictions are constantly generated from diverse sources to optimize cognitive functions in the ever-changing environment. However, the neural origin and generation process of top-down induced prediction remain elusive. We hypothesized that motor-based and memory-based predictions are mediated by distinct descending networks from motor and memory systems to the sensory cortices. Using functional magnetic resonance imaging (fMRI) and a dual imagery paradigm, we found that motor and memory upstream systems activated the auditory cortex in a content-specific manner. Moreover, the inferior and posterior parts of the parietal lobe differentially relayed predictive signals in motor-to-sensory and memory-to-sensory networks. Dynamic causal modeling of directed connectivity revealed selective enabling and modulation of connections that mediate top-down sensory prediction and ground the distinctive neurocognitive basis of predictive processing.

Variational relevance evaluation of individual fMRI data enables deconstruction of task-dependent neural dynamics

In neuroimaging research, univariate analysis has always been used to localize "representations" at the microscale, whereas network approaches have been applied to characterize transregional "operations". How are representations and operations linked through dynamic interactions? We developed the variational relevance evaluation (VRE) method to analyze individual task fMRI data, which selects informative voxels during model training to localize the "representation", and quantifies the dynamic contributions of single voxels across the whole-brain to different cognitive functions to characterize the "operation". Using 15 individual fMRI data files for higher visual area localizers, we evaluated the characterization of selected voxel positions of VRE and revealed different object-selective regions functioning in similar dynamics. Using another 15 individual fMRI data files for memory retrieval after offline learning, we found similar task-related regions working in different neural dynamics for tasks with diverse familiarities. VRE demonstrates a promising horizon in individual fMRI research.

Decoding of Working Memory Contents in Auditory Cortex Is Not Distractor-Resistant

Working memory enables the temporary storage of relevant information in the service of behavior. Neuroimaging studies have suggested that sensory cortex is involved in maintaining contents in working memory. This raised the question of how sensory regions maintain memory representations during the exposure to distracting stimuli. Multivariate pattern analysis of fMRI signals in visual cortex has shown that the contents of visual working memory could be decoded concurrently with passively viewed distractors. The present fMRI study tested whether this finding extends to auditory working memory and to active distractor processing. We asked participants to memorize the pitch of a target sound and to compare it with a probe sound presented after a 13 s delay period. In separate conditions, we compared a blank delay phase (no distraction) with either passive listening to, or active processing of, an auditory distractor presented throughout the memory delay. Consistent with previous reports, pitch-specific memory information could be decoded in auditory cortex during the delay in trials without distraction. In contrast, decoding of target sounds in early auditory cortex dropped to chance level during both passive and active distraction. This was paralleled by memory performance decrements under distraction. Extending the analyses beyond sensory cortex yielded some evidence for memory content-specific activity in inferior frontal and superior parietal cortex during active distraction. In summary, while our findings question the involvement of early auditory cortex in the maintenance of distractor-resistant working memory contents, further research should elucidate the role of hierarchically higher regions.SIGNIFICANCE STATEMENT Information about sensory features held in working memory can be read out from hemodynamic activity recorded in human sensory cortices. Moreover, visual cortex can in parallel store visual content and process newly incoming, task-irrelevant visual input. The present study investigated the role of auditory cortex for working memory maintenance under distraction. While memorized sound frequencies could be decoded in auditory cortex in the absence of distraction, auditory distraction during the delay phase impaired memory performance and prevented decoding of information stored in working memory. Apparently, early auditory cortex is not sufficient to represent working memory contents under distraction that impairs performance. However, exploratory analyses indicated that, under distraction, higher-order frontal and parietal regions might contribute to content-specific working memory storage.

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Behavioral reports of sensory information are biased by stimulus history. The nature and direction of such serial-dependence biases can differ between experimental settings; both attractive and repulsive biases toward previous stimuli have been observed. How and when these biases arise in the human brain remains largely unexplored. They could occur either via a change in sensory processing itself and/or during postperceptual processes such as maintenance or decision-making. To address this, we tested 20 participants (11 female) and analyzed behavioral and magnetoencephalographic (MEG) data from a working-memory task in which participants were sequentially presented with two randomly oriented gratings, one of which was cued for recall at the end of the trial. Behavioral responses showed evidence for two distinct biases: (1) a within-trial repulsive bias away from the previously encoded orientation on the same trial, and (2) a between-trial attractive bias toward the task-relevant orientation on the previous trial. Multivariate classification of stimulus orientation revealed that neural representations during stimulus encoding were biased away from the previous grating orientation, regardless of whether we considered the within-trial or between-trial prior orientation, despite opposite effects on behavior. These results suggest that repulsive biases occur at the level of sensory processing and can be overridden at postperceptual stages to result in attractive biases in behavior.SIGNIFICANCE STATEMENT Recent experience biases behavioral reports of sensory information, possibly capitalizing on the temporal regularity in our environment. It is still unclear at what stage of stimulus processing such serial biases arise. Here, we recorded behavior and neurophysiological [magnetoencephalographic (MEG)] data to test whether neural activity patterns during early sensory processing show the same biases seen in participants' reports. In a working-memory task that produced multiple biases in behavior, responses were biased toward previous targets, but away from more recent stimuli. Neural activity patterns were uniformly biased away from all previously relevant items. Our results contradict proposals that all serial biases arise at an early sensory processing stage. Instead, neural activity exhibited mostly adaptation-like responses to recent stimuli.

A common neural code for representing imagined and inferred tastes

Inferences about the taste of foods are a key aspect of our everyday experience of food choice. Despite this, gustatory mental imagery is a relatively under-studied aspect of our mental lives. In the present study, we examined subjects during high-field fMRI as they actively imagined basic tastes and subsequently viewed pictures of foods dominant in those specific taste qualities. Imagined tastes elicited activity in the bilateral dorsal mid-insula, one of the primary cortical regions responsive to the experience of taste. In addition, within this region we reliably decoded imagined tastes according to their dominant quality - sweet, sour, or salty - thus indicating that, like actual taste, imagined taste activates distinct quality-specific neural patterns. Using a cross-task decoding analysis, we found that the neural patterns for imagined tastes and food pictures in the mid-insula were reliably similar and quality-specific, suggesting a common code for representing taste quality regardless of whether explicitly imagined or automatically inferred when viewing food. These findings have important implications for our understanding of the mechanisms of mental imagery and the multimodal nature of presumably primary sensory brain regions like the dorsal mid-insula.

Identifying causal subsequent memory effects

Over 40 y of accumulated research has detailed associations between neuroimaging signals measured during a memory encoding task and later memory performance, across a variety of brain regions, measurement tools, statistical approaches, and behavioral tasks. But the interpretation of these subsequent memory effects (SMEs) remains unclear: if the identified signals reflect cognitive and neural mechanisms of memory encoding, then the underlying neural activity must be causally related to future memory. However, almost all previous SME analyses do not control for potential confounders of this causal interpretation, such as serial position and item effects. We collect a large fMRI dataset and use an experimental design and analysis approach that allows us to statistically adjust for nearly all known exogenous confounding variables. We find that, using standard approaches without adjustment, we replicate several univariate and multivariate subsequent memory effects and are able to predict memory performance across people. However, we are unable to identify any signal that reliably predicts subsequent memory after adjusting for confounding variables, bringing into doubt the causal status of these effects. We apply the same approach to subjects' judgments of learning collected following an encoding period and show that these behavioral measures of mnemonic status do predict memory after adjustments, suggesting that it is possible to measure signals near the time of encoding that reflect causal mechanisms but that existing neuroimaging measures, at least in our data, may not have the precision and specificity to do so.

Right Posterior Temporal Cortex Supports Integration of Phonetic and Talker Information

Though the right hemisphere has been implicated in talker processing, it is thought to play a minimal role in phonetic processing, at least relative to the left hemisphere. Recent evidence suggests that the right posterior temporal cortex may support learning of phonetic variation associated with a specific talker. In the current study, listeners heard a male talker and a female talker, one of whom produced an ambiguous fricative in /s/-biased lexical contexts (e.g., epi?ode) and one who produced it in /∫/-biased contexts (e.g., friend?ip). Listeners in a behavioral experiment (Experiment 1) showed evidence of lexically guided perceptual learning, categorizing ambiguous fricatives in line with their previous experience. Listeners in an fMRI experiment (Experiment 2) showed differential phonetic categorization as a function of talker, allowing for an investigation of the neural basis of talker-specific phonetic processing, though they did not exhibit perceptual learning (likely due to characteristics of our in-scanner headphones). Searchlight analyses revealed that the patterns of activation in the right superior temporal sulcus (STS) contained information about who was talking and what phoneme they produced. We take this as evidence that talker information and phonetic information are integrated in the right STS. Functional connectivity analyses suggested that the process of conditioning phonetic identity on talker information depends on the coordinated activity of a left-lateralized phonetic processing system and a right-lateralized talker processing system. Overall, these results clarify the mechanisms through which the right hemisphere supports talker-specific phonetic processing.

IT IS A MATTER OF PERSPECTIVE: ATTENTIONAL FOCUS RATHER THAN DIETARY RESTRAINT DRIVES BRAIN RESPONSES TO FOOD STIMULI

Brain responses to food are thought to reflect food's rewarding value and to fluctuate with dietary restraint. We propose that brain responses to food are dynamic and depend on attentional focus. Food pictures (high-caloric/low-caloric, palatable/unpalatable) were presented during fMRI-scanning, while attentional focus (hedonic/health/neutral) was induced in 52 female participants varying in dietary restraint. The level of brain activity was hardly different between palatable versus unpalatable foods or high-caloric versus low-caloric foods. Activity in several brain regions was higher in hedonic than in health or neutral attentional focus (p < 0.05, FWE-corrected). Palatability and calorie content could be decoded from multi-voxel activity patterns (p < 0.05, FDR-corrected). Dietary restraint did not significantly influence brain responses to food. So, level of brain activity in response to food stimuli depends on attentional focus, and may reflect salience, not reward value. Palatability and calorie content are reflected in patterns of brain activity.

Decoding semantic representations in mind and brain

A key goal for cognitive neuroscience is to understand the neurocognitive systems that support semantic memory. Recent multivariate analyses of neuroimaging data have contributed greatly to this effort, but the rapid development of these novel approaches has made it difficult to track the diversity of findings and to understand how and why they sometimes lead to contradictory conclusions. We address this challenge by reviewing cognitive theories of semantic representation and their neural instantiation. We then consider contemporary approaches to neural decoding and assess which types of representation each can possibly detect. The analysis suggests why the results are heterogeneous and identifies crucial links between cognitive theory, data collection, and analysis that can help to better connect neuroimaging to mechanistic theories of semantic cognition.

Nonconscious information can be identified as task-relevant but not prioritized in working memory

Two critical features of working memory are the identification and appropriate use of task-relevant information while avoiding distraction. Here, in 3 experiments, we explored if these features can be achieved also for nonconscious stimuli. Participants performed a delayed match-to-sample task in which task relevance of 2 competing stimuli was indicated by a cue, and continuous flash suppression was used to manipulate the conscious/nonconscious visual experience. Experiment 1 revealed better-than-chance performance with nonconscious stimuli, demonstrating goal-directed use of nonconscious task-relevant information. Experiment 2 demonstrated that the cue that defined task relevance must be conscious to allow such goal-directed use. In Experiment 3, multi-voxel pattern analyses of brain activity revealed that only the target was prioritized and maintained during conscious trials. Conversely, during nonconscious trials, both target and distractor were maintained. However, decoding of task relevance during the probe/test phase demonstrated identification of both target and distractor information. These results show that identification of task-relevant information can operate also on nonconscious material. However, they do not support the prioritization of nonconscious task-relevant information, thus suggesting a mismatch in the attentional mechanisms involved during conscious and nonconscious working memory.