Computational mechanisms of distributed value representations and mixed learning strategies

Contributions of Attention to Learning in Multidimensional Reward Environments

Real-world choice options have many features or attributes, whereas the reward outcome from those options only depends on a few features or attributes. It has been shown that humans learn and combine feature-based with more complex conjunction-based learning to tackle challenges of learning in naturalistic reward environments. However, it remains unclear how different learning strategies interact to determine what features or conjunctions should be attended to and control choice behavior, and how subsequent attentional modulations influence future learning and choice. To address these questions, we examined the behavior of male and female human participants during a three-dimensional learning task in which reward outcomes for different stimuli could be predicted based on a combination of an informative feature and conjunction. Using multiple approaches, we found that both choice behavior and reward probabilities estimated by participants were most accurately described by attention-modulated models that learned the predictive values of both the informative feature and the informative conjunction. Specifically, in the reinforcement learning model that best fit choice data, attention was controlled by the difference in the integrated feature and conjunction values. The resulting attention weights modulated learning by increasing the learning rate on attended features and conjunctions. Critically, modulating decision-making by attention weights did not improve the fit of data, providing little evidence for direct attentional effects on choice. These results suggest that in multidimensional environments, humans direct their attention not only to selectively process reward-predictive attributes but also to find parsimonious representations of the reward contingencies for more efficient learning.

Functional pathology of neuroleptic-induced dystonia based on the striatal striosome-matrix dopamine system in humans

Neuroleptic-induced dystonia is a source of great concern in clinical practice because of its iatrogenic nature which can potentially lead to life-threatening conditions. Since all neuroleptics (antipsychotics) share the ability to block the dopamine D2-type receptors (D2Rs) that are highly enriched in the striatum, this drug-induced dystonia is thought to be caused by decreased striatal D2R activity. However, how associations of striatal D2R inactivation with dystonia are formed remains elusive.A growing body of evidence suggests that imbalanced activities between D1R-expressing medium spiny neurons and D2R-expressing medium spiny neurons (D1-MSNs and D2-MSNs) in the striatal striosome-matrix system underlie the pathophysiology of various basal ganglia disorders including dystonia. Given the specificity of the striatal dopamine D1 system in 'humans', this article highlights the striatal striosome hypothesis in causing 'repetitive' and 'stereotyped' motor symptoms which are key clinical features of dystonia. It is suggested that exposure to neuroleptics may reduce striosomal D1-MSN activity and thereby cause dystonia symptoms. This may occur through an increase in the striatal cholinergic activity and the collateral inhibitory action of D2-MSNs onto neighbouring D1-MSNs within the striosome subfields. The article proposes a functional pathology of the striosome-matrix dopamine system for neuroleptic-induced acute dystonia or neuroleptic-withdrawal dystonia. A rationale for the effectiveness of dopaminergic or cholinergic pharmacotherapy is also provided for treating dystonias. This narrative review covers various aspects of the relevant field and provides a detailed discussion of the mechanisms of neuroleptic-induced dystonia.

Visual perceptual learning of feature conjunctions leverages non-linear mixed selectivity

Visual objects are often defined by multiple features. Therefore, learning novel objects entails learning feature conjunctions. Visual cortex is organized into distinct anatomical compartments, each of which is devoted to processing a single feature. A prime example are neurons purely selective to color and orientation, respectively. However, neurons that jointly encode multiple features (mixed selectivity) also exist across the brain and play critical roles in a multitude of tasks. Here, we sought to uncover the optimal policy that our brain adapts to achieve conjunction learning using these available resources. 59 human subjects practiced orientation-color conjunction learning in four psychophysical experiments designed to nudge the visual system towards using one or the other resource. We find that conjunction learning is possible by linear mixing of pure color and orientation information, but that more and faster learning takes place when both pure and mixed selectivity representations are involved. We also find that learning with mixed selectivity confers advantages in performing an untrained "exclusive or" (XOR) task several months after learning the original conjunction task. This study sheds light on possible mechanisms underlying conjunction learning and highlights the importance of learning by mixed selectivity.