Distributional reinforcement learning in prefrontal cortex

The prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories center on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in the prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses. In the present study, we show that distributional RL also better explains macaque anterior cingulate cortex neuronal responses, suggesting that it is a common mechanism for reward-guided learning.

A cognitive map for value-guided choice in ventromedial prefrontal cortex

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation

The hippocampal-entorhinal system is important for spatial and relational memory tasks. We formally link these domains, provide a mechanistic understanding of the hippocampal role in generalization, and offer unifying principles underlying many entorhinal and hippocampal cell types. We propose medial entorhinal cells form a basis describing structural knowledge, and hippocampal cells link this basis with sensory representations. Adopting these principles, we introduce the Tolman-Eichenbaum machine (TEM). After learning, TEM entorhinal cells display diverse properties resembling apparently bespoke spatial responses, such as grid, band, border, and object-vector cells. TEM hippocampal cells include place and landmark cells that remap between environments. Crucially, TEM also aligns with empirically recorded representations in complex non-spatial tasks. TEM also generates predictions that hippocampal remapping is not random as previously believed; rather, structural knowledge is preserved across environments. We confirm this structural transfer over remapping in simultaneously recorded place and grid cells.

Control of entropy in neural models of environmental state

Humans and animals construct internal models of their environment in order to select appropriate courses of action. The representation of uncertainty about the current state of the environment is a key feature of these models that controls the rate of learning as well as directly affecting choice behaviour. To maintain flexibility, given that uncertainty naturally decreases over time, most theoretical inference models include a dedicated mechanism to drive up model uncertainty. Here we probe the long-standing hypothesis that noradrenaline is involved in determining the uncertainty, or entropy, and thus flexibility, of neural models. Pupil diameter, which indexes neuromodulatory state including noradrenaline release, predicted increases (but not decreases) in entropy in a neural state model encoded in human medial orbitofrontal cortex, as measured using multivariate functional MRI. Activity in anterior cingulate cortex predicted pupil diameter. These results provide evidence for top-down, neuromodulatory control of entropy in neural state models.

Different Effects of Aspirin, Dipyridamole and UD-CG 115 on Platelet Activation in a Model of Vascular Injury: Studies with Extracellular Matrix Covered with Endothelial Cells

Cultured endothelial cells produce an extracellular matrix (ECM) which activates platelets, similarly to deendothelialized vascular segments. Platelet-rich plasma (PRP) was incubated with endothelial cells cultures seeded in various densities on ECM. The interaction of the platelets with this artifical intima was evaluated by phase microscopy and by thromboxane A2 (TXA2) and prostacyclin (PGI2) measurement. Large platelet aggregates were formed on exposed ECM. Platelets aggregation but not adhesion on the ECM was markedly inhibited by the presence of endothelial cells. Pretreatment of the endothelial cells with 0.1 mM aspirin reduced their PGI2 synthesis and was associated with platelet aggregation on the ECM. 10 μM dipyridamole markedly inhibited platelet activation by ECM when the drug was added to citrated whole blood before PRP preparation. UD-CG 115 which elevates cyclic AMP in cardiac muscle, inhibited platelet aggregation and TXA2 production induced by ECM, in the presence as well as in the absence of endothelial cells, without any effect on endothelial PGI2 production.

Three molecular structures cause rhesus D category VI phenotypes with distinct immunohematologic features

Rhesus D category VI (DVI) is the clinically most important partial D. DVI red blood cells were assumed to possess very low RhD antigen density and to be caused by two RHD-CE-D hybrid alleles. Because there was no population-based work-up, we screened three populations in central Europe for DVI. Twenty-six DVI samples were detected and examined by exon-specific RHD polymerase chain reaction with sequence-specific primers (PCR-SSP). A new genotype, hereby designated D category VI type III, was characterized as a RHD-Ce(3-6)-D hybrid allele by sequencing of the cDNA, parts of intron 1, and by PCR-restriction fragment length polymorphism (PCR-RFLP) of intron 2. Rhesus introns 5 and 6 were sequenced and the 3' breakpoints of all known DVI types shown to be distinct. We differentiated the 5' breakpoints of DVI type I and DVI type II by a newly devised RHD-PCR. Thus, the DVI phenotype originated in at least three independent molecular events. Each DVI type showed distinct immunohematologic features in flow cytometry. The number of RhD proteins accessible on the red blood cells' surface of DVI type III was normal (about 12,000 antigens/cell; DVI type I, 500; DVI type II, 2,400) based on the determination of an RhD epitope density profile. DVI type II and DVI type III occurred as CDe haplotypes, and DVI type I as a cDE haplotype. The distribution of the DVI types varied significantly in three German-speaking populations. Genotyping strategies should take account of allelic variations in partial RhD. The reconsideration of previous serologic and clinical data for partial D in view of the underlying molecular structures may be worthwhile.

Relationship between vessel wall 13-HODE synthesis and vessel wall thrombogenicity following injury: influence of salicylate and dipyridamole treatment

We performed studies to determine the relationship between injured vessel wall thrombogenicity, vessel wall 13-hydroxyoctadecadienoic acid (13-HODE) synthesis and cAMP levels in rabbit treated with salicylate or dipyridamole. Injured vessel wall thrombogenicity was measured as the number of 3H-adenine labelled platelets adhered to the subendothelial basement membrane exposed by air injury in carotid arteries of rabbits treated orally with salicylate or dipyridamole. Vessel wall 13-HODE was measured by HPLC and vessel wall cAMP was measured by RIA. Vessel wall thrombogenicity was increased two-fold in rabbits treated with salicylate and decreased by half in rabbits treated with dipyridamole. The levels of vessel wall cAMP levels were correlated both with the plasma dipyridamole levels and increases in 13-HODE synthesis. cAMP levels were unaffected by salicylate treatment, but 13-HODE synthesis was decreased. We conclude that there is a significant relationship between vessel wall cAMP levels and 13-HODE synthesis, which in turn, influences subsequent vessel wall thrombogenicity.

Different effects of aspirin, dipyridamole and UD-CG 115 on platelet activation in a model of vascular injury: studies with extracellular matrix covered with endothelial cells

Cultured endothelial cells produce an extracellular matrix (ECM) which activates platelets, similarly to deendothelialized vascular segments. Platelet-rich plasma (PRP) was incubated with endothelial cells cultures seeded in various densities on ECM. The interaction of the platelets with this artificial intima was evaluated by phase microscopy and by thromboxane A2 (TXA2) and prostacyclin (PGI2) measurement. Large platelet aggregates were formed on exposed ECM. Platelets aggregation but not adhesion on the ECM was markedly inhibited by the presence of endothelial cells. Pretreatment of the endothelial cells with 0.1 mM aspirin reduced their PGI2 synthesis and was associated with platelet aggregation on the ECM. 10 microM dipyridamole markedly inhibited platelet activation by ECM when the drug was added to citrated whole blood before PRP preparation. UD-CG 115 which elevates cyclic AMP in cardiac muscle, inhibited platelet aggregation and TXA2 production induced by ECM, in the presence as well as in the absence of endothelial cells, without any effect on endothelial PGI2 production.