A reinforcement learning model with choice traces for a progressive ratio schedule

Distinct sex differences in ethanol consumption and operant self-administration in C57BL/6J mice with uniform regulation by glutamate AMPAR activity

Introduction

Considering sex as a biological variable (SABV) in preclinical research can enhance understanding of the neurobiology of alcohol use disorder (AUD). However, the behavioral and neural mechanisms underlying sex-specific differences remain unclear. This study aims to elucidate SABV in ethanol (EtOH) consumption by evaluating its reinforcing effects and regulation by glutamate AMPA receptor activity in male and female mice.

Methods

C57BL/6J mice (male and female) were assessed for EtOH intake under continuous and limited access conditions in the home cage. Acute sensitivity to EtOH sedation and blood clearance were evaluated as potential modifying factors. Motivation to consume EtOH was measured using operant self-administration procedures. Sex-specific differences in neural regulation of EtOH reinforcement were examined by testing the effects of a glutamate AMPA receptor antagonist on operant EtOH self-administration.

Results

Female C57BL/6J mice exhibited a time-dependent escalation in EtOH intake under both continuous and limited access conditions. They were less sensitive to EtOH sedation and had lower blood levels post-EtOH administration (4 g/kg) despite similar clearance rates. Females also showed increased operant EtOH self-administration and progressive ratio performance over a 30-day baseline period compared to males. The AMPAR antagonist GYKI 52466 (0-10 mg/kg, IP) dose-dependently reduced EtOH-reinforced lever pressing in both sexes, with no differences in potency or efficacy.

Discussion

These findings confirm that female C57BL/6J mice consume more EtOH than males in home-cage conditions and exhibit reduced acute sedation, potentially contributing to higher EtOH intake. Females demonstrated increased operant EtOH self-administration and motivation, indicating higher reinforcing efficacy. The lack of sex differences in the relative effects of GYKI 52466 suggests that AMPAR activity is equally required for EtOH reinforcement in both sexes.

Sex Differences in Home-Cage Ethanol Drinking and Operant Self-Administration in C57BL/6J Mice with Equivalent Regulation by Glutamate AMPAR Activity

Introduction

Considering sex as a biological variable (SABV) in preclinical research can enhance understanding of the neurobiology of alcohol use disorder (AUD). However, the behavioral and neural mechanisms underlying sex-specific differences remain unclear. This study aims to elucidate SABV in ethanol (EtOH) consumption by evaluating its reinforcing effects and regulation by glutamate AMPA receptor activity in male and female mice.

Methods

C57BL/6J mice (male and female) were assessed for EtOH intake under continuous and limited access conditions in the home cage. Acute sensitivity to EtOH sedation and blood clearance were evaluated as potential modifying factors. Motivation to consume EtOH was measured using operant self-administration procedures. Sex-specific differences in neural regulation of EtOH reinforcement were examined by testing the effects of a glutamate AMPA receptor antagonist on operant EtOH self-administration.

Results

Female C57BL/6J mice exhibited a time-dependent escalation in EtOH intake under both continuous and limited access conditions. They were less sensitive to EtOH sedation and had lower blood levels post-EtOH administration (4 g/kg) despite similar clearance rates. Females also showed increased operant EtOH self-administration and progressive ratio performance over a 30-day baseline period compared to males. The AMPAR antagonist GYKI 52466 (0-10 mg/kg, IP) dose-dependently reduced EtOH-reinforced lever pressing in both sexes, with no differences in potency or efficacy.

Discussion

These findings confirm that female C57BL/6J mice consume more EtOH than males in home-cage conditions and exhibit reduced acute sedation, potentially contributing to higher EtOH intake. Females demonstrated increased operant EtOH self-administration and motivation, indicating higher reinforcing efficacy. The lack of sex differences in the relative effects of GYKI 52466 suggests that AMPAR activity is equally required for EtOH reinforcement in both sexes.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.