Building transformers from neurons and astrocytes

Astrocytes as a mechanism for contextually-guided network dynamics and function

Astrocytes are a ubiquitous and enigmatic type of non-neuronal cell and are found in the brain of all vertebrates. While traditionally viewed as being supportive of neurons, it is increasingly recognized that astrocytes play a more direct and active role in brain function and neural computation. On account of their sensitivity to a host of physiological covariates and ability to modulate neuronal activity and connectivity on slower time scales, astrocytes may be particularly well poised to modulate the dynamics of neural circuits in functionally salient ways. In the current paper, we seek to capture these features via actionable abstractions within computational models of neuron-astrocyte interaction. Specifically, we engage how nested feedback loops of neuron-astrocyte interaction, acting over separated time-scales, may endow astrocytes with the capability to enable learning in context-dependent settings, where fluctuations in task parameters may occur much more slowly than within-task requirements. We pose a general model of neuron-synapse-astrocyte interaction and use formal analysis to characterize how astrocytic modulation may constitute a form of meta-plasticity, altering the ways in which synapses and neurons adapt as a function of time. We then embed this model in a bandit-based reinforcement learning task environment, and show how the presence of time-scale separated astrocytic modulation enables learning over multiple fluctuating contexts. Indeed, these networks learn far more reliably compared to dynamically homogeneous networks and conventional non-network-based bandit algorithms. Our results fuel the notion that neuron-astrocyte interactions in the brain benefit learning over different time-scales and the conveyance of task-relevant contextual information onto circuit dynamics.

The role of astrocytic γ-aminobutyric acid in the action of inhalational anesthetics

BACKGROUND

Inhalational anesthetics target the inhibitory extrasynaptic γ-aminobutyric acid type A (GABAA) receptors. Both neuronal and glial GABA mediate tonic inhibition of the extrasynaptic GABAA receptors. However, the role of glial GABA during inhalational anesthesia remains unclear. This study aimed to evaluate whether astrocytic GABA contributes to the action of different inhalational anesthetics.

METHODS

Gene knockout of monoamine oxidase B (MAOB) was used to reduce astrocytic GABA levels in mice. The hypnotic and immobilizing effects of isoflurane, sevoflurane, and desflurane were assessed by evaluating the loss of righting reflex (LORR) and tail-pinch withdrawal response (LTWR) in MAOB knockout and wild-type mice. Minimum alveolar concentration (MAC) for LORR, time to LORR, MAC for LTWR and time to LTWR of isoflurane, sevoflurane, and desflurane were assessed.

RESULTS

Time to LORR and time to LTWR with isoflurane were significantly longer in MAOB knockout mice than in wild-type mice (P < 0.001 and P = 0.032, respectively). Time to LORR with 0.8 MAC of sevoflurane was significantly longer in MAOB knockout mice than in wild-type mice (P < 0.001), but not with 1.0 MAC of sevoflurane (P=0.217). MAC for LTWR was significantly higher in MAOB knockout mice exposed to sevoflurane (P < 0.001). With desflurane, MAOB knockout mice had a significantly higher MAC for LORR (P = 0.003) and higher MAC for LTWR (P < 0.001) than wild-type mice.

CONCLUSIONS

MAOB knockout mice showed reduced sensitivity to the hypnotic and immobilizing effects of isoflurane, sevoflurane, and desflurane. Behavioral tests revealed that the hypnotic and immobilizing effects of inhalational anesthetics would be mediated by astrocytic GABA.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of the brain response associated with each sentence. We then use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress the activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also non-invasively control neural activity in higher-level cortical areas, such as the language network.

Short-term Hebbian learning can implement transformer-like attention

Transformers have revolutionized machine learning models of language and vision, but their connection with neuroscience remains tenuous. Built from attention layers, they require a mass comparison of queries and keys that is difficult to perform using traditional neural circuits. Here, we show that neurons can implement attention-like computations using short-term, Hebbian synaptic potentiation. We call our mechanism the match-and-control principle and it proposes that when activity in an axon is synchronous, or matched, with the somatic activity of a neuron that it synapses onto, the synapse can be briefly strongly potentiated, allowing the axon to take over, or control, the activity of the downstream neuron for a short time. In our scheme, the keys and queries are represented as spike trains and comparisons between the two are performed in individual spines allowing for hundreds of key comparisons per query and roughly as many keys and queries as there are neurons in the network.