Effects of External Stimulation on Psychedelic State Neurodynamics

Recent findings have shown that psychedelics reliably enhance brain entropy (understood as neural signal diversity), and this effect has been associated with both acute and long-term psychological outcomes, such as personality changes. These findings are particularly intriguing, given that a decrease of brain entropy is a robust indicator of loss of consciousness (e.g., from wakefulness to sleep). However, little is known about how context impacts the entropy-enhancing effect of psychedelics, which carries important implications for how it can be exploited in, for example, psychedelic psychotherapy. This article investigates how brain entropy is modulated by stimulus manipulation during a psychedelic experience by studying participants under the effects of lysergic acid diethylamide (LSD) or placebo, either with gross state changes (eyes closed vs open) or different stimuli (no stimulus vs music vs video). Results show that while brain entropy increases with LSD under all of the experimental conditions, it exhibits the largest changes when subjects have their eyes closed. Furthermore, brain entropy changes are consistently associated with subjective ratings of the psychedelic experience, but this relationship is disrupted when participants are viewing a video─potentially due to a "competition" between external stimuli and endogenous LSD-induced imagery. Taken together, our findings provide strong quantitative evidence of the role of context in modulating neural dynamics during a psychedelic experience, underlining the importance of performing psychedelic psychotherapy in a suitable environment.

Greater than the parts: a review of the information decomposition approach to causal emergence

Emergence is a profound subject that straddles many scientific disciplines, including the formation of galaxies and how consciousness arises from the collective activity of neurons. Despite the broad interest that exists on this concept, the study of emergence has suffered from a lack of formalisms that could be used to guide discussions and advance theories. Here, we summarize, elaborate on, and extend a recent formal theory of causal emergence based on information decomposition, which is quantifiable and amenable to empirical testing. This theory relates emergence with information about a system's temporal evolution that cannot be obtained from the parts of the system separately. This article provides an accessible but rigorous introduction to the framework, discussing the merits of the approach in various scenarios of interest. We also discuss several interpretation issues and potential misunderstandings, while highlighting the distinctive benefits of this formalism. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'.

The strength of weak integrated information theory

The integrated information theory of consciousness (IIT) is divisive: while some believe it provides an unprecedentedly powerful approach to address the 'hard problem', others dismiss it on grounds that it is untestable. We argue that the appeal and applicability of IIT can be greatly widened if we distinguish two flavours of the theory: strong IIT, which identifies consciousness with specific properties associated with maxima of integrated information; and weak IIT, which tests pragmatic hypotheses relating aspects of consciousness to broader measures of information dynamics. We review challenges for strong IIT, explain how existing empirical findings are well explained by weak IIT without needing to commit to the entirety of strong IIT, and discuss the outlook for both flavours of IIT.

Integrated information as a common signature of dynamical and information-processing complexity

The apparent dichotomy between information-processing and dynamical approaches to complexity science forces researchers to choose between two diverging sets of tools and explanations, creating conflict and often hindering scientific progress. Nonetheless, given the shared theoretical goals between both approaches, it is reasonable to conjecture the existence of underlying common signatures that capture interesting behavior in both dynamical and information-processing systems. Here, we argue that a pragmatic use of integrated information theory (IIT), originally conceived in theoretical neuroscience, can provide a potential unifying framework to study complexity in general multivariate systems. By leveraging metrics put forward by the integrated information decomposition framework, our results reveal that integrated information can effectively capture surprisingly heterogeneous signatures of complexity-including metastability and criticality in networks of coupled oscillators as well as distributed computation and emergent stable particles in cellular automata-without relying on idiosyncratic, ad hoc criteria. These results show how an agnostic use of IIT can provide important steps toward bridging the gap between informational and dynamical approaches to complex systems.

Decomposing Spectral and Phasic Differences in Nonlinear Features between Datasets

When employing nonlinear methods to characterize complex systems, it is important to determine to what extent they are capturing genuine nonlinear phenomena that could not be assessed by simpler spectral methods. Specifically, we are concerned with the problem of quantifying spectral and phasic effects on an observed difference in a nonlinear feature between two systems (or two states of the same system). Here we derive, from a sequence of null models, a decomposition of the difference in an observable into spectral, phasic, and spectrum-phase interaction components. Our approach makes no assumptions about the structure of the data and adds nuance to a wide range of time series analyses.

Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data

The broad concept of emergence is instrumental in various of the most challenging open scientific questions-yet, few quantitative theories of what constitutes emergent phenomena have been proposed. This article introduces a formal theory of causal emergence in multivariate systems, which studies the relationship between the dynamics of parts of a system and macroscopic features of interest. Our theory provides a quantitative definition of downward causation, and introduces a complementary modality of emergent behaviour-which we refer to as causal decoupling. Moreover, the theory allows practical criteria that can be efficiently calculated in large systems, making our framework applicable in a range of scenarios of practical interest. We illustrate our findings in a number of case studies, including Conway's Game of Life, Reynolds' flocking model, and neural activity as measured by electrocorticography.

Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation

Integrated Information Theory (IIT) is a prominent theory of consciousness that has at its centre measures that quantify the extent to which a system generates more information than the sum of its parts. While several candidate measures of integrated information (" Φ ") now exist, little is known about how they compare, especially in terms of their behaviour on non-trivial network models. In this article, we provide clear and intuitive descriptions of six distinct candidate measures. We then explore the properties of each of these measures in simulation on networks consisting of eight interacting nodes, animated with Gaussian linear autoregressive dynamics. We find a striking diversity in the behaviour of these measures-no two measures show consistent agreement across all analyses. A subset of the measures appears to reflect some form of dynamical complexity, in the sense of simultaneous segregation and integration between system components. Our results help guide the operationalisation of IIT and advance the development of measures of integrated information and dynamical complexity that may have more general applicability.

Loss of consciousness is related to hyper-correlated gamma-band activity in anesthetized macaques and sleeping humans

Loss of consciousness can result from a wide range of causes, including natural sleep and pharmacologically induced anesthesia. Important insights might thus come from identifying neuronal mechanisms of loss and re-emergence of consciousness independent of a specific manipulation. Therefore, to seek neuronal signatures of loss of consciousness common to sleep and anesthesia we analyzed spontaneous electrophysiological activity recorded in two experiments. First, electrocorticography (ECoG) acquired from 4 macaque monkeys anesthetized with different anesthetic agents (ketamine, medetomidine, propofol) and, second, stereo-electroencephalography (sEEG) from 10 epilepsy patients in different wake-sleep stages (wakefulness, NREM, REM). Specifically, we investigated co-activation patterns among brain areas, defined as correlations between local amplitudes of gamma-band activity. We found that resting wakefulness was associated with intermediate levels of gamma-band coupling, indicating neither complete dependence, nor full independence among brain regions. In contrast, loss of consciousness during NREM sleep and propofol anesthesia was associated with excessively correlated brain activity, as indicated by a robust increase of number and strength of positive correlations. However, such excessively correlated brain signals were not observed during REM sleep, and were present only to a limited extent during ketamine anesthesia. This might be related to the fact that, despite suppression of behavioral responsiveness, REM sleep and ketamine anesthesia often involve presence of dream-like conscious experiences. We conclude that hyper-correlated gamma-band activity might be a signature of loss of consciousness common across various manipulations and independent of behavioral responsiveness.

Theta-burst transcranial magnetic stimulation to the prefrontal or parietal cortex does not impair metacognitive visual awareness

Neuroimaging studies commonly associate dorsolateral prefrontal cortex (DLPFC) and posterior parietal cortex with conscious perception. However, such studies only investigate correlation, rather than causation. In addition, many studies conflate objective performance with subjective awareness. In an influential recent paper, Rounis and colleagues addressed these issues by showing that continuous theta burst transcranial magnetic stimulation (cTBS) applied to the DLPFC impaired metacognitive (subjective) awareness for a perceptual task, while objective performance was kept constant. We attempted to replicate this finding, with minor modifications, including an active cTBS control site. Using a between-subjects design for both DLPFC and posterior parietal cortices, we found no evidence of a cTBS-induced metacognitive impairment. In a second experiment, we devised a highly rigorous within-subjects cTBS design for DLPFC, but again failed to find any evidence of metacognitive impairment. One crucial difference between our results and the Rounis study is our strict exclusion of data deemed unsuitable for a signal detection theory analysis. Indeed, when we included this unstable data, a significant, though invalid, metacognitive impairment was found. These results cast doubt on previous findings relating metacognitive awareness to DLPFC, and inform the current debate concerning whether or not prefrontal regions are preferentially implicated in conscious perception.

Global and local complexity of intracranial EEG decreases during NREM sleep

Key to understanding the neuronal basis of consciousness is the characterization of the neural signatures of changes in level of consciousness during sleep. Here we analysed three measures of dynamical complexity on spontaneous depth electrode recordings from 10 epilepsy patients during wakeful rest (WR) and different stages of sleep: (i) Lempel-Ziv complexity, which is derived from how compressible the data are; (ii) amplitude coalition entropy, which measures the variability over time of the set of channels active above a threshold; (iii) synchrony coalition entropy, which measures the variability over time of the set of synchronous channels. When computed across sets of channels that are broadly distributed across multiple brain regions, all three measures decreased substantially in all participants during early-night non-rapid eye movement (NREM) sleep. This decrease was partially reversed during late-night NREM sleep, while the measures scored similar to WR during rapid eye movement (REM) sleep. This global pattern was in almost all cases mirrored at the local level by groups of channels located in a single region. In testing for differences between regions, we found elevated signal complexity in the frontal lobe. These differences could not be attributed solely to changes in spectral power between conditions. Our results provide further evidence that the level of consciousness correlates with neural dynamical complexity.

Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems

To fully characterize the information that two source variables carry about a third target variable, one must decompose the total information into redundant, unique, and synergistic components, i.e., obtain a partial information decomposition (PID). However, Shannon's theory of information does not provide formulas to fully determine these quantities. Several recent studies have begun addressing this. Some possible definitions for PID quantities have been proposed and some analyses have been carried out on systems composed of discrete variables. Here we present an in-depth analysis of PIDs on Gaussian systems, both static and dynamical. We show that, for a broad class of Gaussian systems, previously proposed PID formulas imply that (i) redundancy reduces to the minimum information provided by either source variable and hence is independent of correlation between sources, and (ii) synergy is the extra information contributed by the weaker source when the stronger source is known and can either increase or decrease with correlation between sources. We find that Gaussian systems frequently exhibit net synergy, i.e., the information carried jointly by both sources is greater than the sum of information carried by each source individually. Drawing from several explicit examples, we discuss the implications of these findings for measures of information transfer and information-based measures of complexity, both generally and within a neuroscience setting. Importantly, by providing independent formulas for synergy and redundancy applicable to continuous time-series data, we provide an approach to characterizing and quantifying information sharing amongst complex system variables.

Blind insight: metacognitive discrimination despite chance task performance

Blindsight and other examples of unconscious knowledge and perception demonstrate dissociations between judgment accuracy and metacognition: Studies reveal that participants' judgment accuracy can be above chance while their confidence ratings fail to discriminate right from wrong answers. Here, we demonstrated the opposite dissociation: a reliable relationship between confidence and judgment accuracy (demonstrating metacognition) despite judgment accuracy being no better than chance. We evaluated the judgments of 450 participants who completed an AGL task. For each trial, participants decided whether a stimulus conformed to a given set of rules and rated their confidence in that judgment. We identified participants who performed at chance on the discrimination task, utilizing a subset of their responses, and then assessed the accuracy and the confidence-accuracy relationship of their remaining responses. Analyses revealed above-chance metacognition among participants who did not exhibit decision accuracy. This important new phenomenon, which we term blind insight, poses critical challenges to prevailing models of metacognition grounded in signal detection theory.

An extended case study on the phenomenology of sequence-space synesthesia

Investigation of synesthesia phenomenology in adults is needed to constrain accounts of developmental trajectories of this trait. We report an extended phenomenological investigation of sequence-space synesthesia in a single case (AB). We used the Elicitation Interview (EI) method to facilitate repeated exploration of AB's synesthetic experience. During an EI the subject's attention is selectively guided by the interviewer in order to reveal precise details about the experience. Detailed analysis of the resulting 9 h of interview transcripts provided a comprehensive description of AB's synesthetic experience, including several novel observations. For example, we describe a specific spatial reference frame (a “mental room”) in which AB's concurrents occur, and which overlays his perception of the real world (the “physical room”). AB is able to switch his attention voluntarily between this mental room and the physical room. Exemplifying the EI method, some of our observations were previously unknown even to AB. For example, AB initially reported to experience concurrents following visual presentation, yet we determined that in the majority of cases the concurrent followed an internal verbalization of the inducer, indicating an auditory component to sequence-space synesthesia. This finding is congruent with typical rehearsal of inducer sequences during development, implicating cross-modal interactions between auditory and visual systems in the genesis of this synesthetic form. To our knowledge, this paper describes the first application of an EI to synesthesia, and the first systematic longitudinal investigation of the first-person experience of synesthesia since the re-emergence of interest in this topic in the 1980's. These descriptions move beyond rudimentary graphical or spatial representations of the synesthetic spatial form, thereby providing new targets for neurobehavioral analysis.

Accurate metacognition for visual sensory memory representations

The capacity to attend to multiple objects in the visual field is limited. However, introspectively, people feel that they see the whole visual world at once. Some scholars suggest that this introspective feeling is based on short-lived sensory memory representations, whereas others argue that the feeling of seeing more than can be attended to is illusory. Here, we investigated this phenomenon by combining objective memory performance with subjective confidence ratings during a change-detection task. This allowed us to compute a measure of metacognition--the degree of knowledge that subjects have about the correctness of their decisions--for different stages of memory. We show that subjects store more objects in sensory memory than they can attend to but, at the same time, have similar metacognition for sensory memory and working memory representations. This suggests that these subjective impressions are not an illusion but accurate reflections of the richness of visual perception.

An integration of integrated information theory with fundamental physics

To truly eliminate Cartesian ghosts from the science of consciousness, we must describe consciousness as an aspect of the physical. Integrated Information Theory states that consciousness arises from intrinsic information generated by dynamical systems; however existing formulations of this theory are not applicable to standard models of fundamental physical entities. Modern physics has shown that fields are fundamental entities, and in particular that the electromagnetic field is fundamental. Here I hypothesize that consciousness arises from information intrinsic to fundamental fields. This hypothesis unites fundamental physics with what we know empirically about the neuroscience underlying consciousness, and it bypasses the need to consider quantum effects.

What the heart forgets: Cardiac timing influences memory for words and is modulated by metacognition and interoceptive sensitivity

Mental functions are influenced by states of physiological arousal. Afferent neural activity from arterial baroreceptors at systole conveys the strength and timing of individual heartbeats to the brain. We presented words under limited attentional resources time-locked to different phases of the cardiac cycle, to test a hypothesis that natural baroreceptor stimulation influences detection and subsequent memory of words. We show memory for words presented around systole was decreased relative to words at diastole. The deleterious memory effect of systole was greater for words detected with low confidence and amplified in individuals with low interoceptive sensitivity, as indexed using a heartbeat counting task. Our observations highlight an important cardiovascular channel through which autonomic arousal impacts a cognitive function, an effect mitigated by metacognition (perceptual confidence) and interoceptive sensitivity.

Soft-bound synaptic plasticity increases storage capacity

Accurate models of synaptic plasticity are essential to understand the adaptive properties of the nervous system and for realistic models of learning and memory. Experiments have shown that synaptic plasticity depends not only on pre- and post-synaptic activity patterns, but also on the strength of the connection itself. Namely, weaker synapses are more easily strengthened than already strong ones. This so called soft-bound plasticity automatically constrains the synaptic strengths. It is known that this has important consequences for the dynamics of plasticity and the synaptic weight distribution, but its impact on information storage is unknown. In this modeling study we introduce an information theoretic framework to analyse memory storage in an online learning setting. We show that soft-bound plasticity increases a variety of performance criteria by about 18% over hard-bound plasticity, and likely maximizes the storage capacity of synapses.

It is generally believed that our memories are stored in the synaptic connections between neurons. Numerous experimental studies have therefore examined when and how the synaptic connections change. In parallel, many computational studies have examined the properties of memory and synaptic plasticity, aiming to better understand human memory and allow for neural network models of the brain. However, the plasticity rules used in most studies are highly simplified and do not take into account the rich behaviour found in experiments. For instance, it has been observed in experiments that it is hard to make strong synapses even stronger. Here we show that this saturation of plasticity enhances the number of memories that can be stored and introduce a general framework to calculate information storage in online learning paradigms.

Granger causality analysis of steady-state electroencephalographic signals during propofol-induced anaesthesia

Changes in conscious level have been associated with changes in dynamical integration and segregation among distributed brain regions. Recent theoretical developments emphasize changes in directed functional (i.e., causal) connectivity as reflected in quantities such as 'integrated information' and 'causal density'. Here we develop and illustrate a rigorous methodology for assessing causal connectivity from electroencephalographic (EEG) signals using Granger causality (GC). Our method addresses the challenges of non-stationarity and bias by dividing data into short segments and applying permutation analysis. We apply the method to EEG data obtained from subjects undergoing propofol-induced anaesthesia, with signals source-localized to the anterior and posterior cingulate cortices. We found significant increases in bidirectional GC in most subjects during loss-of-consciousness, especially in the beta and gamma frequency ranges. Corroborating a previous analysis we also found increases in synchrony in these ranges; importantly, the Granger causality analysis showed higher inter-subject consistency than the synchrony analysis. Finally, we validate our method using simulated data generated from a model for which GC values can be analytically derived. In summary, our findings advance the methodology of Granger causality analysis of EEG data and carry implications for integrated information and causal density theories of consciousness.

Fluctuations in the open time of synaptic channels: an application to noise analysis based on charge

Synaptic channels are stochastic devices. Even recording from large ensembles of channels, the fluctuations, described by Markov transition matrices, can be used to extract single channel properties. Here we study fluctuations in the open time of channels, which is proportional to the charge flowing through the channel. We use the results to implement a novel type of noise analysis that uses the charge rather than the current to extract fundamental channel parameters. We show in simulations that this charge based noise analysis is more robust if the synapse is located on the dendrites and thus subject to cable filtering. However, we also demonstrate that when multiple synapses are distributed on the dendrites, noise analysis breaks down. We finally discuss applications of our results to other biological processes.

Causal density and integrated information as measures of conscious level

An outstanding challenge in neuroscience is to develop theoretically grounded and practically applicable quantitative measures that are sensitive to conscious level. Such measures should be high for vivid alert conscious wakefulness, and low for unconscious states such as dreamless sleep, coma and general anaesthesia. Here, we describe recent progress in the development of measures of dynamical complexity, in particular causal density and integrated information. These and similar measures capture in different ways the extent to which a system's dynamics are simultaneously differentiated and integrated. Because conscious scenes are distinguished by the same dynamical features, these measures are therefore good candidates for reflecting conscious level. After reviewing the theoretical background, we present new simulation results demonstrating similarities and differences between the measures, and we discuss remaining challenges in the practical application of the measures to empirically obtained data.

Practical measures of integrated information for time-series data

A recent measure of ‘integrated information’, Φ_DM, quantifies the extent to which a system generates more information than the sum of its parts as it transitions between states, possibly reflecting levels of consciousness generated by neural systems. However, Φ_DM is defined only for discrete Markov systems, which are unusual in biology; as a result, Φ_DM can rarely be measured in practice. Here, we describe two new measures, Φ_E and Φ_AR, that overcome these limitations and are easy to apply to time-series data. We use simulations to demonstrate the in-practice applicability of our measures, and to explore their properties. Our results provide new opportunities for examining information integration in real and model systems and carry implications for relations between integrated information, consciousness, and other neurocognitive processes. However, our findings pose challenges for theories that ascribe physical meaning to the measured quantities.

A key feature of the human brain is its ability to represent a vast amount of information, and to integrate this information in order to produce specific and selective behaviour, as well as a stream of unified conscious scenes. Attempts have been made to quantify so-called ‘integrated information’ by formalizing in mathematics the extent to which a system as a whole generates more information than the sum of its parts. However, so far, the resulting measures have turned out to be inapplicable to real neural systems. In this paper we introduce two new measures that can be applied to both realistic neural models and to time-series data garnered from a broad range of neuroimaging and electrophysiological methods. Our work provides new opportunities for examining the role of integrated information in cognition and consciousness, and indeed in the function of any complex biological system. However, our results also pose challenges for theories that ascribe a direct physical meaning to any version of integrated information so far described.

Neural theories need to account for, not discount, introspection and behavior

Abstract A satisfying neuroscience of consciousness must account for phenomenological properties in terms of neural properties. While pursuing this project may challenge our intuitions about what we are conscious of, evidence from behavior and introspection should not be discounted. All three lines of evidence need to be integrated in order to naturalize phenomenal experience.

Multivariate Granger causality and generalized variance

Granger causality analysis is a popular method for inference on directed interactions in complex systems of many variables. A shortcoming of the standard framework for Granger causality is that it only allows for examination of interactions between single (univariate) variables within a system, perhaps conditioned on other variables. However, interactions do not necessarily take place between single variables but may occur among groups or "ensembles" of variables. In this study we establish a principled framework for Granger causality in the context of causal interactions among two or more multivariate sets of variables. Building on Geweke's seminal 1982 work, we offer additional justifications for one particular form of multivariate Granger causality based on the generalized variances of residual errors. Taken together, our results support a comprehensive and theoretically consistent extension of Granger causality to the multivariate case. Treated individually, they highlight several specific advantages of the generalized variance measure, which we illustrate using applications in neuroscience as an example. We further show how the measure can be used to define "partial" Granger causality in the multivariate context and we also motivate reformulations of "causal density" and "Granger autonomy." Our results are directly applicable to experimental data and promise to reveal new types of functional relations in complex systems, neural and otherwise.

Dynamics and robustness of familiarity memory

When presented with an item or a face, one might have a sense of recognition without the ability to recall when or where the stimulus has been encountered before. This sense of recognition is called familiarity memory. Following previous computational studies of familiarity memory, we investigate the dynamical properties of familiarity discrimination and contrast two different familiarity discriminators: one based on the energy of the neural network and the other based on the time derivative of the energy. We show how the familiarity signal decays rapidly after stimulus presentation. For both discriminators, we calculate the capacity using mean field analysis. Compared to recall capacity (the classical associative memory in Hopfield nets), both the energy and the slope discriminators have bigger capacity, yet the energy-based discriminator has a higher capacity than one based on its time derivative. Finally, both discriminators are found to have a different noise dependence.

Granger causality and transfer entropy are equivalent for Gaussian variables

Granger causality is a statistical notion of causal influence based on prediction via vector autoregression. Developed originally in the field of econometrics, it has since found application in a broader arena, particularly in neuroscience. More recently transfer entropy, an information-theoretic measure of time-directed information transfer between jointly dependent processes, has gained traction in a similarly wide field. While it has been recognized that the two concepts must be related, the exact relationship has until now not been formally described. Here we show that for Gaussian variables, Granger causality and transfer entropy are entirely equivalent, thus bridging autoregressive and information-theoretic approaches to data-driven causal inference.

State based model of long-term potentiation and synaptic tagging and capture

Recent data indicate that plasticity protocols have not only synapse-specific but also more widespread effects. In particular, in synaptic tagging and capture (STC), tagged synapses can capture plasticity-related proteins, synthesized in response to strong stimulation of other synapses. This leads to long-lasting modification of only weakly stimulated synapses. Here we present a biophysical model of synaptic plasticity in the hippocampus that incorporates several key results from experiments on STC. The model specifies a set of physical states in which a synapse can exist, together with transition rates that are affected by high- and low-frequency stimulation protocols. In contrast to most standard plasticity models, the model exhibits both early- and late-phase LTP/D, de-potentiation, and STC. As such, it provides a useful starting point for further theoretical work on the role of STC in learning and memory.

Optimal learning rules for discrete synapses

There is evidence that biological synapses have a limited number of discrete weight states. Memory storage with such synapses behaves quite differently from synapses with unbounded, continuous weights, as old memories are automatically overwritten by new memories. Consequently, there has been substantial discussion about how this affects learning and storage capacity. In this paper, we calculate the storage capacity of discrete, bounded synapses in terms of Shannon information. We use this to optimize the learning rules and investigate how the maximum information capacity depends on the number of synapses, the number of synaptic states, and the coding sparseness. Below a certain critical number of synapses per neuron (comparable to numbers found in biology), we find that storage is similar to unbounded, continuous synapses. Hence, discrete synapses do not necessarily have lower storage capacity.

It is believed that the neural basis of learning and memory is change in the strength of synaptic connections between neurons. Much theoretical work on this topic assumes that the strength, or weight, of a synapse may vary continuously and be unbounded. More recent studies have considered synapses that have a limited number of discrete states. In dynamical models of such synapses, old memories are automatically overwritten by new memories, and it has been previously difficult to optimize performance using standard capacity measures, for stronger learning typically implies faster forgetting. Here, we propose an information theoretic measure of storage capacity of such forgetting systems, and use this to optimize the learning rules. We find that for parameters comparable to those found in biology, capacity of discrete synapses is similar to that of unbounded, continuous synapses, provided the number of synapses per neuron is limited. Our findings are relevant for experiments investigating the precise nature of synaptic changes during learning, and also pave a path for further work on building biologically realistic memory models.