Brain connectivity meets reservoir computing

Chaotic recurrent neural networks for brain modelling: A review

Even in the absence of external stimuli, the brain is spontaneously active. Indeed, most cortical activity is internally generated by recurrence. Both theoretical and experimental studies suggest that chaotic dynamics characterize this spontaneous activity. While the precise function of brain chaotic activity is still puzzling, we know that chaos confers many advantages. From a computational perspective, chaos enhances the complexity of network dynamics. From a behavioural point of view, chaotic activity could generate the variability required for exploration. Furthermore, information storage and transfer are maximized at the critical border between order and chaos. Despite these benefits, many computational brain models avoid incorporating spontaneous chaotic activity due to the challenges it poses for learning algorithms. In recent years, however, multiple approaches have been proposed to overcome this limitation. As a result, many different algorithms have been developed, initially within the reservoir computing paradigm. Over time, the field has evolved to increase the biological plausibility and performance of the algorithms, sometimes going beyond the reservoir computing framework. In this review article, we examine the computational benefits of chaos and the unique properties of chaotic recurrent neural networks, with a particular focus on those typically utilized in reservoir computing. We also provide a detailed analysis of the algorithms designed to train chaotic RNNs, tracing their historical evolution and highlighting key milestones in their development. Finally, we explore the applications and limitations of chaotic RNNs for brain modelling, consider their potential broader impacts beyond neuroscience, and outline promising directions for future research.

Computational memory capacity predicts aging and cognitive decline

Memory is a crucial cognitive function that deteriorates with age. However, this ability is normally assessed using cognitive tests instead of the architecture of brain networks. Here, we use reservoir computing, a recurrent neural network computing paradigm, to assess the linear memory capacities of neural-network reservoirs extracted from brain anatomical connectivity data in a lifespan cohort of 636 individuals. The computational memory capacity emerges as a robust marker of aging, being associated with resting-state functional activity, white matter integrity, locus coeruleus signal intensity, and cognitive performance. We replicate our findings in an independent cohort of 154 young and 72 old individuals. By linking the computational memory capacity of the brain network with cognition, brain function and integrity, our findings open new pathways to employ reservoir computing to investigate aging and age-related disorders.

Reduced connection strength leads to enhancement of working memory capacity in cognitive training

It has been widely observed that cognitive training can enhance the working memory capacity (WMC) of participants, yet the underlying mechanisms remain unexplained. Previous research has confirmed that abacus-based mental calculation (AMC) training can enhance the WMC of subjects and suggested its possible association with changes in functional connectivity. With fMRI data, we construct whole brain resting state connectivity of subjects who underwent long-term AMC training and other subjects from a control group. Their working memory capacity is simulated based on their whole brain resting state connectivity and reservoir computing. It is found that the AMC group has higher WMC than the control group, and especially the WMC involved in the frontoparietal network (FPN), visual network (VIS) and sensorimotor network (SMN) associated with the AMC training is even higher in the AMC group. However, the advantage of the AMC group disappears if the connection strengths between brain regions are neglected. The effects on WMC from the connection strength differences between the AMC and control groups are evaluated. The results show that the WMC of the control group is enhanced and achieved consistency with or even better than that the AMC group if the connection strength of the control group are weakened. And the advantage of FPN, VIS and SMN is reproduced too. In conclusion, our work reveals a correlation between reduction in functional connection strength and enhancements in the WMC of subjects undergoing cognitive training.

Simulating the impact of white matter connectivity on processing time scales using brain network models

The capacity of the brain to process input across temporal scales is exemplified in human narrative, which requires integration of information ranging from words, over sentences to long paragraphs. It has been shown that this processing is distributed in a hierarchy across multiple areas in the brain with areas close to the sensory cortex, processing on a faster time scale than areas in associative cortex. In this study we used reservoir computing with human derived connectivity to investigate the effect of the structural connectivity on time scales across brain regions during a narrative task paradigm. We systematically tested the effect of removal of selected fibre bundles (IFO, ILF, MLF, SLF I/II/III, UF, AF) on the processing time scales across brain regions. We show that long distance pathways such as the IFO provide a form of shortcut whereby input driven activation in the visual cortex can directly impact distant frontal areas. To validate our model we demonstrated significant correlation of our predicted time scale ordering with empirical results from the intact/scrambled narrative fMRI task paradigm. This study emphasizes structural connectivity's role in brain temporal processing hierarchies, providing a framework for future research on structure and neural dynamics across cognitive tasks.

Role of short-term plasticity and slow temporal dynamics in enhancing time series prediction with a brain-inspired recurrent neural network

Typical reservoir networks are based on random connectivity patterns that differ from brain circuits in two important ways. First, traditional reservoir networks lack synaptic plasticity among recurrent units, whereas cortical networks exhibit plasticity across all neuronal types and cortical layers. Second, reservoir networks utilize random Gaussian connectivity, while cortical networks feature a heavy-tailed distribution of synaptic strengths. It is unclear what are the computational advantages of these features for predicting complex time series. In this study, we integrated short-term plasticity (STP) and lognormal connectivity into a novel recurrent neural network (RNN) framework. The model exhibited rich patterns of population activity characterized by slow coordinated fluctuations. Using graph spectral decomposition, we show that weighted networks with lognormal connectivity and STP yield higher complexity than several graph types. When tested on various tasks involving the prediction of complex time series data, the RNN model outperformed a baseline model with random connectivity as well as several other network architectures. Overall, our results underscore the potential of incorporating brain-inspired features such as STP and heavy-tailed connectivity to enhance the robustness and performance of artificial neural networks in complex data prediction and signal processing tasks.

Evolution beats random chance: Performance-dependent network evolution for enhanced computational capacity

The quest to understand structure-function relationships in networks across scientific disciplines has intensified. However, the optimal network architecture remains elusive, particularly for complex information processing. Therefore, we investigate how optimal and specific network structures form to efficiently solve distinct tasks using a framework of performance-dependent network evolution, leveraging reservoir computing principles. Our study demonstrates that task-specific minimal network structures obtained through this framework consistently outperform networks generated by alternative growth strategies and Erdős-Rényi random networks. Evolved networks exhibit unexpected sparsity and adhere to scaling laws in node-density space while showcasing a distinctive asymmetry in input and information readout node distribution. Consequently, we propose a heuristic for quantifying task complexity from performance-dependently evolved networks, offering valuable insights into the evolutionary dynamics of the network structure-function relationship. Our findings advance the fundamental understanding of process-specific network evolution and shed light on the design and optimization of complex information processing mechanisms, notably in machine learning.

Asymmetrically connected reservoir networks learn better

We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. To this end, we systematically investigate the impact of network connectivity on its performance, i.e., we examine the symmetry and structure of the reservoir in relation to its computational power. Reservoirs with random and asymmetric connections are found to perform better for an exemplary Mackey-Glass time series than all structured reservoirs, including biologically inspired connectivities, such as small-world topologies. This result is quantified by the information processing capacity of the different network topologies which becomes highest for asymmetric and randomly connected networks.

A biologically inspired computational model of human ventral temporal cortex

Our minds represent miscellaneous objects in the physical world metaphorically in an abstract and complex high-dimensional object space, which is implemented in a two-dimensional surface of the ventral temporal cortex (VTC) with topologically organized object selectivity. Here we investigated principles guiding the topographical organization of object selectivities in the VTC by constructing a hybrid Self-Organizing Map (SOM) model that harnesses a biologically inspired algorithm of wiring cost minimization and adheres to the constraints of the lateral wiring span of human VTC neurons. In a series of in silico experiments with functional brain neuroimaging and neurophysiological single-unit data from humans and non-human primates, the VTC-SOM predicted the topographical structure of fine-scale category-selective regions (face-, tool-, body-, and place-selective regions) and the boundary in large-scale abstract functional maps (animate vs. inanimate, real-word small-size vs. big-size, central vs. peripheral), with no significant loss in functionality (e.g., categorical selectivity and view-invariant representations). In addition, when the same principle was applied to V1 orientation preferences, a pinwheel-like topology emerged, suggesting the model's broad applicability. In summary, our study illustrates that the simple principle of wiring cost minimization, coupled with the appropriate biological constraint of lateral wiring span, is able to implement the high-dimensional object space in a two-dimensional cortical surface.

A QR Code for the Brain: A dynamical systems framework for computing neurophysiological biomarkers

Neural circuits are often considered the bridge connecting genetic causes and behavior. Whereas prenatal neural circuits are believed to be derived from a combination of genetic and intrinsic activity, postnatal circuits are largely influenced by exogenous activity and experience. A dynamical neuroelectric field maintained by neural activity is proposed as the fundamental information processing substrate of cognitive function. Time series measurements of the neuroelectric field can be collected by scalp sensors and used to mathematically quantify the essential dynamical features of the neuroelectric field by constructing a digital twin of the dynamical system phase space. The multiscale nonlinear values that result can be organized into tensor data structures, from which latent features can be extracted using tensor factorization. These latent features can be mapped to behavioral constructs to derive digital biomarkers. This computational framework provides a robust method for incorporating neurodynamical measures into neuropsychiatric biomarker discovery.

Novel efficient reservoir computing methodologies for regular and irregular time series classification

Time series is a data structure prevalent in a wide range of fields such as healthcare, finance and meteorology. It goes without saying that analyzing time series data holds the key to gaining insight into our day-to-day observations. Among the vast spectrum of time series analysis, time series classification offers the unique opportunity to classify the sequences into their respective categories for the sake of automated detection. To this end, two types of mainstream approaches, recurrent neural networks and distance-based methods, have been commonly employed to address this specific problem. Despite their enormous success, methods like Long Short-Term Memory networks typically require high computational resources. It is largely as a consequence of the nature of backpropagation, driving the search for some backpropagation-free alternatives. Reservoir computing is an instance of recurrent neural networks that is known for its efficiency in processing time series sequences. Therefore, in this article, we will develop two reservoir computing based methods that can effectively deal with regular and irregular time series with minimal computational cost, both while achieving a desirable level of classification accuracy.

Emergence of brain-inspired small-world spiking neural network through neuroevolution

Studies suggest that the brain's high efficiency and low energy consumption may be closely related to its small-world topology and critical dynamics. However, existing efforts on the performance-oriented structural evolution of spiking neural networks (SNNs) are time-consuming and ignore the core structural properties of the brain. Here, we introduce a multi-objective Evolutionary Liquid State Machine (ELSM), which blends the small-world coefficient and criticality to evolve models and guide the emergence of brain-inspired, efficient structures. Experiments reveal ELSM's consistent and comparable performance, achieving 97.23% on NMNIST and outperforming LSM models on MNIST and Fashion-MNIST with 98.12% and 88.81% accuracies, respectively. Further analysis shows its versatility and spontaneous evolution of topologies such as hub nodes, short paths, long-tailed degree distributions, and numerous communities. This study evolves recurrent spiking neural networks into brain-inspired energy-efficient structures, showcasing versatility in multiple tasks and potential for adaptive general artificial intelligence.

Reservoir computing using self-sustained oscillations in a locally connected neural network

Understanding how the structural organization of neural networks influences their computational capabilities is of great interest to both machine learning and neuroscience communities. In our previous work, we introduced a novel learning system, called the reservoir of basal dynamics (reBASICS), which features a modular neural architecture (small-sized random neural networks) capable of reducing chaoticity of neural activity and of producing stable self-sustained limit cycle activities. The integration of these limit cycles is achieved by linear summation of their weights, and arbitrary time series are learned by modulating these weights. Despite its excellent learning performance, interpreting a modular structure of isolated small networks as a brain network has posed a significant challenge. Here, we investigate how local connectivity, a well-known characteristic of brain networks, contributes to reducing neural system chaoticity and generates self-sustained limit cycles based on empirical experiments. Moreover, we present the learning performance of the locally connected reBASICS in two tasks: a motor timing task and a learning task of the Lorenz time series. Although its performance was inferior to that of modular reBASICS, locally connected reBASICS could learn a time series of tens of seconds while the time constant of neural units was ten milliseconds. This work indicates that the locality of connectivity in neural networks may contribute to generation of stable self-sustained oscillations to learn arbitrary long-term time series, as well as the economy of wiring cost.

Precise Traits from Sloppy Components: Perception and the Origin of Phenotypic Response

Organisms perceive their environment and respond. The origin of perception-response traits presents a puzzle. Perception provides no value without response. Response requires perception. Recent advances in machine learning may provide a solution. A randomly connected network creates a reservoir of perceptive information about the recent history of environmental states. In each time step, a relatively small number of inputs drives the dynamics of the relatively large network. Over time, the internal network states retain a memory of past inputs. To achieve a functional response to past states or to predict future states, a system must learn only how to match states of the reservoir to the target response. In the same way, a random biochemical or neural network of an organism can provide an initial perceptive basis. With a solution for one side of the two-step perception-response challenge, evolving an adaptive response may not be so difficult. Two broader themes emerge. First, organisms may often achieve precise traits from sloppy components. Second, evolutionary puzzles often follow the same outlines as the challenges of machine learning. In each case, the basic problem is how to learn, either by artificial computational methods or by natural selection.

A biomarker discovery framework for childhood anxiety

Introduction

Anxiety is the most common manifestation of psychopathology in youth, negatively affecting academic, social, and adaptive functioning and increasing risk for mental health problems into adulthood. Anxiety disorders are diagnosed only after clinical symptoms emerge, potentially missing opportunities to intervene during critical early prodromal periods. In this study, we used a new empirical approach to extracting nonlinear features of the electroencephalogram (EEG), with the goal of discovering differences in brain electrodynamics that distinguish children with anxiety disorders from healthy children. Additionally, we examined whether this approach could distinguish children with externalizing disorders from healthy children and children with anxiety.

Methods

We used a novel supervised tensor factorization method to extract latent factors from repeated multifrequency nonlinear EEG measures in a longitudinal sample of children assessed in infancy and at ages 3, 5, and 7 years of age. We first examined the validity of this method by showing that calendar age is highly correlated with latent EEG complexity factors (r = 0.77). We then computed latent factors separately for distinguishing children with anxiety disorders from healthy controls using a 5-fold cross validation scheme and similarly for distinguishing children with externalizing disorders from healthy controls.

Results

We found that latent factors derived from EEG recordings at age 7 years were required to distinguish children with an anxiety disorder from healthy controls; recordings from infancy, 3 years, or 5 years alone were insufficient. However, recordings from two (5, 7 years) or three (3, 5, 7 years) recordings gave much better results than 7 year recordings alone. Externalizing disorders could be detected using 3- and 5 years EEG data, also giving better results with two or three recordings than any single snapshot. Further, sex assigned at birth was an important covariate that improved accuracy for both disorder groups, and birthweight as a covariate modestly improved accuracy for externalizing disorders. Recordings from infant EEG did not contribute to the classification accuracy for either anxiety or externalizing disorders.

Conclusion

This study suggests that latent factors extracted from EEG recordings in childhood are promising candidate biomarkers for anxiety and for externalizing disorders if chosen at appropriate ages.