Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI

Computation-through-Dynamics Benchmark: Simulated datasets and quality metrics for dynamical models of neural activity

A primary goal of systems neuroscience is to discover how ensembles of neurons transform inputs into goal-directed behavior, a process known as neural computation. A powerful framework for understanding neural computation uses neural dynamics - the rules that describe the temporal evolution of neural activity - to explain how goal-directed input-output transformations occur. As dynamical rules are not directly observable, we need computational models that can infer neural dynamics from recorded neural activity. We typically validate such models using synthetic datasets with known ground-truth dynamics, but unfortunately existing synthetic datasets don't reflect fundamental features of neural computation and are thus poor proxies for neural systems. Further, the field lacks validated metrics for quantifying the accuracy of the dynamics inferred by models. The Computation-through-Dynamics Benchmark (CtDB) fills these critical gaps by providing: 1) synthetic datasets that reflect computational properties of biological neural circuits, 2) interpretable metrics for quantifying model performance, and 3) a standardized pipeline for training and evaluating models with or without known external inputs. In this manuscript, we demonstrate how CtDB can help guide the development, tuning, and troubleshooting of neural dynamics models. In summary, CtDB provides a critical platform for model developers to better understand and characterize neural computation through the lens of dynamics.

Sex-dependent nonlinear Granger connectivity patterns of brain aging in healthy population

BACKGROUND

Brain aging is a complex process that involves functional alterations in multiple subnetworks and brain regions. However, most previous studies investigating aging-related functional connectivity (FC) changes using resting-state functional magnetic resonance images (rs-fMRIs) have primarily focused on the linear correlation between brain subnetworks, ignoring the nonlinear casual properties of fMRI signals.

METHODS

We introduced the neural Granger causality technique to investigate the sex-dependent nonlinear Granger connectivity (NGC) during aging on a publicly available dataset of 227 healthy participants acquired cross-sectionally in Leipzig, Germany.

RESULTS

Our findings indicate that brain aging may cause widespread declines in NGC at both regional and subnetwork scales. These findings exhibit high reproducibility across different network sparsities, demonstrating the efficacy of static and dynamic analysis strategies. Females exhibit greater heterogeneity and reduced stability in NGC compared to males during aging, especially the NGC between the visual network and other subnetworks. Besides, NGC strengths can well reflect the individual cognitive function, which may therefore work as a sensitive metric in cognition-related experiments for individual-scale or group-scale mechanism understanding.

CONCLUSION

These findings indicate that NGC analysis is a potent tool for identifying sex-dependent brain aging patterns. Our results offer valuable perspectives that could substantially enhance the understanding of sex differences in neurological diseases in the future, especially in degenerative disorders.

The multi-frequency decomposition entropy learning for nonlinear fMRI data analysis

Functional magnetic resonance imaging (fMRI) have been widely adopted to explore the underlying neural mechanisms between psychiatric disorders which share common neurobiology and clinical manifestations. However, the existing studies mainly focus on linear relationships and ignore nonlinear contributions. To address the above issues, we propose a new method named multi-frequency decomposition entropy (MDE) learning for inferring nonlinear functional connectivity between brain regions. Firstly, the variational mode decomposition was used to divide fMRI data into five groups of frequency. Next, the copula entropy was used to calculate the nonlinear relationship between brain regions in each frequency group, and then the best important nonlinear relationships were screen out by using statistical t-test. Lastly, a gyrus importance index was proposed to reflect the distribution trend of gyri in different frequency groups. The results of applying MDE for the fMRI data analysis of schizophrenia, bipolar disorder, and attention-deficit hyperactivity disorder showed that the difference between the three groups of patient and healthy control is large at the hub nodes, and the nonlinear relationship between the patient groups is weak when they are at the same hub node. In addition, each disease exhibits unique characteristics compared with other diseases and healthy control. In a word, the nonlinear functional connectivity of different frequency groups reflect the differences and commonalities between diseases and reveal possible discriminating biomarkers among mental diseases.

Kalman contrastive unsupervised representation learning

We first propose a Kalman contrastive (KalCo) framework for unsupervised representation learning by dictionary lookup. It builds a dynamic dictionary of encoded representation keys with a queue and a Kalman filter encoder, to which the encoded queries are matched. The large and consistent dictionaries built this way increase the accuracy of KalCo to values much higher than those of the famous momentum contrastive (MoCo) unsupervised learning, which is actually a very simplified version of KalCo with only a fixed scaler momentum coefficient. For a standard pretext task of instance discrimination on the ImageNet-1M (IN-1M) dataset; e.g., KalCo yields an accuracy of 80%, compared to 55% for MoCo. Similar results are obtained also on Instagram-1B (IG-1B). For the same task on a bunch of OpenfMRI datasets, the accuracy is 84%. We then upgrade KalCo to KalCo v2 by using an MLP projection head and more data augmentation, along also with a larger memory bank. The accuracy of KalCo v2 is around the even more impressive amounts of 90% on IN-1M and IG-1B, and 95% on OpenfMRI, the first being about 3% higher than those of three most-cited recent alternatives.

Characterizing the dynamics, reactivity and controllability of moods in depression with a Kalman filter

BACKGROUND

Mood disorders involve a complex interplay between multifaceted internal emotional states, and complex external inputs. Dynamical systems theory suggests that this interplay between aspects of moods and environmental stimuli may hence determine key psychopathological features of mood disorders, including the stability of mood states, the response to external inputs, how controllable mood states are, and what interventions are most likely to be effective. However, a comprehensive computational approach to all these aspects has not yet been undertaken.

METHODS

Here, we argue that the combination of ecological momentary assessments (EMA) with a well-established dynamical systems framework-the humble Kalman filter-enables a comprehensive account of all these aspects. We first introduce the key features of the Kalman filter and optimal control theory and their relationship to aspects of psychopathology. We then examine the psychometric and inferential properties of combining EMA data with Kalman filtering across realistic scenarios. Finally, we apply the Kalman filter to a series of EMA datasets comprising over 700 participants with and without symptoms of depression.

RESULTS

The results show a naive Kalman filter approach performs favourably compared to the standard vector autoregressive approach frequently employed, capturing key aspects of the data better. Furthermore, it suggests that the depressed state involves alterations to interactions between moods; alterations to how moods responds to external inputs; and as a result an alteration in how controllable mood states are. We replicate these findings qualitatively across datasets and explore an extension to optimal control theory to guide therapeutic interventions.

CONCLUSIONS

Mood dynamics are richly and profoundly altered in depressed states. The humble Kalman filter is a well-established, rich framework to characterise mood dynamics. Its application to EMA data is valid; straightforward; and likely to result in substantial novel insights both into mechanisms and treatments.

Quantifying brain-functional dynamics using deep dynamical systems: Technical considerations

Both mental health and mental illness unfold in complex and unpredictable ways. Novel artificial intelligence approaches from the area of dynamical systems reconstruction can characterize such dynamics and help understand the underlying brain mechanisms, which can also be used as potential biomarkers. However, applying deep learning to model dynamical systems at the individual level must overcome numerous computational challenges to be reproducible and clinically useful. In this study, we performed an extensive analysis of these challenges using generative modeling of brain dynamics from fMRI data as an example and demonstrated their impact on classifying patients with schizophrenia and major depression. This study highlights the tendency of deep learning models to identify functionally unique solutions during parameter optimization, which severely impacts the reproducibility of downstream predictions. We hope this study guides the future development of individual-level generative models and similar machine learning approaches aimed at identifying reproducible biomarkers of mental illness.

Parameter estimation in a whole-brain network model of epilepsy: Comparison of parallel global optimization solvers

The Virtual Epileptic Patient (VEP) refers to a computer-based representation of a patient with epilepsy that combines personalized anatomical data with dynamical models of abnormal brain activities. It is capable of generating spatio-temporal seizure patterns that resemble those recorded with invasive methods such as stereoelectro EEG data, allowing for the evaluation of clinical hypotheses before planning surgery. This study highlights the effectiveness of calibrating VEP models using a global optimization approach. The approach utilizes SaCeSS, a cooperative metaheuristic algorithm capable of parallel computation, to yield high-quality solutions without requiring excessive computational time. Through extensive benchmarking on synthetic data, our proposal successfully solved a set of different configurations of VEP models, demonstrating better scalability and superior performance against other parallel solvers. These results were further enhanced using a Bayesian optimization framework for hyperparameter tuning, with significant gains in terms of both accuracy and computational cost. Additionally, we added a scalable uncertainty quantification phase after model calibration, and used it to assess the variability in estimated parameters across different problems. Overall, this study has the potential to improve the estimation of pathological brain areas in drug-resistant epilepsy, thereby to inform the clinical decision-making process.

Reconstructing computational system dynamics from neural data with recurrent neural networks

Computational models in neuroscience usually take the form of systems of differential equations. The behaviour of such systems is the subject of dynamical systems theory. Dynamical systems theory provides a powerful mathematical toolbox for analysing neurobiological processes and has been a mainstay of computational neuroscience for decades. Recently, recurrent neural networks (RNNs) have become a popular machine learning tool for studying the non-linear dynamics of neural and behavioural processes by emulating an underlying system of differential equations. RNNs have been routinely trained on similar behavioural tasks to those used for animal subjects to generate hypotheses about the underlying computational mechanisms. By contrast, RNNs can also be trained on the measured physiological and behavioural data, thereby directly inheriting their temporal and geometrical properties. In this way they become a formal surrogate for the experimentally probed system that can be further analysed, perturbed and simulated. This powerful approach is called dynamical system reconstruction. In this Perspective, we focus on recent trends in artificial intelligence and machine learning in this exciting and rapidly expanding field, which may be less well known in neuroscience. We discuss formal prerequisites, different model architectures and training approaches for RNN-based dynamical system reconstructions, ways to evaluate and validate model performance, how to interpret trained models in a neuroscience context, and current challenges.

Resolving the prefrontal mechanisms of adaptive cognitive behaviors: A cross-species perspective

The prefrontal cortex (PFC) enables a staggering variety of complex behaviors, such as planning actions, solving problems, and adapting to new situations according to external information and internal states. These higher-order abilities, collectively defined as adaptive cognitive behavior, require cellular ensembles that coordinate the tradeoff between the stability and flexibility of neural representations. While the mechanisms underlying the function of cellular ensembles are still unclear, recent experimental and theoretical studies suggest that temporal coordination dynamically binds prefrontal neurons into functional ensembles. A so far largely separate stream of research has investigated the prefrontal efferent and afferent connectivity. These two research streams have recently converged on the hypothesis that prefrontal connectivity patterns influence ensemble formation and the function of neurons within ensembles. Here, we propose a unitary concept that, leveraging a cross-species definition of prefrontal regions, explains how prefrontal ensembles adaptively regulate and efficiently coordinate multiple processes in distinct cognitive behaviors.

Characterization of regional differences in resting-state fMRI with a data-driven network model of brain dynamics

Model-based data analysis of whole-brain dynamics links the observed data to model parameters in a network of neural masses. Recently, studies focused on the role of regional variance of model parameters. Such analyses however necessarily depend on the properties of preselected neural mass model. We introduce a method to infer from the functional data both the neural mass model representing the regional dynamics and the region- and subject-specific parameters while respecting the known network structure. We apply the method to human resting-state fMRI. We find that the underlying dynamics can be described as noisy fluctuations around a single fixed point. The method reliably discovers three regional parameters with clear and distinct role in the dynamics, one of which is strongly correlated with the first principal component of the gene expression spatial map. The present approach opens a novel way to the analysis of resting-state fMRI with possible applications for understanding the brain dynamics during aging or neurodegeneration.

Altered nonlinear Granger causality interactions in the large-scale brain networks of patients with schizophrenia

Objective.It has been demonstrated that schizophrenia (SZ) is characterized by functional dysconnectivity involving extensive brain networks. However, the majority of previous studies utilizing resting-state functional magnetic resonance imaging (fMRI) to infer abnormal functional connectivity (FC) in patients with SZ have focused on the linear correlation that one brain region may influence another, ignoring the inherently nonlinear properties of fMRI signals.Approach. In this paper, we present a neural Granger causality (NGC) technique for examining the changes in SZ's nonlinear causal couplings. We develop static and dynamic NGC-based analyses of large-scale brain networks at several network levels, estimating complicated temporal and causal relationships in SZ patients.Main results. We find that the NGC-based FC matrices can detect large and significant differences between the SZ and healthy control groups at both the regional and subnetwork scales. These differences are persistent and significantly overlapped at various network sparsities regardless of whether the brain networks were built using static or dynamic techniques. In addition, compared to controls, patients with SZ exhibited extensive NGC confusion patterns throughout the entire brain.Significance. These findings imply that the NGC-based FCs may be a useful method for quantifying the abnormalities in the causal influences of patients with SZ, hence shedding fresh light on the pathophysiology of this disorder.

A large-scale neural network training framework for generalized estimation of single-trial population dynamics

Achieving state-of-the-art performance with deep neural population dynamics models requires extensive hyperparameter tuning for each dataset. AutoLFADS is a model-tuning framework that automatically produces high-performing autoencoding models on data from a variety of brain areas and tasks, without behavioral or task information. We demonstrate its broad applicability on several rhesus macaque datasets: from motor cortex during free-paced reaching, somatosensory cortex during reaching with perturbations, and dorsomedial frontal cortex during a cognitive timing task.

It's about time: Linking dynamical systems with human neuroimaging to understand the brain

Most human neuroscience research to date has focused on statistical approaches that describe stationary patterns of localized neural activity or blood flow. While these patterns are often interpreted in light of dynamic, information-processing concepts, the static, local, and inferential nature of the statistical approach makes it challenging to directly link neuroimaging results to plausible underlying neural mechanisms. Here, we argue that dynamical systems theory provides the crucial mechanistic framework for characterizing both the brain's time-varying quality and its partial stability in the face of perturbations, and hence, that this perspective can have a profound impact on the interpretation of human neuroimaging results and their relationship with behavior. After briefly reviewing some key terminology, we identify three key ways in which neuroimaging analyses can embrace a dynamical systems perspective: by shifting from a local to a more global perspective, by focusing on dynamics instead of static snapshots of neural activity, and by embracing modeling approaches that map neural dynamics using "forward" models. Through this approach, we envisage ample opportunities for neuroimaging researchers to enrich their understanding of the dynamic neural mechanisms that support a wide array of brain functions, both in health and in the setting of psychopathology.

Generative Models of Brain Dynamics

This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.

Multimodal Assessment of Schizophrenia and Depression Utilizing Video, Acoustic, Locomotor, Electroencephalographic, and Heart Rate Technology: Protocol for an Observational Study

Background

Current standards of psychiatric assessment and diagnostic evaluation rely primarily on the clinical subjective interpretation of a patient’s outward manifestations of their internal state. While psychometric tools can help to evaluate these behaviors more systematically, the tools still rely on the clinician’s interpretation of what are frequently nuanced speech and behavior patterns. With advances in computing power, increased availability of clinical data, and improving resolution of recording and sensor hardware (including acoustic, video, accelerometer, infrared, and other modalities), researchers have begun to demonstrate the feasibility of cutting-edge technologies in aiding the assessment of psychiatric disorders.

Objective

We present a research protocol that utilizes facial expression, eye gaze, voice and speech, locomotor, heart rate, and electroencephalography monitoring to assess schizophrenia symptoms and to distinguish patients with schizophrenia from those with other psychiatric disorders and control subjects.

Methods

We plan to recruit three outpatient groups: (1) 50 patients with schizophrenia, (2) 50 patients with unipolar major depressive disorder, and (3) 50 individuals with no psychiatric history. Using an internally developed semistructured interview, psychometrically validated clinical outcome measures, and a multimodal sensing system utilizing video, acoustic, actigraphic, heart rate, and electroencephalographic sensors, we aim to evaluate the system’s capacity in classifying subjects (schizophrenia, depression, or control), to evaluate the system’s sensitivity to within-group symptom severity, and to determine if such a system can further classify variations in disorder subtypes.

Results

Data collection began in July 2020 and is expected to continue through December 2022.

Conclusions

If successful, this study will help advance current progress in developing state-of-the-art technology to aid clinical psychiatric assessment and treatment. If our findings suggest that these technologies are capable of resolving diagnoses and symptoms to the level of current psychometric testing and clinician judgment, we would be among the first to develop a system that can eventually be used by clinicians to more objectively diagnose and assess schizophrenia and depression with the possibility of less risk of bias. Such a tool has the potential to improve accessibility to care; to aid clinicians in objectively evaluating diagnoses, severity of symptoms, and treatment efficacy through time; and to reduce treatment-related morbidity.

International Registered Report Identifier (IRRID)

DERR1-10.2196/36417

Classification of amyotrophic lateral sclerosis by brain volume, connectivity, and network dynamics

Emerging studies corroborate the importance of neuroimaging biomarkers and machine learning to improve diagnostic classification of amyotrophic lateral sclerosis (ALS). While most studies focus on structural data, recent studies assessing functional connectivity between brain regions by linear methods highlight the role of brain function. These studies have yet to be combined with brain structure and nonlinear functional features. We investigate the role of linear and nonlinear functional brain features, and the benefit of combining brain structure and function for ALS classification. ALS patients (N = 97) and healthy controls (N = 59) underwent structural and functional resting state magnetic resonance imaging. Based on key hubs of resting state networks, we defined three feature sets comprising brain volume, resting state functional connectivity (rsFC), as well as (nonlinear) resting state dynamics assessed via recurrent neural networks. Unimodal and multimodal random forest classifiers were built to classify ALS. Out-of-sample prediction errors were assessed via five-fold cross-validation. Unimodal classifiers achieved a classification accuracy of 56.35-61.66%. Multimodal classifiers outperformed unimodal classifiers achieving accuracies of 62.85-66.82%. Evaluating the ranking of individual features' importance scores across all classifiers revealed that rsFC features were most dominant in classification. While univariate analyses revealed reduced rsFC in ALS patients, functional features more generally indicated deficits in information integration across resting state brain networks in ALS. The present work undermines that combining brain structure and function provides an additional benefit to diagnostic classification, as indicated by multimodal classifiers, while emphasizing the importance of capturing both linear and nonlinear functional brain properties to identify discriminative biomarkers of ALS.

Representation learning of resting state fMRI with variational autoencoder

Resting state functional magnetic resonance imaging (rsfMRI) data exhibits complex but structured patterns. However, the underlying origins are unclear and entangled in rsfMRI data. Here we establish a variational auto-encoder, as a generative model trainable with unsupervised learning, to disentangle the unknown sources of rsfMRI activity. After being trained with large data from the Human Connectome Project, the model has learned to represent and generate patterns of cortical activity and connectivity using latent variables. The latent representation and its trajectory represent the spatiotemporal characteristics of rsfMRI activity. The latent variables reflect the principal gradients of the latent trajectory and drive activity changes in cortical networks. Representational geometry captured as covariance or correlation between latent variables, rather than cortical connectivity, can be used as a more reliable feature to accurately identify subjects from a large group, even if only a short period of data is available in each subject. Our results demonstrate that VAE is a valuable addition to existing tools, particularly suited for unsupervised representation learning of resting state fMRI activity.

Deep learning for small and big data in psychiatry

Psychiatry today must gain a better understanding of the common and distinct pathophysiological mechanisms underlying psychiatric disorders in order to deliver more effective, person-tailored treatments. To this end, it appears that the analysis of 'small' experimental samples using conventional statistical approaches has largely failed to capture the heterogeneity underlying psychiatric phenotypes. Modern algorithms and approaches from machine learning, particularly deep learning, provide new hope to address these issues given their outstanding prediction performance in other disciplines. The strength of deep learning algorithms is that they can implement very complicated, and in principle arbitrary predictor-response mappings efficiently. This power comes at a cost, the need for large training (and test) samples to infer the (sometimes over millions of) model parameters. This appears to be at odds with the as yet rather 'small' samples available in psychiatric human research to date (n < 10,000), and the ambition of predicting treatment at the single subject level (n = 1). Here, we aim at giving a comprehensive overview on how we can yet use such models for prediction in psychiatry. We review how machine learning approaches compare to more traditional statistical hypothesis-driven approaches, how their complexity relates to the need of large sample sizes, and what we can do to optimally use these powerful techniques in psychiatric neuroscience.

Advances in the computational understanding of mental illness

Computational psychiatry is a rapidly growing field attempting to translate advances in computational neuroscience and machine learning into improved outcomes for patients suffering from mental illness. It encompasses both data-driven and theory-driven efforts. Here, recent advances in theory-driven work are reviewed. We argue that the brain is a computational organ. As such, an understanding of the illnesses arising from it will require a computational framework. The review divides work up into three theoretical approaches that have deep mathematical connections: dynamical systems, Bayesian inference and reinforcement learning. We discuss both general and specific challenges for the field, and suggest ways forward.