Longitudinal Pooling & Consistency Regularization to Model Disease Progression From MRIs

Machine learning with neuroimaging biomarkers: Application in the diagnosis and prediction of drug addiction

Drug abuse is a serious problem worldwide. Owing to intermittent intake of certain substances and the early inconspicuous clinical symptoms, this brings huge challenges for timely diagnosing addiction status and preventing substance use disorders (SUDs). As a non-invasive technique, neuroimaging can capture neurobiological signatures of abnormality in multiple brain regions caused by drug consumption in each clinical stage, like parenchymal morphology alteration as well as aberrant functional activity and connectivity of cerebral areas, making it realizable to diagnosis, prediction and even preemptive therapy of addiction. Machine learning (ML) algorithms primarily used for classification have been extensively applied in analysing medical imaging datasets. Significant neurobiological characteristics employed and revealed by classifiers were used to diagnose addictive states and predict initiation and vulnerability to drug usage, treatment abstinence, relapse and resilience of addicts and the risk of SUD. In this review, we summarize application of ML methods in neuroimaging focusing on addicts' diagnosis of clinical status and risk prediction and elucidate the discriminative neurobiological features from brain electrophysiological, morphological and functional perspectives that contribute most to the classifier, finally highlighting the auxiliary role of ML in addiction treatment.

Liver lesion changes analysis in longitudinal CECT scans by simultaneous deep learning voxel classification with SimU-Net

The identification and quantification of liver lesions changes in longitudinal contrast enhanced CT (CECT) scans is required to evaluate disease status and to determine treatment efficacy in support of clinical decision-making. This paper describes a fully automatic end-to-end pipeline for liver lesion changes analysis in consecutive (prior and current) abdominal CECT scans of oncology patients. The three key novelties are: (1) SimU-Net, a simultaneous multi-channel 3D R2U-Net model trained on pairs of registered scans of each patient that identifies the liver lesions and their changes based on the lesion and healthy tissue appearance differences; (2) a model-based bipartite graph lesions matching method for the analysis of lesion changes at the lesion level; (3) a method for longitudinal analysis of one or more of consecutive scans of a patient based on SimU-Net that handles major liver deformations and incorporates lesion segmentations from previous analysis. To validate our methods, five experimental studies were conducted on a unique dataset of 3491 liver lesions in 735 pairs from 218 clinical abdominal CECT scans of 71 patients with metastatic disease manually delineated by an expert radiologist. The pipeline with the SimU-Net model, trained and validated on 385 pairs and tested on 249 pairs, yields a mean lesion detection recall of 0.86±0.14, a precision of 0.74±0.23 and a lesion segmentation Dice of 0.82±0.14 for lesions > 5 mm. This outperforms a reference standalone 3D R2-UNet mdel that analyzes each scan individually by ∼50% in precision with similar recall and Dice score on the same training and test datasets. For lesions matching, the precision is 0.86±0.18 and the recall is 0.90±0.15. For lesion classification, the specificity is 0.97±0.07, the precision is 0.85±0.31, and the recall is 0.86±0.23. Our new methods provide accurate and comprehensive results that may help reduce radiologists' time and effort and improve radiological oncology evaluation.

Self-supervised learning of neighborhood embedding for longitudinal MRI

In recent years, several deep learning models recommend first to represent Magnetic Resonance Imaging (MRI) as latent features before performing a downstream task of interest (such as classification or regression). The performance of the downstream task generally improves when these latent representations are explicitly associated with factors of interest. For example, we derived such a representation for capturing brain aging by applying self-supervised learning to longitudinal MRIs and then used the resulting encoding to automatically identify diseases accelerating the aging of the brain. We now propose a refinement of this representation by replacing the linear modeling of brain aging with one that is consistent in local neighborhoods in the latent space. Called Longitudinal Neighborhood Embedding (LNE), we derive an encoding so that neighborhoods are age-consistent (i.e., brain MRIs of different subjects with similar brain ages are in close proximity of each other) and progression-consistent, i.e., the latent space is defined by a smooth trajectory field where each trajectory captures changes in brain ages between a pair of MRIs extracted from a longitudinal sequence. To make the problem computationally tractable, we further propose a strategy for mini-batch sampling so that the resulting local neighborhoods accurately approximate the ones that would be defined based on the whole cohort. We evaluate LNE on three different downstream tasks: (1) to predict chronological age from T1-w MRI of 274 healthy subjects participating in a study at SRI International; (2) to distinguish Normal Control (NC) from Alzheimer's Disease (AD) and stable Mild Cognitive Impairment (sMCI) from progressive Mild Cognitive Impairment (pMCI) based on T1-w MRI of 632 participants of the Alzheimer's Disease Neuroimaging Initiative (ADNI); and (3) to distinguish no-to-low from moderate-to-heavy alcohol drinkers based on fractional anisotropy derived from diffusion tensor MRIs of 764 adolescents recruited by the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA). Across the three data sets, the visualization of the smooth trajectory vector fields and superior accuracy on downstream tasks demonstrate the strength of the proposed method over existing self-supervised methods in extracting information related to brain aging, which could help study the impact of substance use and neurodegenerative disorders. The code is available at https://github.com/ouyangjiahong/longitudinal-neighbourhood-embedding.

Disentangling Normal Aging From Severity of Disease via Weak Supervision on Longitudinal MRI

The continuous progression of neurological diseases are often categorized into conditions according to their severity. To relate the severity to changes in brain morphometry, there is a growing interest in replacing these categories with a continuous severity scale that longitudinal MRIs are mapped onto via deep learning algorithms. However, existing methods based on supervised learning require large numbers of samples and those that do not, such as self-supervised models, fail to clearly separate the disease effect from normal aging. Here, we propose to explicitly disentangle those two factors via weak-supervision. In other words, training is based on longitudinal MRIs being labelled either normal or diseased so that the training data can be augmented with samples from disease categories that are not of primary interest to the analysis. We do so by encouraging trajectories of controls to be fully encoded by the direction associated with brain aging. Furthermore, an orthogonal direction linked to disease severity captures the residual component from normal aging in the diseased cohort. Hence, the proposed method quantifies disease severity and its progression speed in individuals without knowing their condition. We apply the proposed method on data from the Alzheimer's Disease Neuroimaging Initiative (ADNI, N =632 ). We then show that the model properly disentangled normal aging from the severity of cognitive impairment by plotting the resulting disentangled factors of each subject and generating simulated MRIs for a given chronological age and condition. Moreover, our representation obtains higher balanced accuracy when used for two downstream classification tasks compared to other pre-training approaches. The code for our weak-supervised approach is available at https://github.com/ouyangjiahong/longitudinal-direction-disentangle.

Multiple Instance Neuroimage Transformer

For the first time, we propose using a multiple instance learning based convolution-free transformer model, called Multiple Instance Neuroimage Transformer (MINiT), for the classification of T1-weighted (T1w) MRIs. We first present several variants of transformer models adopted for neuroimages. These models extract non-overlapping 3D blocks from the input volume and perform multi-headed self-attention on a sequence of their linear projections. MINiT, on the other hand, treats each of the non-overlapping 3D blocks of the input MRI as its own instance, splitting it further into non-overlapping 3D patches, on which multi-headed self-attention is computed. As a proof-of-concept, we evaluate the efficacy of our model by training it to identify sex from T1w-MRIs of two public datasets: Adolescent Brain Cognitive Development (ABCD) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA). The learned attention maps highlight voxels contributing to identifying sex differences in brain morphometry. The code is available at https://github.com/singlaayush/MINIT.

Alcohol and the Adolescent Brain: What We've Learned and Where the Data Are Taking Us

This article is part of a Festschrift commemorating the 50th anniversary of the National Institute on Alcohol Abuse and Alcoholism (NIAAA). Established in 1970, first as part of the National Institute of Mental Health and later as an independent institute of the National Institutes of Health, NIAAA today is the world's largest funding agency for alcohol research. In addition to its own intramural research program, NIAAA supports the entire spectrum of innovative basic, translational, and clinical research to advance the diagnosis, prevention, and treatment of alcohol use disorder and alcohol-related problems. To celebrate the anniversary, NIAAA hosted a 2-day symposium, "Alcohol Across the Lifespan: 50 Years of Evidence-Based Diagnosis, Prevention, and Treatment Research," devoted to key topics within the field of alcohol research. This article is based on Dr. Tapert's presentation at the event. NIAAA Director George F. Koob, Ph.D., serves as editor of the Festschrift.

Self-Supervised Longitudinal Neighbourhood Embedding

Longitudinal MRIs are often used to capture the gradual deterioration of brain structure and function caused by aging or neurological diseases. Analyzing this data via machine learning generally requires a large number of ground-truth labels, which are often missing or expensive to obtain. Reducing the need for labels, we propose a self-supervised strategy for representation learning named Longitudinal Neighborhood Embedding (LNE). Motivated by concepts in contrastive learning, LNE explicitly models the similarity between trajectory vectors across different subjects. We do so by building a graph in each training iteration defining neighborhoods in the latent space so that the progression direction of a subject follows the direction of its neighbors. This results in a smooth trajectory field that captures the global morphological change of the brain while maintaining the local continuity. We apply LNE to longitudinal T1w MRIs of two neuroimaging studies: a dataset composed of 274 healthy subjects, and Alzheimer's Disease Neuroimaging Initiative (ADNI, N = 632). The visualization of the smooth trajectory vector field and superior performance on downstream tasks demonstrate the strength of the proposed method over existing self-supervised methods in extracting information associated with normal aging and in revealing the impact of neurodegenerative disorders. The code is available at https://github.com/ouyangjiahong/longitudinal-neighbourhood-embedding.

Longitudinal self-supervised learning

Machine learning analysis of longitudinal neuroimaging data is typically based on supervised learning, which requires large number of ground-truth labels to be informative. As ground-truth labels are often missing or expensive to obtain in neuroscience, we avoid them in our analysis by combing factor disentanglement with self-supervised learning to identify changes and consistencies across the multiple MRIs acquired of each individual over time. Specifically, we propose a new definition of disentanglement by formulating a multivariate mapping between factors (e.g., brain age) associated with an MRI and a latent image representation. Then, factors that evolve across acquisitions of longitudinal sequences are disentangled from that mapping by self-supervised learning in such a way that changes in a single factor induce change along one direction in the representation space. We implement this model, named Longitudinal Self-Supervised Learning (LSSL), via a standard autoencoding structure with a cosine loss to disentangle brain age from the image representation. We apply LSSL to two longitudinal neuroimaging studies to highlight its strength in extracting the brain-age information from MRI and revealing informative characteristics associated with neurodegenerative and neuropsychological disorders. Moreover, the representations learned by LSSL facilitate supervised classification by recording faster convergence and higher (or similar) prediction accuracy compared to several other representation learning techniques.

Going Beyond Saliency Maps: Training Deep Models to Interpret Deep Models

Interpretability is a critical factor in applying complex deep learning models to advance the understanding of brain disorders in neuroimaging studies. To interpret the decision process of a trained classifier, existing techniques typically rely on saliency maps to quantify the voxel-wise or feature-level importance for classification through partial derivatives. Despite providing some level of localization, these maps are not human-understandable from the neuroscience perspective as they often do not inform the specific type of morphological changes linked to the brain disorder. Inspired by the image-to-image translation scheme, we propose to train simulator networks to inject (or remove) patterns of the disease into a given MRI based on a warping operation, such that the classifier increases (or decreases) its confidence in labeling the simulated MRI as diseased. To increase the robustness of training, we propose to couple the two simulators into a unified model based on conditional convolution. We applied our approach to interpreting classifiers trained on a synthetic dataset and two neuroimaging datasets to visualize the effect of Alzheimer's disease and alcohol dependence. Compared to the saliency maps generated by baseline approaches, our simulations and visualizations based on the Jacobian determinants of the warping field reveal meaningful and understandable patterns related to the diseases.