Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review

Artificial intelligence in insanity evaluation. Potential opportunities and current challenges

The formulation of a scientific opinion on whether the individual who committed a crime should be held responsible for his/her actions or should be considered not responsible by reason of insanity is very difficult. Indeed, forensic psychopathological decision on insanity is highly prone to errors and is affected by human cognitive biases, resulting in low inter-rater reliability. In this context, artificial intelligence can be extremely useful to improve the inter-subjectivity of insanity evaluation. In this paper, we discuss the possible applications of artificial intelligence in this field as well as the challenges and pitfalls that hamper the effective implementation of AI in insanity evaluation. In particular, thus far, it is possible to apply only supervised algorithms without knowing which is the ground truth and which data should be used to train and test the algorithms. In addition, it is not known which percentage of accuracy of the algorithms is sufficient to support partial or total insanity, nor which are the boundaries between sanity and partial or total insanity. Finally, ethical aspects have not been sufficiently investigated. We conclude that these pitfalls should be resolved before AI can be safely and reliably applied in criminal trials.

Bias recognition and mitigation strategies in artificial intelligence healthcare applications

Artificial intelligence (AI) is delivering value across all aspects of clinical practice. However, bias may exacerbate healthcare disparities. This review examines the origins of bias in healthcare AI, strategies for mitigation, and responsibilities of relevant stakeholders towards achieving fair and equitable use. We highlight the importance of systematically identifying bias and engaging relevant mitigation activities throughout the AI model lifecycle, from model conception through to deployment and longitudinal surveillance.

Integrative diagnosis of psychiatric conditions using ChatGPT and fMRI data

BACKGROUND

Traditional diagnostic methods for psychiatric disorders often rely on subjective assessments, leading to inconsistent diagnoses. Integrating advanced natural language processing (NLP) techniques with neuroimaging data may improve diagnostic accuracy.

METHODS

We propose a novel approach that uses ChatGPT to conduct interactive patient interviews, capturing nuanced emotional and psychological data. By analyzing these dialogues using NLP, we generate a comprehensive feature matrix. This matrix, combined with 4D fMRI data, is input into a neural network to predict psychiatric diagnoses. We conducted comparative analysis with survey-based and app-based methods, providing detailed statistical validation.

RESULTS

Our model achieved an accuracy of 85.7%, significantly outperforming traditional methods. Statistical analysis confirmed the superiority of the ChatGPT-based approach in capturing nuanced patient information, with p-values indicating significant improvements over baseline models.

CONCLUSIONS

Integrating NLP-driven patient interactions with fMRI data offers a promising approach to psychiatric diagnosis, enhancing precision and reliability. This method could advance clinical practice by providing a more objective and comprehensive diagnostic tool, although more research is needed to generalize these findings.

Applications of MRI in Schizophrenia: Current Progress in Establishing Clinical Utility

Schizophrenia is a severe mental illness that significantly impacts the lives of affected individuals and with increasing mortality rates. Early detection and intervention are crucial for improving outcomes but the lack of validated biomarkers poses great challenges in such efforts. The use of magnetic resonance imaging (MRI) in schizophrenia enables the investigation of the disorder's etiological and neuropathological substrates in vivo. After decades of research, promising findings of MRI have been shown to aid in screening high-risk individuals and predicting illness onset, and predicting symptoms and treatment outcomes of schizophrenia. The integration of machine learning and deep learning techniques makes it possible to develop intelligent diagnostic and prognostic tools with extracted or selected imaging features. In this review, we aimed to provide an overview of current progress and prospects in establishing clinical utility of MRI in schizophrenia. We first provided an overview of MRI findings of brain abnormalities that might underpin the symptoms or treatment response process in schizophrenia patients. Then, we summarized the ongoing efforts in the computer-aided utility of MRI in schizophrenia and discussed the gap between MRI research findings and real-world applications. Finally, promising pathways to promote clinical translation were provided. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 3.

The Transition From Homogeneous to Heterogeneous Machine Learning in Neuropsychiatric Research

Despite the advantage of neuroimaging-based machine learning (ML) models as pivotal tools for investigating brain-behavior relationships in neuropsychiatric studies, these data-driven predictive approaches have yet to yield substantial, clinically actionable insights for mental health care. A notable impediment lies in the inadequate accommodation of most ML research to the natural heterogeneity within large samples. Although commonly thought of as individual-level analyses, many ML algorithms are unimodal and homogeneous and thus incapable of capturing the potentially heterogeneous relationships between biology and psychopathology. We review the current landscape of computational research targeting population heterogeneity and argue that there is a need to expand from brain subtyping and behavioral phenotyping to analyses that focus on heterogeneity at the relational level. To this end, we review and suggest several existing ML models with the capacity to discern how external environmental and sociodemographic factors moderate the brain-behavior mapping function in a data-driven fashion. These heterogeneous ML models hold promise for enhancing the discovery of individualized brain-behavior associations and advancing precision psychiatry.

A scoping review of magnetic resonance angiography and perfusion image synthesis

Introduction

Deregulation of the cerebrovascular system has been linked to neurodegeneration, part of a putative causal pathway into etiologies such as Alzheimer's disease (AD). In medical imaging, time-of-flight magnetic resonance angiography (TOF-MRA) and perfusion MRI are the most common modalities used to study this system. However, due to lack of resources, many large-scale studies of AD are not acquiring these images; this creates a conundrum, as the lack of evidence limits our knowledge of the interaction between the cerebrovascular system and AD. Deep learning approaches have been used in recent developments to generate synthetic medical images from existing contrasts. In this review, we study the use of artificial intelligence in the generation of synthetic TOF-MRA and perfusion-related images from existing neuroanatomical and neurovascular acquisitions for the study of the cerebrovascular system.

Method

Following the PRISMA reporting guidelines we conducted a scoping review of 729 studies relating to image synthesis of TOF-MRA or perfusion imaging, from which 13 met our criteria.

Results

Studies showed that T1-w, T2-w, and FLAIR can be used to synthesize perfusion map and TOF-MRA. Other studies demonstrated that synthetic images could have a greater signal-to-noise ratio compared to real images and that some models trained on healthy subjects could generalize their outputs to an unseen population, such as stroke patients.

Discussion

These findings suggest that generating TOF-MRA and perfusion MRI images holds significant potential for enhancing neurovascular studies, particularly in cases where direct acquisition is not feasible. This approach could provide valuable insights for retrospective studies of several cerebrovascular related diseases such as stroke and AD. While promising, further research is needed to assess their sensitivity and specificity, and ensure their applicability across diverse populations. The use of models to generate TOF-MRA and perfusion MRI using commonly acquired data could be the key for the retrospective study of the cerebrovascular system and elucidate its role in the development of dementia.

Racial and ethnic socioenvironmental inequity and neuroimaging in psychiatry: a brief review of the past and recommendations for the future

Neuroimaging is a major tool that holds immense translational potential for understanding psychiatric disorder phenomenology and treatment. However, although epidemiological and social research highlights the many ways inequity and representativeness influences mental health, there is a lack of consideration of how such issues may impact neuroimaging features in psychiatric research. More specifically, the potential extent to which racialized inequities may affect underlying neurobiology and impact the generalizability of neural models of disorders is unclear. The present review synthesizes research focused on understanding the potential consequences of racial/ethnic inequities relevant to neuroimaging in psychiatry. We first discuss historical and contemporary drivers of inequities that persist today. We then discuss the neurobiological consequences of these inequities as revealed through current research, and note emergent research demonstrating the impact such inequities have on our ability to use neuroimaging to understand psychiatric disease. We end with a set of recommendations and practices to move the field towards more equitable approaches that will advance our abilities to develop truly generalizable neurobiological models of psychiatric disorders.

A new era in cognitive neuroscience: the tidal wave of artificial intelligence (AI)

Translating artificial intelligence techniques into the realm of cognitive neuroscience holds promise for significant breakthroughs in our ability to probe the intrinsic mechanisms of the brain. The recent unprecedented development of robust AI models is changing how and what we understand about the brain. In this Editorial, we invite contributions for a BMC Neuroscience Collection on "AI and Cognitive Neuroscience".

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia

Background

Schizophrenia is a polygenic disorder associated with changes in brain structure and function. Integrating macroscale brain features with microscale genetic data may provide a more complete overview of the disease etiology and may serve as potential diagnostic markers for schizophrenia.

Objective

We aim to systematically evaluate the impact of multi-scale neuroimaging and transcriptomic data fusion in schizophrenia classification models.

Methods

We collected brain imaging data and blood RNA sequencing data from 43 patients with schizophrenia and 60 age- and gender-matched healthy controls, and we extracted multi-omics features of macroscale brain morphology, brain structural and functional connectivity, and gene transcription of schizophrenia risk genes. Multi-scale data fusion was performed using a machine learning integration framework, together with several conventional machine learning methods and neural networks for patient classification.

Results

We found that multi-omics data fusion in conventional machine learning models achieved the highest accuracy (AUC ~0.76-0.92) in contrast to the single-modality models, with AUC improvements of 8.88 to 22.64%. Similar findings were observed for the neural network, showing an increase of 16.57% for the multimodal classification model (accuracy 71.43%) compared to the single-modal average. In addition, we identified several brain regions in the left posterior cingulate and right frontal pole that made a major contribution to disease classification.

Conclusion

We provide empirical evidence for the increased accuracy achieved by imaging genetic data integration in schizophrenia classification. Multi-scale data fusion holds promise for enhancing diagnostic precision, facilitating early detection and personalizing treatment regimens in schizophrenia.

Data leakage inflates prediction performance in connectome-based machine learning models

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

The effects of data leakage on connectome-based machine learning models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Updated Perspectives on the Neurobiology of Substance Use Disorders Using Neuroimaging

Substance use problems impair social functioning, academic achievement, and employability. Psychological, biological, social, and environmental factors can contribute to substance use disorders. In recent years, neuroimaging breakthroughs have helped elucidate the mechanisms of substance misuse and its effects on the brain. Functional magnetic resonance imaging (MRI), positron emission tomography (PET), single-photon emission computed tomography (SPECT), and magnetic resonance spectroscopy (MRS) are all examples. Neuroimaging studies suggest substance misuse affects executive function, reward, memory, and stress systems. Recent neuroimaging research attempts have provided clinicians with improved tools to diagnose patients who misuse substances, comprehend the complicated neuroanatomy and neurobiology involved, and devise individually tailored and monitorable treatment regimens for individuals with substance use disorders. This review describes the most recent developments in drug misuse neuroimaging, including the neurobiology of substance use disorders, neuroimaging, and substance use disorders, established neuroimaging techniques, recent developments with established neuroimaging techniques and substance use disorders, and emerging clinical neuroimaging technology.

Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

BACKGROUND

The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation.

METHODS

Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses.

RESULTS

A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β =  - 2.75, p < .001, R²_adj = 0.40; r =  - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF₁₀ > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance.

CONCLUSIONS

Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.