Response to Dr Fried & Dr Kievit, and Dr Malhi et al

A Systematic Evaluation of Machine Learning-Based Biomarkers for Major Depressive Disorder

Importance

Biological psychiatry aims to understand mental disorders in terms of altered neurobiological pathways. However, for one of the most prevalent and disabling mental disorders, major depressive disorder (MDD), no informative biomarkers have been identified.

Objective

To evaluate whether machine learning (ML) can identify a multivariate biomarker for MDD.

Design, Setting, and Participants

This study used data from the Marburg-Münster Affective Disorders Cohort Study, a case-control clinical neuroimaging study. Patients with acute or lifetime MDD and healthy controls aged 18 to 65 years were recruited from primary care and the general population in Münster and Marburg, Germany, from September 11, 2014, to September 26, 2018. The Münster Neuroimaging Cohort (MNC) was used as an independent partial replication sample. Data were analyzed from April 2022 to June 2023.

Exposure

Patients with MDD and healthy controls.

Main Outcome and Measure

Diagnostic classification accuracy was quantified on an individual level using an extensive ML-based multivariate approach across a comprehensive range of neuroimaging modalities, including structural and functional magnetic resonance imaging and diffusion tensor imaging as well as a polygenic risk score for depression.

Results

Of 1801 included participants, 1162 (64.5%) were female, and the mean (SD) age was 36.1 (13.1) years. There were a total of 856 patients with MDD (47.5%) and 945 healthy controls (52.5%). The MNC replication sample included 1198 individuals (362 with MDD [30.1%] and 836 healthy controls [69.9%]). Training and testing a total of 4 million ML models, mean (SD) accuracies for diagnostic classification ranged between 48.1% (3.6%) and 62.0% (4.8%). Integrating neuroimaging modalities and stratifying individuals based on age, sex, treatment, or remission status does not enhance model performance. Findings were replicated within study sites and also observed in structural magnetic resonance imaging within MNC. Under simulated conditions of perfect reliability, performance did not significantly improve. Analyzing model errors suggests that symptom severity could be a potential focus for identifying MDD subgroups.

Conclusion and Relevance

Despite the improved predictive capability of multivariate compared with univariate neuroimaging markers, no informative individual-level MDD biomarker-even under extensive ML optimization in a large sample of diagnosed patients-could be identified.

Lifetime Exposure to Depression and Neuroimaging Measures of Brain Structure and Function

Importance

Despite decades of neuroimaging studies reporting brain structural and functional alterations in depression, discrepancies in findings across studies and limited convergence across meta-analyses have raised questions about the consistency and robustness of the observed brain phenotypes.

Objective

To investigate the associations between 6 operational criteria of lifetime exposure to depression and functional and structural neuroimaging measures.

Design, Setting, and Participants

This cross-sectional study analyzed data from a UK Biobank cohort of individuals aged 45 to 80 years who were enrolled between January 1, 2014, and December 31, 2018. Participants included individuals with a lifetime exposure to depression and matched healthy controls without indications of psychosis, mental illness, behavior disorder, and disease of the nervous system. Six operational criteria of lifetime exposure to depression were evaluated: help seeking for depression; self-reported depression; antidepressant use; depression definition by Smith et al; hospital International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) diagnosis codes F32 and F33; and Composite International Diagnostic Interview Short Form score. Six increasingly restrictive depression definitions and groups were defined based on the 6 depression criteria, ranging from meeting only 1 criterion to meeting all 6 criteria. Data were analyzed between January and October 2022.

Main Outcomes and Measures

Functional measures were calculated using voxel-wise fractional amplitude of low-frequency fluctuation (fALFF), global correlation (GCOR), and local correlation (LCOR). Structural measures were calculated using gray matter volume (GMV).

Results

The study included 20 484 individuals with lifetime depression (12 645 females [61.7%]; mean [SD] age, 63.91 [7.60] years) and 25 462 healthy controls (14 078 males [55.3%]; mean [SD] age, 65.05 [7.8] years). Across all depression criteria, individuals with lifetime depression displayed regionally consistent decreases in fALFF, LCOR, and GCOR (Cohen d range, -0.53 [95% CI, -0.88 to -0.15] to -0.04 [95% CI, -0.07 to -0.01]) but not in GMV (Cohen d range, -0.47 [95 % CI, -0.75 to -0.12] to 0.26 [95% CI, 0.15-0.37]). Hospital ICD-10 diagnosis codes F32 and F33 (median [IQR] difference in effect sizes, -0.14 [-0.17 to -0.11]) and antidepressant use (median [IQR] difference in effect sizes, -0.12 [-0.16 to -0.10]) were criteria associated with the most pronounced alterations.

Conclusions and Relevance

Results of this cross-sectional study indicate that lifetime exposure to depression was associated with robust functional changes, with a more restrictive depression definition revealing more pronounced alterations. Different inclusion criteria for depression may be associated with the substantial variation in imaging findings reported in the literature.

Quantifying Deviations of Brain Structure and Function in Major Depressive Disorder Across Neuroimaging Modalities

IMPORTANCE

Identifying neurobiological differences between patients with major depressive disorder (MDD) and healthy individuals has been a mainstay of clinical neuroscience for decades. However, recent meta-analyses have raised concerns regarding the replicability and clinical relevance of brain alterations in depression.

OBJECTIVE

To quantify the upper bounds of univariate effect sizes, estimated predictive utility, and distributional dissimilarity of healthy individuals and those with depression across structural magnetic resonance imaging (MRI), diffusion-tensor imaging, and functional task-based as well as resting-state MRI, and to compare results with an MDD polygenic risk score (PRS) and environmental variables.

DESIGN, SETTING, AND PARTICIPANTS

This was a cross-sectional, case-control clinical neuroimaging study. Data were part of the Marburg-Münster Affective Disorders Cohort Study. Patients with depression and healthy controls were recruited from primary care and the general population in Münster and Marburg, Germany. Study recruitment was performed from September 11, 2014, to September 26, 2018. The sample comprised patients with acute and chronic MDD as well as healthy controls in the age range of 18 to 65 years. Data were analyzed from October 29, 2020, to April 7, 2022.

MAIN OUTCOMES AND MEASURES

Primary analyses included univariate partial effect size (η2), classification accuracy, and distributional overlapping coefficient for healthy individuals and those with depression across neuroimaging modalities, controlling for age, sex, and additional modality-specific confounding variables. Secondary analyses included patient subgroups for acute or chronic depressive status.

RESULTS

A total of 1809 individuals (861 patients [47.6%] and 948 controls [52.4%]) were included in the analysis (mean [SD] age, 35.6 [13.2] years; 1165 female patients [64.4%]). The upper bound of the effect sizes of the single univariate measures displaying the largest group difference ranged from partial η2 of 0.004 to 0.017, and distributions overlapped between 87% and 95%, with classification accuracies ranging between 54% and 56% across neuroimaging modalities. This pattern remained virtually unchanged when considering either only patients with acute or chronic depression. Differences were comparable with those found for PRS but substantially smaller than for environmental variables.

CONCLUSIONS AND RELEVANCE

Results of this case-control study suggest that even for maximum univariate biological differences, deviations between patients with MDD and healthy controls were remarkably small, single-participant prediction was not possible, and similarity between study groups dominated. Biological psychiatry should facilitate meaningful outcome measures or predictive approaches to increase the potential for a personalization of the clinical practice.