Brain regional volume estimations with NeuroQuant and FIRST: a study in patients with a clinically isolated syndrome

Thalamic atrophy and dysconnectivity are associated with cognitive impairment in a multi-center, clinical routine, real-word study of people with relapsing-remitting multiple sclerosis

BACKGROUND

Prior research has established a link between thalamic pathology and cognitive impairment (CI) in people with multiple sclerosis (pwMS). However, the translation of these findings to pwMS in everyday clinical settings has been insufficient.

OBJECTIVE

To assess which global and/or thalamic imaging biomarkers can be used to identify pwMS at risk for CI and cognitive worsening (CW) in a real-world setting.

METHODS

This was an international, multi-center (11 centers), longitudinal, retrospective, real-word study of people with relapsing-remitting MS (pwRRMS). Brain MRI exams acquired at baseline and follow-up were collected. Cognitive status was evaluated using the Symbol Digit Modalities Test (SDMT). Thalamic volume (TV) measurement was performed on T2-FLAIR, as well as on T1-WI, when available. Thalamic dysconnectivity, T2-lesion volume (T2-LV), and volumes of gray matter (GM), whole brain (WB) and lateral ventricles (LVV) were also assessed.

RESULTS

332 pwMS were followed for an average of 2.8 years. At baseline, T2-LV, LVV, TV and thalamic dysconnectivity on T2-FLAIR (p < 0.016), and WB, GM and TV volumes on T1-WI (p < 0.039) were significantly worse in 90 (27.1 %) CI vs. 242 (62.9 %) non-CI pwRRMS. Greater SDMT decline over the follow-up was associated with lower baseline TV on T2-FLAIR (standardized β = 0.203, p = 0.002) and greater thalamic dysconnectivity (standardized β = -0.14, p = 0.028) in a linear regression model.

CONCLUSIONS

PwRRMS with thalamic atrophy and worse thalamic dysconnectivity present more frequently with CI and experience greater CW over mid-term follow-up in a real-world setting.

FDA-approved machine learning algorithms in neuroradiology: A systematic review of the current evidence for approval

Over the past decade, machine learning (ML) and artificial intelligence (AI) have become increasingly prevalent in the medical field. In the United States, the Food and Drug Administration (FDA) is responsible for regulating AI algorithms as "medical devices" to ensure patient safety. However, recent work has shown that the FDA approval process may be deficient. In this study, we evaluate the evidence supporting FDA-approved neuroalgorithms, the subset of machine learning algorithms with applications in the central nervous system (CNS), through a systematic review of the primary literature. Articles covering the 53 FDA-approved algorithms with applications in the CNS published in PubMed, EMBASE, Google Scholar and Scopus between database inception and January 25, 2022 were queried. Initial searches identified 1505 studies, of which 92 articles met the criteria for extraction and inclusion. Studies were identified for 26 of the 53 neuroalgorithms, of which 10 algorithms had only a single peer-reviewed publication. Performance metrics were available for 15 algorithms, external validation studies were available for 24 algorithms, and studies exploring the use of algorithms in clinical practice were available for 7 algorithms. Papers studying the clinical utility of these algorithms focused on three domains: workflow efficiency, cost savings, and clinical outcomes. Our analysis suggests that there is a meaningful gap between the FDA approval of machine learning algorithms and their clinical utilization. There appears to be room for process improvement by implementation of the following recommendations: the provision of compelling evidence that algorithms perform as intended, mandating minimum sample sizes, reporting of a predefined set of performance metrics for all algorithms and clinical application of algorithms prior to widespread use. This work will serve as a baseline for future research into the ideal regulatory framework for AI applications worldwide.

Brain Parcellation Repeatability and Reproducibility Using Conventional and Quantitative 3D MR Imaging

BACKGROUND AND PURPOSE

Automatic brain parcellation is typically performed on dedicated MR imaging sequences, which require valuable examination time. In this study, a 3D MR imaging quantification sequence to retrieve R1 and R2 relaxation rates and proton density maps was used to synthesize a T1-weighted image stack for brain volume measurement, thereby combining image data for multiple purposes. The repeatability and reproducibility of using the conventional and synthetic input data were evaluated.

MATERIALS AND METHODS

Twelve subjects with a mean age of 54 years were scanned twice at 1.5T and 3T with 3D-QALAS and a conventionally acquired T1-weighted sequence. Using SyMRI, we converted the R1, R2, and proton density maps into synthetic T1-weighted images. Both the conventional T1-weighted and the synthetic 3D-T1-weighted inversion recovery images were processed for brain parcellation by NeuroQuant. Bland-Altman statistics were used to correlate the volumes of 12 brain structures. The coefficient of variation was used to evaluate the repeatability.

RESULTS

A high correlation with medians of 0.97 for 1.5T and 0.92 for 3T was found. A high repeatability was shown with a median coefficient of variation of 1.2% for both T1-weighted and synthetic 3D-T1-weighted inversion recovery at 1.5T, and 1.5% for T1-weighted imaging and 4.4% for synthetic 3D-T1-weighted inversion recovery at 3T. However, significant biases were observed between the methods and field strengths.

CONCLUSIONS

It is possible to perform MR imaging quantification of R1, R2, and proton density maps to synthesize a 3D-T1-weighted image stack, which can be used for automatic brain parcellation. Synthetic parameter settings should be reinvestigated to reduce the observed bias.

Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence

PURPOSE

MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation.

METHODS

We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment.

RESULTS

We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation.

CONCLUSION

We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.

Thalamic atrophy measured by artificial intelligence in a multicentre clinical routine real-word study is associated with disability progression

BACKGROUND

The thalamus is a key grey matter structure, and sensitive marker of neurodegeneration in multiple sclerosis (MS). Previous reports indicated that thalamic volumetry using artificial intelligence (AI) on clinical-quality T2-fluid-attenuated inversion recovery (FLAIR) images alone is fast and reliable.

OBJECTIVE

To investigate whether thalamic volume (TV) loss, measured longitudinally by AI, is associated with disability progression (DP) in patients with MS, participating in a large multicentre study.

METHODS

The DeepGRAI (Deep Grey Rating via Artificial Intelligence) Registry is a multicentre (30 USA sites), longitudinal, observational, retrospective, real-word study of relapsing-remitting (RR) MS patients. Each centre enrolled between 30 and 35 patients. Brain MRI exams acquired at baseline and follow-up on 1.5T or 3T scanners with no prior standardisation were collected. TV measurement was performed on T2-FLAIR using DeepGRAI, and on two dimensional (D)-weighted and 3D T1-weighted images (WI) by using FMRIB's Integrated Registration and Segmentation Tool software where possible.

RESULTS

1002 RRMS patients were followed for an average of 2.6 years. Longitudinal TV analysis was more readily available on T2-FLAIR (96.1%), compared with 2D-T1-WI (61.8%) or 3D-T1-WI (33.2%). Over the follow-up, DeepGRAI TV loss was significantly higher in patients with DP, compared with those with disability improvement (DI) or disease stability (-1.35% in DP, -0.87% in DI and -0.57% in Stable, p=0.045, Bonferroni-adjusted, age-adjusted and follow-up time-adjusted analysis of covariance). In a regression model including MRI scanner change, age, sex, disease duration and follow-up time, DP was associated with DeepGRAI TV loss (p=0.022).

CONCLUSIONS

Thalamic atrophy measured by AI in a multicentre clinical routine real-word setting is associated with DP over mid-term follow-up.

Updated Review of the Evidence Supporting the Medical and Legal Use of NeuroQuant® and NeuroGage® in Patients With Traumatic Brain Injury

Over 40 years of research have shown that traumatic brain injury affects brain volume. However, technical and practical limitations made it difficult to detect brain volume abnormalities in patients suffering from chronic effects of mild or moderate traumatic brain injury. This situation improved in 2006 with the FDA clearance of NeuroQuant®, a commercially available, computer-automated software program for measuring MRI brain volume in human subjects. More recent strides were made with the introduction of NeuroGage®, commercially available software that is based on NeuroQuant® and extends its utility in several ways. Studies using these and similar methods have found that most patients with chronic mild or moderate traumatic brain injury have brain volume abnormalities, and several of these studies found-surprisingly-more abnormal enlargement than atrophy. More generally, 102 peer-reviewed studies have supported the reliability and validity of NeuroQuant® and NeuroGage®. Furthermore, this updated version of a previous review addresses whether NeuroQuant® and NeuroGage® meet the Daubert standard for admissibility in court. It concludes that NeuroQuant® and NeuroGage® meet the Daubert standard based on their reliability, validity, and objectivity. Due to the improvements in technology over the years, these brain volumetric techniques are practical and readily available for clinical or forensic use, and thus they are important tools for detecting signs of brain injury.

Prognosis of a second clinical event from baseline MRI in patients with a CIS: a multicenter study using a machine learning approach

PURPOSE

To predict the occurrence of a second clinical event in patients with a CIS suggestive of MS, from baseline magnetic resonance imaging (MRI), by means of a pattern recognition approach.

METHODS

Two hundred sixty-six patients with a CIS were recruited from four participating centers. Over a follow-up of 3 years, 130 patients had a second clinical episode and 136 did not. Grey matter and white matter T1-hypointensities masks segmented from 3D T1-weighted images acquired on 3 T scanners were used as features for the classification approach. Differences between CIS that remained CIS and those that developed a second event were assessed at a global level and at a regional level, arranging the regions according to their contribution to the classification model.

RESULTS

All classification metrics were around or even below 50% for both global and regional approaches. Accuracies did not change when T1-hypointensity maps were added to the model; just the specificity was increased up to 80%. Among the 30 regions with the largest contribution, 26 were grey matter and 4 were white matter regions. For grey matter, regions contributing showed either a larger or a smaller volume in the group of patients that remained CIS, compared to those with a second event. The volume of T1-hypointensities was always larger for the group that presented a second event.

CONCLUSIONS

Prediction of a second clinical event in CIS patients from baseline MRI seems to present a highly heterogeneous pattern, leading to very low classification accuracies. Adding the T1-hypointensity maps does not seem to improve the accuracy of the classification model.

Technical and clinical validation of commercial automated volumetric MRI tools for dementia diagnosis-a systematic review

Developments in neuroradiological MRI analysis offer promise in enhancing objectivity and consistency in dementia diagnosis through the use of quantitative volumetric reporting tools (QReports). Translation into clinical settings should follow a structured framework of development, including technical and clinical validation steps. However, published technical and clinical validation of the available commercial/proprietary tools is not always easy to find and pathways for successful integration into the clinical workflow are varied. The quantitative neuroradiology initiative (QNI) framework highlights six necessary steps for the development, validation and integration of quantitative tools in the clinic. In this paper, we reviewed the published evidence regarding regulatory-approved QReports for use in the memory clinic and to what extent this evidence fulfils the steps of the QNI framework. We summarize unbiased technical details of available products in order to increase the transparency of evidence and present the range of reporting tools on the market. Our intention is to assist neuroradiologists in making informed decisions regarding the adoption of these methods in the clinic. For the 17 products identified, 11 companies have published some form of technical validation on their methods, but only 4 have published clinical validation of their QReports in a dementia population. Upon systematically reviewing the published evidence for regulatory-approved QReports in dementia, we concluded that there is a significant evidence gap in the literature regarding clinical validation, workflow integration and in-use evaluation of these tools in dementia MRI diagnosis.

LesionQuant for Assessment of MRI in Multiple Sclerosis-A Promising Supplement to the Visual Scan Inspection

Background and Goals: Multiple sclerosis (MS) is a central nervous system inflammatory disease where magnetic resonance imaging (MRI) is an important tool for diagnosis and disease monitoring. Quantitative measurements of lesion volume, lesion count, distribution of lesions, and brain atrophy have a potentially significant value for evaluating disease progression. We hypothesize that utilizing software designed for evaluating MRI data in MS will provide more accurate and detailed analyses compared to the visual neuro-radiological evaluation.

Methods: A group of 56 MS patients (mean age 35 years, 70% females and 96% relapsing-remitting MS) was examined with brain MRI one and 5 years after diagnosis. The T1 and FLAIR brain MRI sequences for all patients were analyzed using the LesionQuant (LQ) software. These data were compared with data from structured visual evaluations of the MRI scans performed by neuro-radiologists, including assessments of atrophy, and lesion count. The data from LQ were also compared with data from other validated research methods for brain segmentation, including assessments of whole brain volume and lesion volume. Correlations with clinical tests like the timed 25-foot walk test (T25FT) were performed to explore additional value of LQ analyses.

Results: Lesion count assessments by LQ and by the neuro-radiologist were significantly correlated one year (cor = 0.92, p = 2.2 × 10⁻¹⁶) and 5 years (cor = 0.84, p = 2.7 × 10⁻¹⁶) after diagnosis. Analyzes of the intra- and interrater variability also correlated significantly (cor = 0.96, p < 0.001, cor = 0.97, p < 0.001). Significant positive correlation was found between lesion volume measured by LQ and by the software Cascade (cor = 0.7, p < 0.001. LQ detected a reduction in whole brain percentile >10 in 10 patients across the time-points, whereas the neuro-radiologist assessment identified six of these. The neuro-radiologist additionally identified five patients with increased atrophy in the follow-up period, all of them displayed decreasing low whole brain percentiles (median 11, range 8–28) in the LQ analysis. Significant positive correlation was identified between lesion volume measured by LQ and test performance on the T25FT both at 1 and 5 years after diagnosis.

Conclusion: For the number of MS lesions at both time-points, we demonstrated strong correlations between the assessments done by LQ and the neuro-radiologist. Lesion volume evaluated with LQ correlated with T25FT performance. LQ-analyses classified more patients to have brain atrophy than the visual neuro-radiological evaluation. In conclusion, LQ seems like a promising supplement to the evaluation performed by neuro-radiologists, providing an automated tool for evaluating lesions in MS patients and also detecting early signs of atrophy in both a longitudinal and cross-sectional setting.

Hippocampal Sclerosis Detection with NeuroQuant Compared with Neuroradiologists

BACKGROUND AND PURPOSE

NeuroQuant is an FDA-approved software that performs automated MR imaging quantitative volumetric analysis. This study aimed to compare the accuracy of NeuroQuant analysis with visual MR imaging analysis by neuroradiologists with expertise in epilepsy in identifying hippocampal sclerosis.

MATERIALS AND METHODS

We reviewed 144 adult patients who underwent presurgical evaluation for temporal lobe epilepsy. The reference standard for hippocampal sclerosis was defined by having hippocampal sclerosis on pathology (n = 61) or not having hippocampal sclerosis on pathology (n = 83). Sensitivities, specificities, positive predictive values, and negative predictive values were compared between NeuroQuant analysis and visual MR imaging analysis by using a McNemar paired test of proportions and the Bayes theorem.

RESULTS

NeuroQuant analysis had a similar specificity to neuroradiologist visual MR imaging analysis (90.4% versus 91.6%; P = .99) but a lower sensitivity (69.0% versus 93.0%, P < .001). The positive predictive value of NeuroQuant analysis was comparable with visual MR imaging analysis (84.0% versus 89.1%), whereas the negative predictive value was not comparable (79.8% versus 95.0%).

CONCLUSIONS

Visual MR imaging analysis by a neuroradiologist with expertise in epilepsy had a higher sensitivity than did NeuroQuant analysis, likely due to the inability of NeuroQuant to evaluate changes in hippocampal T2 signal or architecture. Given that there was no significant difference in specificity between NeuroQuant analysis and visual MR imaging analysis, NeuroQuant can be a valuable tool when the results are positive, particularly in centers that lack neuroradiologists with expertise in epilepsy, to help identify and refer candidates for temporal lobe epilepsy resection. In contrast, a negative test could justify a case referral for further evaluation to ensure that false-negatives are detected.

Brain Atrophy in Relapsing Optic Neuritis Is Associated With Crion Phenotype

Background and objective: Chronic relapsing inflammatory optic neuritis (CRION) is one of the more common phenotypes related to myelin oligodendrocyte glycoprotein antibodies (MOG-Abs). The absence of specific biomarkers makes distinguishing between CRION and relapsing inflammatory ON (RION) difficult. A recent work has suggested a widespread affectation of the central nervous system in CRION patients. In order to search for a potential CRION marker we have measured brain atrophy in a cohort of patients, stratified by phenotypes: CRION, RION, multiple sclerosis with a history of optic neuritis (MS-ON), and MOG-Abs status. Methods: A cross-sectional study was conducted in 31 patients (seven CRION, 11 RION, and 13 MS-ON). All patients were tested for MOG and aquaporin-4 antibodies (AQ4-Abs). Clinical data were collected. Brain atrophy was calculated by measuring the brain parenchyma fraction (BPF) with Neuroquant® software. Results: Four of seven CRION patients and one of 11 RION patients were positive for MOG-Abs (p = 0.046) and no MS-ON patients tested positive to MOG-Abs. All patients were negative to AQ4-Abs. The BPF was lower in patients with CRION than patients with RION (70.6 vs. 75.3%, p = 0.019) and similar to that in MS-ON patients. Conclusions: Brain atrophy in idiopathic inflammatory relapsing ON is present in patients with the CRION phenotype. Data from this study reflect that the optic nerve is a main target involved in these patients but not the only one. Our results should be further investigated in comprehensive and prospective studies.