Automated Detection of Cerebral Aneurysms on TOF-MRA Using a Deep Learning Approach: An External Validation Study

Multi-centric AI Model for Unruptured Intracranial Aneurysm Detection and Volumetric Segmentation in 3D TOF-MRI

The aim of this study was to develop an open-source nnU-Net-based AI model for combined detection and segmentation of unruptured intracranial aneurysms (UICA) in 3D TOF-MRI and compare models trained on datasets with aneurysm-like differential diagnoses. This retrospective study (2020–2023) included 385 anonymized 3D TOF-MRI images from 345 patients (mean age 59 years, 60% female) at multiple centers plus 113 subjects from the ADAM challenge. Images featured untreated or possible UICA and differential diagnoses. Four distinct training datasets were created, and the nnU-Net framework was used for model development. Performance was assessed on a separate test set using sensitivity and false positive (FP)/case rate for detection and DICE score and NSD (normalized surface distance, 0.5 mm threshold) for segmentation. Segmentation performance on the test set was also compared to a second human reader. The four models achieved overall sensitivity between 82 and 85% and an FP/case rate of 0.20 to 0.31, with no significant differences (p = 0.90 and p = 0.16) between them. The primary model showed 85% sensitivity and 0.23 FP/case rate, outperforming the ADAM-challenge winner (61%) and a nnU-Net trained on ADAM data (51%) in sensitivity (p < 0.05). Mean DICE (0.73) and NSD (0.84 for 0.5 mm threshold) for correctly detected UICA did not significantly differ from human reader performance. Our open-source, nnU-Net-based AI model (available at https://zenodo.org/records/13386859) demonstrates high sensitivity, low FP rates, and consistent segmentation accuracy for UICA detection and segmentation in 3D TOF-MRI, suggesting its potential to improve clinical diagnosis and monitoring of UICA.

Multi-centric AI Model for Unruptured Intracranial Aneurysm Detection and Volumetric Segmentation in 3D TOF-MRI

The aim of this study was to develop an open-source nnU-Net-based AI model for combined detection and segmentation of unruptured intracranial aneurysms (UICA) in 3D TOF-MRI and compare models trained on datasets with aneurysm-like differential diagnoses. This retrospective study (2020-2023) included 385 anonymized 3D TOF-MRI images from 345 patients (mean age 59 years, 60% female) at multiple centers plus 113 subjects from the ADAM challenge. Images featured untreated or possible UICA and differential diagnoses. Four distinct training datasets were created, and the nnU-Net framework was used for model development. Performance was assessed on a separate test set using sensitivity and false positive (FP)/case rate for detection and DICE score and NSD (normalized surface distance, 0.5 mm threshold) for segmentation. Segmentation performance on the test set was also compared to a second human reader. The four models achieved overall sensitivity between 82 and 85% and an FP/case rate of 0.20 to 0.31, with no significant differences (p = 0.90 and p = 0.16) between them. The primary model showed 85% sensitivity and 0.23 FP/case rate, outperforming the ADAM-challenge winner (61%) and a nnU-Net trained on ADAM data (51%) in sensitivity (p < 0.05). Mean DICE (0.73) and NSD (0.84 for 0.5 mm threshold) for correctly detected UICA did not significantly differ from human reader performance. Our open-source, nnU-Net-based AI model (available at https://zenodo.org/records/13386859 ) demonstrates high sensitivity, low FP rates, and consistent segmentation accuracy for UICA detection and segmentation in 3D TOF-MRI, suggesting its potential to improve clinical diagnosis and monitoring of UICA.

Diagnostic Accuracy of a Deep Learning Algorithm for Detecting Unruptured Intracranial Aneurysms in Magnetic Resonance Angiography: A Multicenter Pivotal Trial

BACKGROUND

Intracranial aneurysm rupture is associated with high mortality and disability rates. Early detection is crucial, but increasing diagnostic workloads place significant strain on radiologists. We evaluated the efficacy of a deep learning algorithm in detecting unruptured intracranial aneurysms (UIAs) using time-of-flight (TOF) magnetic resonance angiography (MRA).

METHODS

Data from 675 participants (189 aneurysm-positive [221 UIAs] and 486 aneurysm-negative) were collected from 2 hospitals (2019-2023). Positive cases were confirmed by digital subtraction angiography, and images were annotated by vascular experts. The 3D U-Net-based model was trained on 988 nonoverlapped TOF MRA datasets and evaluated by patient- and lesion-level sensitivity, specificity, and false-positive rates.

RESULTS

The mean age was 59.6 years (standard deviation 11.3), and 52.0% were female. The model achieved patient-level sensitivity of 95.2% and specificity of 80.5%, with lesion-level sensitivity of 89.6% and a false-positive rate of 0.19 per patient. Sensitivity by aneurysm size was 72.3% for lesions <3 mm, 91.8% for 3-5 mm, and 94.3% for >5 mm. Performance was consistent across institutions, with an area under the receiver operating characteristic curve of 0.949.

CONCLUSIONS

The software demonstrated high sensitivity and low false-positive rates for UIA detection in TOF MRA, suggesting its utility in reducing diagnostic errors and alleviating radiologist workload. Expert review remains essential, particularly for small or complex aneurysms.

Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography

PURPOSE

To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.

MATERIAL AND METHODS

Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.

RESULTS

False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.

CONCLUSION

Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.

Evaluation of AI-Powered Routine Screening of Clinically Acquired cMRIs for Incidental Intracranial Aneurysms

Objectives: To quantify the clinical value of integrating a commercially available artificial intelligence (AI) algorithm for intracranial aneurysm detection in a screening setting that utilizes cranial magnetic resonance imaging (cMRI) scans acquired primarily for other clinical purposes. Methods: A total of 907 consecutive cMRI datasets, including time-of-flight-angiography (TOF-MRA), were retrospectively identified from patients unaware of intracranial aneurysms. cMRIs were analyzed by a commercial AI algorithm and reassessed by consultant-level neuroradiologists, who provided confidence scores and workup recommendations for suspicious findings. Patients with newly identified findings (relative to initial cMRI reports) were contacted for on-site consultations, including cMRI follow-up or catheter angiography. The number needed to screen (NNS) was defined as the cMRI quantity that must undergo AI screening to achieve various clinical endpoints. Results: The algorithm demonstrates high sensitivities (100% for findings >4 mm in diameter), a 17.8% MRA alert rate and positive predictive values of 11.5-43.8% (depending on whether inconclusive findings are considered or not). Initial cMRI reports missed 50 out of 59 suspicious findings, including 13 certain intradural aneurysms. The NNS for additionally identifying highly suspicious and therapeutically relevant (unruptured intracranial aneurysm treatment scores balanced or in favor of treatment) findings was 152. The NNS for recommending additional follow-/workup imaging (cMRI or catheter angiography) was 26, suggesting an additional up to 4% increase in imaging procedures resulting from a preceding AI screening. Conclusions: AI-powered routine screening of cMRIs clearly lowers the high risk of incidental aneurysm non-reporting but results in a substantial burden of additional imaging follow-up for minor or inconclusive findings.

Patch-Wise Deep Learning Method for Intracranial Stenosis and Aneurysm Detection-the Tromsø Study

Intracranial atherosclerotic stenosis (ICAS) and intracranial aneurysms are prevalent conditions in the cerebrovascular system. ICAS causes a narrowing of the arterial lumen, thereby restricting blood flow, while aneurysms involve the ballooning of blood vessels. Both conditions can lead to severe outcomes, such as stroke or vessel rupture, which can be fatal. Early detection is crucial for effective intervention. In this study, we introduced a method that combines classical computer vision techniques with deep learning to detect intracranial aneurysms and ICAS in time-of-flight magnetic resonance angiography images. The process began with skull-stripping, followed by an affine transformation to align the images to a common atlas space. We then focused on the region of interest, including the circle of Willis, by cropping the relevant area. A segmentation algorithm was used to isolate the arteries, after which a patch-wise residual neural network was applied across the image. A voting mechanism was then employed to identify the presence of atrophies. Our method achieved accuracies of 76.5% for aneurysms and 82.4% for ICAS. Notably, when occlusions were not considered, the accuracy for ICAS detection improved to 85.7%. While the algorithm performed well for localized pathological findings, it was less effective at detecting occlusions, which involved long-range dependencies in the MRIs. This limitation was due to the architectural design of the patch-wise deep learning approach. Regardless, this can, in the future, be mitigated in a multi-scale patch-wise algorithm.

Artificial intelligence can help detecting incidental intracranial aneurysm on routine brain MRI using TOF MRA data sets and improve the time required for analysis of these images

PURPOSE

The aim of our study was to assess the diagnostic performance of commercially available AI software for intracranial aneurysm detection and to determine if the AI system enhances the radiologist's accuracy in identifying aneurysms and reduces image analysis time.

METHODS

TOF-MRA clinical brain examinations were analyzed using commercially available software and by an consultant neuroradiologist for the presence of intracranial aneurysms. The results were compared with the reference standard, to measure the sensitivity and specificity of the software and the consultant neuroradiologist. Furthermore, we examined the time required for the neuroradiologist to analyze the TOF-MRA image set, both with and without use of the AI software.

RESULTS

In 500 TOF-MRI brain studies, 106 aneurysms were detected in 85 examinations by combining AI software with neuroradiologist readings. The neuroradiologist identified 98 aneurysms (92.5% sensitivity), while AI detected 77 aneurysms (72.6% sensitivity). Specificity and sensitivity were calculated from the combined effort as reference. Combining AI and neuroradiologist readings significantly improves detection reliability. Additionally, AI integration reduced TOF-MRA analysis time by 19 s (23% reduction).

CONCLUSIONS

Our findings indicate that the AI-based software can support neuroradiologists in interpreting brain TOF-MRA. A combined reading of the AI-based software and the neuroradiologist demonstrated higher reliability in identifying intracranial aneurysms as compared to reading by either neuroradiologist or software, thus improving diagnostic accuracy of the neuroradiologist. Simultaneously, reading time for the neuroradiologist was reduced by approximately one quarter.

Deep learning-based platform performs high detection sensitivity of intracranial aneurysms in 3D brain TOF-MRA: An external clinical validation study

PURPOSE

To evaluate the diagnostic efficacy of a developed artificial intelligence (AI) platform incorporating deep learning algorithms for the automated detection of intracranial aneurysms in time-of-flight (TOF) magnetic resonance angiography (MRA).

METHOD

This retrospective study encompassed 3D TOF MRA images acquired between January 2023 and June 2023, aiming to validate the presence of intracranial aneurysms via our developed AI platform. The manual segmentation results by experienced neuroradiologists served as the "gold standard". Following annotation of MRA images by neuroradiologists using InferScholar software, the AI platform conducted automatic segmentation of intracranial aneurysms. Various metrics including accuracy (ACC), balanced ACC, area under the curve (AUC), sensitivity (SE), specificity (SP), F1 score, Brier Score, and Net Benefit were utilized to evaluate the generalization of AI platform. Comparison of intracranial aneurysm identification performance was conducted between the AI platform and six radiologists with experience ranging from 3 to 12 years in interpreting MR images. Additionally, a comparative analysis was carried out between radiologists' detection performance based on independent visual diagnosis and AI-assisted diagnosis. Subgroup analyses were also performed based on the size and location of the aneurysms to explore factors impacting aneurysm detectability.

RESULTS

510 patients were enrolled including 215 patients (42.16 %) with intracranial aneurysms and 295 patients (57.84 %) without aneurysms. Compared with six radiologists, the AI platform showed competitive discrimination power (AUC, 0.96), acceptable calibration (Brier Score loss, 0.08), and clinical utility (Net Benefit, 86.96 %). The AI platform demonstrated superior performance in detecting aneurysms with an overall SE, SP, ACC, balanced ACC, and F1 score of 91.63 %, 92.20 %, 91.96 %, 91.92 %, and 90.57 % respectively, outperforming the detectability of the two resident radiologists. For subgroup analysis based on aneurysm size and location, we observed that the SE of the AI platform for identifying tiny (diameter＜3mm), small (3 mm ≤ diameter＜5mm), medium (5 mm ≤ diameter＜7mm) and large aneurysms (diameter ≥ 7 mm) was 87.80 %, 93.14 %, 95.45 %, and 100 %, respectively. Furthermore, the SE for detecting aneurysms in the anterior circulation was higher than that in the posterior circulation. Utilizing the AI assistance, six radiologists (i.e., two residents, two attendings and two professors) achieved statistically significant improvements in mean SE (residents: 71.40 % vs. 88.37 %; attendings: 82.79 % vs. 93.26 %; professors: 90.07 % vs. 97.44 %; P < 0.05) and ACC (residents: 85.29 % vs. 94.12 %; attendings: 91.76 % vs. 97.06 %; professors: 95.29 % vs. 98.82 %; P < 0.05) while no statistically significant change was observed in SP. Overall, radiologists' mean SE increased by 11.40 %, mean SP increased by 1.86 %, and mean ACC increased by 5.88 %, mean balanced ACC promoted by 6.63 %, mean F1 score grew by 7.89 %, and Net Benefit rose by 12.52 %, with a concurrent decrease in mean Brier score declined by 0.06.

CONCLUSIONS

The deep learning algorithms implemented in the AI platform effectively detected intracranial aneurysms on TOF-MRA and notably enhanced the diagnostic capabilities of radiologists. This indicates that the AI-based auxiliary diagnosis model can provide dependable and precise prediction to improve the diagnostic capacity of radiologists.

Impact of an AI software on the diagnostic performance and reading time for the detection of cerebral aneurysms on time of flight MR-angiography

PURPOSE

To evaluate the impact of an AI-based software trained to detect cerebral aneurysms on TOF-MRA on the diagnostic performance and reading times across readers with varying experience levels.

METHODS

One hundred eighty-six MRI studies were reviewed by six readers to detect cerebral aneurysms. Initially, readings were assisted by the CNN-based software mdbrain. After 6 weeks, a second reading was conducted without software assistance. The results were compared to the consensus reading of two neuroradiological specialists and sensitivity (lesion and patient level), specificity (patient level), and false positives per case were calculated for the group of all readers, for the subgroup of physicians, and for each individual reader. Also, reading times for each reader were measured.

RESULTS

The dataset contained 54 aneurysms. The readers had no experience (three medical students), 2 years experience (resident in neuroradiology), 6 years experience (radiologist), and 12 years (neuroradiologist). Significant improvements of overall specificity and the overall number of false positives per case were observed in the reading with AI support. For the physicians, we found significant improvements of sensitivity on lesion and patient level and false positives per case. Four readers experienced reduced reading times with the software, while two encountered increased times.

CONCLUSION

In the reading with the AI-based software, we observed significant improvements in terms of specificity and false positives per case for the group of all readers and significant improvements of sensitivity and false positives per case for the physicians. Further studies are needed to investigate the effects of the AI-based software in a prospective setting.

External evaluation of a deep learning-based approach for automated brain volumetry in patients with huntington's disease

A crucial step in the clinical adaptation of an AI-based tool is an external, independent validation. The aim of this study was to investigate brain atrophy in patients with confirmed, progressed Huntington's disease using a certified software for automated volumetry and to compare the results with the manual measurement methods used in clinical practice as well as volume calculations of the caudate nuclei based on manual segmentations. Twenty-two patients were included retrospectively, consisting of eleven patients with Huntington's disease and caudate nucleus atrophy and an age- and sex-matched control group. To quantify caudate head atrophy, the frontal horn width to intercaudate distance ratio and the intercaudate distance to inner table width ratio were obtained. The software mdbrain was used for automated volumetry. Manually measured ratios and automatically measured volumes of the groups were compared using two-sample t-tests. Pearson correlation analyses were performed. The relative difference between automatically and manually determined volumes of the caudate nuclei was calculated. Both ratios were significantly different between the groups. The automatically and manually determined volumes of the caudate nuclei showed a high level of agreement with a mean relative discrepancy of - 2.3 ± 5.5%. The Huntington's disease group showed significantly lower volumes in a variety of supratentorial brain structures. The highest degree of atrophy was shown for the caudate nucleus, putamen, and pallidum (all p < .0001). The caudate nucleus volume and the ratios were found to be strongly correlated in both groups. In conclusion, in patients with progressed Huntington's disease, it was shown that the automatically determined caudate nucleus volume correlates strongly with measured ratios commonly used in clinical practice. Both methods allowed clear differentiation between groups in this collective. The software additionally allows radiologists to more objectively assess the involvement of a variety of brain structures that are less accessible to standard semiquantitative methods.

Machine learning based outcome prediction of microsurgically treated unruptured intracranial aneurysms

Machine learning (ML) has revolutionized data processing in recent years. This study presents the results of the first prediction models based on a long-term monocentric data registry of patients with microsurgically treated unruptured intracranial aneurysms (UIAs) using a temporal train-test split. Temporal train-test splits allow to simulate prospective validation, and therefore provide more accurate estimations of a model's predictive quality when applied to future patients. ML models for the prediction of the Glasgow outcome scale, modified Rankin Scale (mRS), and new transient or permanent neurological deficits (output variables) were created from all UIA patients that underwent microsurgery at the Kepler University Hospital Linz (Austria) between 2002 and 2020 (n = 466), based on 18 patient- and 10 aneurysm-specific preoperative parameters (input variables). Train-test splitting was performed with a temporal split for outcome prediction in microsurgical therapy of UIA. Moreover, an external validation was conducted on an independent external data set (n = 256) of the Department of Neurosurgery, University Medical Centre Hamburg-Eppendorf. In total, 722 aneurysms were included in this study. A postoperative mRS > 2 was best predicted by a quadratic discriminant analysis (QDA) estimator in the internal test set, with an area under the receiver operating characteristic curve (ROC-AUC) of 0.87 ± 0.03 and a sensitivity and specificity of 0.83 ± 0.08 and 0.71 ± 0.07, respectively. A Multilayer Perceptron predicted the post- to preoperative mRS difference > 1 with a ROC-AUC of 0.70 ± 0.02 and a sensitivity and specificity of 0.74 ± 0.07 and 0.50 ± 0.04, respectively. The QDA was the best model for predicting a permanent new neurological deficit with a ROC-AUC of 0.71 ± 0.04 and a sensitivity and specificity of 0.65 ± 0.24 and 0.60 ± 0.12, respectively. Furthermore, these models performed significantly better than the classic logistic regression models (p < 0.0001). The present results showed good performance in predicting functional and clinical outcomes after microsurgical therapy of UIAs in the internal data set, especially for the main outcome parameters, mRS and permanent neurological deficit. The external validation showed poor discrimination with ROC-AUC values of 0.61, 0.53 and 0.58 respectively for predicting a postoperative mRS > 2, a pre- and postoperative difference in mRS > 1 point and a GOS < 5. Therefore, generalizability of the models could not be demonstrated in the external validation. A SHapley Additive exPlanations (SHAP) analysis revealed that this is due to the most important features being distributed quite differently in the internal and external data sets. The implementation of newly available data and the merging of larger databases to form more broad-based predictive models is imperative in the future.