Impact of an AI software on the diagnostic performance and reading time for the detection of cerebral aneurysms on time of flight MR-angiography

Advancing Intracranial Aneurysm Detection: A Comprehensive Systematic Review and Meta-analysis of Deep Learning Models Performance, Clinical Integration, and Future Directions

BACKGROUND

Cerebral aneurysms pose a significant risk to patient safety, particularly when ruptured, emphasizing the need for early detection and accurate prediction. Traditional diagnostic methods, reliant on clinician-based evaluations, face challenges in sensitivity and consistency, prompting the exploration of deep learning (DL) systems for improved performance.

METHODS

This systematic review and meta-analysis assessed the performance of DL models in detecting and predicting intracranial aneurysms compared to clinician-based evaluations. Imaging modalities included CT angiography (CTA), digital subtraction angiography (DSA), and time-of-flight MR angiography (TOF-MRA). Data on lesion-wise sensitivity, specificity, and the impact of DL assistance on clinician performance were analyzed. Subgroup analyses evaluated DL sensitivity by aneurysm size and location, and interrater agreement was measured using Fleiss' κ.

RESULTS

DL systems achieved an overall lesion-wise sensitivity of 90 % and specificity of 94 %, outperforming human diagnostics. Clinician specificity improved significantly with DL assistance, increasing from 83 % to 85 % in the patient-wise scenario and from 93 % to 95 % in the lesion-wise scenario. Similarly, clinician sensitivity also showed notable improvement with DL assistance, rising from 82 % to 96 % in the patient-wise scenario and from 82 % to 88 % in the lesion-wise scenario. Subgroup analysis showed DL sensitivity varied with aneurysm size and location, reaching 100 % for aneurysms larger than 10 mm. Additionally, DL assistance improved interrater agreement among clinicians, with Fleiss' κ increasing from 0.668 to 0.862.

CONCLUSIONS

DL models demonstrate transformative potential in managing cerebral aneurysms by enhancing diagnostic accuracy, reducing missed cases, and supporting clinical decision-making. However, further validation in diverse clinical settings and seamless integration into standard workflows are necessary to fully realize the benefits of DL-driven diagnostics.

Multi-centric AI Model for Unruptured Intracranial Aneurysm Detection and Volumetric Segmentation in 3D TOF-MRI

The aim of this study was to develop an open-source nnU-Net-based AI model for combined detection and segmentation of unruptured intracranial aneurysms (UICA) in 3D TOF-MRI and compare models trained on datasets with aneurysm-like differential diagnoses. This retrospective study (2020–2023) included 385 anonymized 3D TOF-MRI images from 345 patients (mean age 59 years, 60% female) at multiple centers plus 113 subjects from the ADAM challenge. Images featured untreated or possible UICA and differential diagnoses. Four distinct training datasets were created, and the nnU-Net framework was used for model development. Performance was assessed on a separate test set using sensitivity and false positive (FP)/case rate for detection and DICE score and NSD (normalized surface distance, 0.5 mm threshold) for segmentation. Segmentation performance on the test set was also compared to a second human reader. The four models achieved overall sensitivity between 82 and 85% and an FP/case rate of 0.20 to 0.31, with no significant differences (p = 0.90 and p = 0.16) between them. The primary model showed 85% sensitivity and 0.23 FP/case rate, outperforming the ADAM-challenge winner (61%) and a nnU-Net trained on ADAM data (51%) in sensitivity (p < 0.05). Mean DICE (0.73) and NSD (0.84 for 0.5 mm threshold) for correctly detected UICA did not significantly differ from human reader performance. Our open-source, nnU-Net-based AI model (available at https://zenodo.org/records/13386859) demonstrates high sensitivity, low FP rates, and consistent segmentation accuracy for UICA detection and segmentation in 3D TOF-MRI, suggesting its potential to improve clinical diagnosis and monitoring of UICA.

Multi-centric AI Model for Unruptured Intracranial Aneurysm Detection and Volumetric Segmentation in 3D TOF-MRI

The aim of this study was to develop an open-source nnU-Net-based AI model for combined detection and segmentation of unruptured intracranial aneurysms (UICA) in 3D TOF-MRI and compare models trained on datasets with aneurysm-like differential diagnoses. This retrospective study (2020-2023) included 385 anonymized 3D TOF-MRI images from 345 patients (mean age 59 years, 60% female) at multiple centers plus 113 subjects from the ADAM challenge. Images featured untreated or possible UICA and differential diagnoses. Four distinct training datasets were created, and the nnU-Net framework was used for model development. Performance was assessed on a separate test set using sensitivity and false positive (FP)/case rate for detection and DICE score and NSD (normalized surface distance, 0.5 mm threshold) for segmentation. Segmentation performance on the test set was also compared to a second human reader. The four models achieved overall sensitivity between 82 and 85% and an FP/case rate of 0.20 to 0.31, with no significant differences (p = 0.90 and p = 0.16) between them. The primary model showed 85% sensitivity and 0.23 FP/case rate, outperforming the ADAM-challenge winner (61%) and a nnU-Net trained on ADAM data (51%) in sensitivity (p < 0.05). Mean DICE (0.73) and NSD (0.84 for 0.5 mm threshold) for correctly detected UICA did not significantly differ from human reader performance. Our open-source, nnU-Net-based AI model (available at https://zenodo.org/records/13386859 ) demonstrates high sensitivity, low FP rates, and consistent segmentation accuracy for UICA detection and segmentation in 3D TOF-MRI, suggesting its potential to improve clinical diagnosis and monitoring of UICA.

Diagnostic Accuracy of a Deep Learning Algorithm for Detecting Unruptured Intracranial Aneurysms in Magnetic Resonance Angiography: A Multicenter Pivotal Trial

BACKGROUND

Intracranial aneurysm rupture is associated with high mortality and disability rates. Early detection is crucial, but increasing diagnostic workloads place significant strain on radiologists. We evaluated the efficacy of a deep learning algorithm in detecting unruptured intracranial aneurysms (UIAs) using time-of-flight (TOF) magnetic resonance angiography (MRA).

METHODS

Data from 675 participants (189 aneurysm-positive [221 UIAs] and 486 aneurysm-negative) were collected from 2 hospitals (2019-2023). Positive cases were confirmed by digital subtraction angiography, and images were annotated by vascular experts. The 3D U-Net-based model was trained on 988 nonoverlapped TOF MRA datasets and evaluated by patient- and lesion-level sensitivity, specificity, and false-positive rates.

RESULTS

The mean age was 59.6 years (standard deviation 11.3), and 52.0% were female. The model achieved patient-level sensitivity of 95.2% and specificity of 80.5%, with lesion-level sensitivity of 89.6% and a false-positive rate of 0.19 per patient. Sensitivity by aneurysm size was 72.3% for lesions <3 mm, 91.8% for 3-5 mm, and 94.3% for >5 mm. Performance was consistent across institutions, with an area under the receiver operating characteristic curve of 0.949.

CONCLUSIONS

The software demonstrated high sensitivity and low false-positive rates for UIA detection in TOF MRA, suggesting its utility in reducing diagnostic errors and alleviating radiologist workload. Expert review remains essential, particularly for small or complex aneurysms.

Automation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography

PURPOSE

To determine how automation bias (inclination of humans to overly trust-automated decision-making systems) can affect radiologists when interpreting AI-detected cerebral aneurysm findings in time-of-flight magnetic resonance angiography (TOF-MRA) studies.

MATERIAL AND METHODS

Nine radiologists with varying levels of experience evaluated twenty TOF-MRA examinations for the presence of cerebral aneurysms. Every case was evaluated with and without assistance by the AI software © mdbrain, with a washout period of at least four weeks in-between. Half of the cases included at least one false-positive AI finding. Aneurysm ratings, follow-up recommendations, and reading times were assessed using the Wilcoxon signed-rank test.

RESULTS

False-positive AI results led to significantly higher suspicion of aneurysm findings (p = 0.01). Inexperienced readers further recommended significantly more intense follow-up examinations when presented with false-positive AI findings (p = 0.005). Reading times were significantly shorter with AI assistance in inexperienced (164.1 vs 228.2 s; p < 0.001), moderately experienced (126.2 vs 156.5 s; p < 0.009), and very experienced (117.9 vs 153.5 s; p < 0.001) readers alike.

CONCLUSION

Our results demonstrate the susceptibility of radiology readers to automation bias in detecting cerebral aneurysms in TOF-MRA studies when encountering false-positive AI findings. While AI systems for cerebral aneurysm detection can provide benefits, challenges in human-AI interaction need to be mitigated to ensure safe and effective adoption.

Spectra of intracranial diseases in Chinese military pilots (cadets) unqualified for transfer to pilot modified high performance aircraft

BACKGROUND

With very high mortality and disability rates, cerebrovascular diseases and intracranial tumors severely threaten the health and fighting strength of flying personnel, requiring great concern and intensive screening in clinic, early warning in an early and accurate manner and early intervention of diseases possibly resulting in inflight incapacitation are key emphases of aeromedical support in clinic.

AIM

To probe into the spectra of intracranial diseases, flight factors and medical imaging characteristics of military pilots (cadets) in the physical examination for transfer to pilot modified high performance aircraft, thus rendering theoretical references for clinical aeromedical support of pilots.

METHODS

A total of 554 military pilots (cadets) undergoing physical examination for transfer to pilot modified high performance aircraft from December 2020 to April 2024 in a military medical center were enrolled in this study. Then, a retrospective study was carried out on intracranial disease spectra and head magnetic resonance imaging (MRI) data of 36 pilots (cadets) who were unqualified for transfer to pilot modified high performance aircraft. Besides, a descriptive statistical analysis was conducted on the clinical data, age, fighter type and head MRI data of such pilots (cadets).

RESULTS

Abnormal head images were found in 36 out of 554 pilots (cadets) participating in the physical examination for transfer to pilot modified high performance aircraft, including arachnoid cyst in 17 (3.1%) military pilots (cadets), suspected very small aneurysm in 11 (2.0%), cavernous hemangioma in 4 (0.7%), vascular malformation in 2 (0.4%), and pituitary tumor in 3 (0.5%, one of which developed cavernous hemangioma simultaneously). Among the 17 pilots (cadets) with arachnoid cyst, 4 were identified as unqualified for transfer to pilot modified high performance aircraft because the marginal brain tissues were compressed by the cyst > 6 cm in length and diameter. The 11 pilots (cadets) with suspected very small aneurysms identified by 3.0T MRI consisted of 6 diagnosed with conus arteriosus by digital subtraction angiography and qualified for transfer to pilot modified high performance aircraft, and 5 identified as very small intracranial aneurysms with diameter < 3 mm and unqualified for transfer to pilot modified high performance aircraft. No symptoms and signs were observed in the 4 military pilots (cadets) with cavernous hemangioma, and the results of MRI revealed bleeding. The 1 of the 4 had the lesion located in pons and developed Rathke cyst in pituitary gland at the same time, and unqualified for transfer to pilot modified high performance aircraft. The 2 of the 4 were unqualified for flying, and 2 transferred to air combat service division. The 2 pilots (cadets) with vascular malformation were identified as unqualified for transfer to pilot modified high performance aircraft. Among the 3 pilots (cadets) with pituitary tumor, one pilot cadet was identified as unqualified for flying since the tumor compressed the optic chiasma, one had cavernous hemangioma in pons in the meantime and transferred to air combat service division, and one was diagnosed with nonfunctional microadenoma and qualified for transfer to pilot modified high performance aircraft.

CONCLUSION

High-resolution head MRI examination is of great significance for screening and detecting cerebrovascular diseases and intracranial tumors in military flying personnel, and attention should be paid to its clinical application to physical examination for transfer to pilot modified high performance aircraft.