Artificial Intelligence Assistance Improves the Accuracy and Efficiency of Intracranial Aneurysm Detection with CT Angiography

Advancing Intracranial Aneurysm Detection: A Comprehensive Systematic Review and Meta-analysis of Deep Learning Models Performance, Clinical Integration, and Future Directions

BACKGROUND

Cerebral aneurysms pose a significant risk to patient safety, particularly when ruptured, emphasizing the need for early detection and accurate prediction. Traditional diagnostic methods, reliant on clinician-based evaluations, face challenges in sensitivity and consistency, prompting the exploration of deep learning (DL) systems for improved performance.

METHODS

This systematic review and meta-analysis assessed the performance of DL models in detecting and predicting intracranial aneurysms compared to clinician-based evaluations. Imaging modalities included CT angiography (CTA), digital subtraction angiography (DSA), and time-of-flight MR angiography (TOF-MRA). Data on lesion-wise sensitivity, specificity, and the impact of DL assistance on clinician performance were analyzed. Subgroup analyses evaluated DL sensitivity by aneurysm size and location, and interrater agreement was measured using Fleiss' κ.

RESULTS

DL systems achieved an overall lesion-wise sensitivity of 90 % and specificity of 94 %, outperforming human diagnostics. Clinician specificity improved significantly with DL assistance, increasing from 83 % to 85 % in the patient-wise scenario and from 93 % to 95 % in the lesion-wise scenario. Similarly, clinician sensitivity also showed notable improvement with DL assistance, rising from 82 % to 96 % in the patient-wise scenario and from 82 % to 88 % in the lesion-wise scenario. Subgroup analysis showed DL sensitivity varied with aneurysm size and location, reaching 100 % for aneurysms larger than 10 mm. Additionally, DL assistance improved interrater agreement among clinicians, with Fleiss' κ increasing from 0.668 to 0.862.

CONCLUSIONS

DL models demonstrate transformative potential in managing cerebral aneurysms by enhancing diagnostic accuracy, reducing missed cases, and supporting clinical decision-making. However, further validation in diverse clinical settings and seamless integration into standard workflows are necessary to fully realize the benefits of DL-driven diagnostics.

Accuracy of segment anything model for classification of vascular stenosis in digital subtraction angiography

BACKGROUND

This retrospective study evaluates the diagnostic performance of an optimized comprehensive multi-stage framework based on the Segment Anything Model (SAM), which we named Dr-SAM, for detecting and grading vascular stenosis in the abdominal aorta and iliac arteries using digital subtraction angiography (DSA).

MATERIALS AND METHODS

A total of 100 DSA examinations were conducted on 100 patients. The infrarenal abdominal aorta (AAI), common iliac arteries (CIA), and external iliac arteries (EIA) were independently evaluated by two experienced radiologists using a standardized 5-point grading scale. Dr-SAM analyzed the same DSA images, and its assessments were compared with the average stenosis grading provided by the radiologists. Diagnostic accuracy was evaluated using Cohen's kappa, specificity, sensitivity, and Wilcoxon signed-rank tests.

RESULTS

Interobserver agreement between radiologists, which established the reference standard, was strong (Cohen's kappa: CIA right = 0.95, CIA left = 0.94, EIA right = 0.98, EIA left = 0.98, AAI = 0.79). Dr-SAM showed high agreement with radiologist consensus for CIA (κ = 0.93 right, 0.91 left), moderate agreement for EIA (κ = 0.79 right, 0.76 left), and fair agreement for AAI (κ = 0.70). Dr-SAM demonstrated excellent specificity (up to 1.0) and robust sensitivity (0.67-0.83). Wilcoxon tests revealed no significant differences between Dr-SAM and radiologist grading (p > 0.05).

CONCLUSION

Dr-SAM proved to be an accurate and efficient tool for vascular assessment, with the potential to streamline diagnostic workflows and reduce variability in stenosis grading. Its ability to deliver rapid and consistent evaluations may contribute to earlier detection of disease and the optimization of treatment strategies. Further studies are needed to confirm these findings in prospective settings and to enhance its capabilities, particularly in the detection of occlusions.

Evaluating a clinically available artificial intelligence model for intracranial aneurysm detection: a multi-reader study and algorithmic audit

PURPOSE

We aimed to validate a clinically available artificial intelligence (AI) model to assist general radiologists in the detection of intracranial aneurysm (IA) in a multi-reader multi-case (MRMC) study, and to explore its performance in routine clinical settings.

METHODS

Two distinct cohorts of head CT angiography (CTA) data were assembled to validate an AI model. Cohort 1, comprising gold-standard consecutive CTA cases, was used in an MRMC study involving six board-certified general radiologists. Cohort 2, representing clinical CTA cases, was used to simulate a routine clinical setting. Following these evaluations, an algorithmic audit was conducted to identify any unusual or unexpected behaviors exhibited by the model.

RESULTS

Cohort 1 consisted of 131 CTA cases, while Cohort 2 included 515 CTA cases. In the MRMC study, the AI-assisted strategy demonstrated a significant improvement in aneurysm diagnostic performance, with the area under the receiver operating characteristic curve increasing from 0.815 (95%CI: 0.754-0.875) to 0.875 (95%CI: 0.831-0.921; p = 0.008). In the AI-based first-reader study, 60.4% of the CTA cases were identified as negative by the AI, with a high negative predictive value of 0.994 (95%CI: 0.977-0.999). The algorithmic audit highlighted two issues for improvement: the accurate detection of tiny aneurysms and the effective exclusion of false-positive lesions.

CONCLUSION

This study highlights the clinical utility of a high-performance AI model in detecting IAs, significantly improving general radiologists' diagnostic performance with the potential to reduce their workload in routine clinical practice. The algorithmic audit offers insights to guide the development and validation of future AI models.

Systematic review of artificial intelligence methods for detection and segmentation of unruptured intracranial aneurysms using medical imaging

Unruptured intracranial aneurysms are protuberances that appear in cerebral arteries, and their diagnostic evaluation can be a complex, time-consuming, and exhaustive task. In recent years, computer-aided systems have been developed to improve diagnostic processes. Although the proposed methods have already been reviewed to assess their suitability for clinical use, the segmentation methods have not been reviewed in detail, nor has there been a standardized way to compare segmentation and detection tasks. A systematic review was conducted to examine the technical and methodological factors contributing to this limitation. The analysis encompassed 49 studies conducted between 2019 and 2023 that utilized artificial intelligence methods and any medical imaging modality for the detection or segmentation of intracranial aneurysms. Most of the included studies focused exclusively on detection (57%), magnetic resonance angiography was the predominant imaging modality (47%), and the methodologies generally used 3D imaging as the input (71%). The reported sensitivities ranged from 0.68 to 0.90, specificities from 0.18 to 1.0, false positives per case from 0.18 to 13.8, and the Dice similarity coefficient from 0.53 to 0.98. Variations in aneurysm size were found to have a substantial impact on system performance. Studies were evaluated using a diagnostic accuracy study quality assessment tool, which revealed significant concerns regarding applicability. These concerns primarily stem from the poor reproducibility and inconsistent reporting of metrics. Recommendations for reporting outcomes were made to compare procedures across different types of imaging and tasks.

Enhancing Radiologists' Performance in Detecting Cerebral Aneurysms Using a Deep Learning Model: A Multicenter Study

RATIONALE AND OBJECTIVES

This study aimed to develop a deep learning (DL)-based model for detecting and diagnosing cerebral aneurysms in clinical settings, with and without human assistance.

MATERIALS AND METHODS

The DL model was trained using data from 3829 patients across 11 clinical centers and tested on 484 patients from three institutions. Image interpretations were conducted by 10 radiologists (four junior, six senior), the DL model alone, and a combination of radiologists with the DL model. Time spent on post-processing and reading was recorded. The analysis of the area under the curve (AUC), sensitivity, and specificity for the above-mentioned three reading modes was performed at both the lesion and patient levels.

RESULTS

Combining the DL model with radiologists reduced image interpretation time by 37.2% and post-processing time by 90.8%. With DL model assistance, the AUC increased from 0.842 to 0.881 (P = 0.008) for junior radiologists (JRs) and from 0.853 to 0.895 (P < 0.001) for senior radiologists (SRs). With DL model assistance, sensitivity significantly improved at both lesion (JR: 68.9% to 81.6%, P = 0.011; SR: 72.4% to 83.5%, P < 0.001) and patient levels (JR: 76.2% to 86.9%, P = 0.011; SR: 80.1% to 88.2%, P < 0.001). Specificity at the patient level showed improvement (JR: 82.6% to 82.7%, P = 0.005; SR: 82.6% to 86.1%, P = 0.021).

CONCLUSIONS

The DL model enhanced radiologists' diagnostic performance in detecting cerebral aneurysms, especially for JRs, and expedited the workflow.

Added value of artificial intelligence solutions for arterial stenosis detection on head and neck CT angiography: A randomized crossover multi-reader multi-case study

PURPOSE

The purpose of this study was to investigate the added value of artificial intelligence (AI) solutions for the detection of arterial stenosis (AS) on head and neck CT angiography (CTA).

MATERIALS AND METHODS

Patients who underwent head and neck CTA examinations at two hospitals were retrospectively included. CTA examinations were randomized into group 1 (without AI-washout-with AI) and group 2 (with AI-washout-without AI), and six readers (two radiology residents, two non-neuroradiologists, and two neuroradiologists) independently interpreted each CTA examination without and with AI solutions. Additionally, reading time was recorded for each patient. Digital subtraction angiography was used as the standard of reference. The diagnostic performance for AS at lesion and patient levels with four AS thresholds (30 %, 50 %, 70 %, and 100 %) was assessed by calculating sensitivity, false-positive lesions index (FPLI), specificity, and accuracy.

RESULTS

A total of 268 patients (169 men, 63.1 %) with a median age of 65 years (first quartile, 57; third quartile, 72; age range: 28-88 years) were included. At the lesion level, AI improved the sensitivity of all readers by 5.2 % for detecting AS ≥ 30 % (P < 0.001). Concurrently, AI reduced the FPLI of all readers and specifically neuroradiologists for detecting non-occlusive AS (all P < 0.05). At the patient level, AI improved the accuracy of all readers by 4.1 % (73.9 % [1189/1608] without AI vs. 78.0 % [1254/1608] with AI) (P < 0.001). Sensitivity for AS ≥ 30 % and the specificity for AS ≥ 70 % increased for all readers with AI assistance (P = 0.01). The median reading time for all readers was reduced from 268 s without AI to 241 s with AI (P< 0.001).

CONCLUSION

AI-assisted diagnosis improves the performance of radiologists in detecting head and neck AS, and shortens reading time.

Integrated Deep Learning Model for the Detection, Segmentation, and Morphologic Analysis of Intracranial Aneurysms Using CT Angiography

Purpose To develop a deep learning model for the morphologic measurement of unruptured intracranial aneurysms (UIAs) based on CT angiography (CTA) data and validate its performance using a multicenter dataset. Materials and Methods In this retrospective study, patients with CTA examinations, including those with and without UIAs, in a tertiary referral hospital from February 2018 to February 2021 were included as the training dataset. Patients with UIAs who underwent CTA at multiple centers between April 2021 and December 2022 were included as the multicenter external testing set. An integrated deep learning (IDL) model was developed for UIA detection, segmentation, and morphologic measurement using an nnU-Net algorithm. Model performance was evaluated using the Dice similarity coefficient (DSC) and intraclass correlation coefficient (ICC), with measurements by senior radiologists serving as the reference standard. The ability of the IDL model to improve performance of junior radiologists in measuring morphologic UIA features was assessed. Results The study included 1182 patients with UIAs and 578 controls without UIAs as the training dataset (median age, 55 years [IQR, 47-62 years], 1012 [57.5%] female) and 535 patients with UIAs as the multicenter external testing set (median age, 57 years [IQR, 50-63 years], 353 [66.0%] female). The IDL model achieved 97% accuracy in detecting UIAs and achieved a DSC of 0.90 (95% CI: 0.88, 0.92) for UIA segmentation. Model-based morphologic measurements showed good agreement with reference standard measurements (all ICCs > 0.85). Within the multicenter external testing set, the IDL model also showed agreement with reference standard measurements (all ICCs > 0.80). Junior radiologists assisted by the IDL model showed significantly improved performance in measuring UIA size (ICC improved from 0.88 [95% CI: 0.80, 0.92] to 0.96 [95% CI: 0.92, 0.97], P < .001). Conclusion The developed integrated deep learning model using CTA data showed good performance in UIA detection, segmentation, and morphologic measurement and may be used to assist less experienced radiologists in morphologic analysis of UIAs. Keywords: Segmentation, CT Angiography, Head/Neck, Aneurysms, Comparative Studies Supplemental material is available for this article. © RSNA, 2024 See also the commentary by Wang in this issue.

Impact of human and artificial intelligence collaboration on workload reduction in medical image interpretation

Clinicians face increasing workloads in medical imaging interpretation, and artificial intelligence (AI) offers potential relief. This meta-analysis evaluates the impact of human-AI collaboration on image interpretation workload. Four databases were searched for studies comparing reading time or quantity for image-based disease detection before and after AI integration. The Quality Assessment of Studies of Diagnostic Accuracy was modified to assess risk of bias. Workload reduction and relative diagnostic performance were pooled using random-effects model. Thirty-six studies were included. AI concurrent assistance reduced reading time by 27.20% (95% confidence interval, 18.22%-36.18%). The reading quantity decreased by 44.47% (40.68%-48.26%) and 61.72% (47.92%-75.52%) when AI served as the second reader and pre-screening, respectively. Overall relative sensitivity and specificity are 1.12 (1.09, 1.14) and 1.00 (1.00, 1.01), respectively. Despite these promising results, caution is warranted due to significant heterogeneity and uneven study quality.

A new artificial intelligence system for both stomach and small-bowel capsule endoscopy

BACKGROUND AND AIMS

Despite the benefits of artificial intelligence in small-bowel (SB) capsule endoscopy (CE) image reading, information on its application in the stomach and SB CE is lacking.

METHODS

In this multicenter, retrospective diagnostic study, gastric imaging data were added to the deep learning-based SmartScan (SS), which has been described previously. A total of 1069 magnetically controlled GI CE examinations (comprising 2,672,542 gastric images) were used in the training phase for recognizing gastric pathologies, producing a new artificial intelligence algorithm named SS Plus. A total of 342 fully automated, magnetically controlled CE examinations were included in the validation phase. The performance of both senior and junior endoscopists with both the SS Plus-assisted reading (SSP-AR) and conventional reading (CR) modes was assessed.

RESULTS

SS Plus was designed to recognize 5 types of gastric lesions and 17 types of SB lesions. SS Plus reduced the number of CE images required for review to 873.90 (median, 1000; interquartile range [IQR], 814.50-1000) versus 44,322.73 (median, 42,393; IQR, 31,722.75-54,971.25) for CR. Furthermore, with SSP-AR, endoscopists took 9.54 minutes (median, 8.51; IQR, 6.05-13.13) to complete the CE video reading. In the 342 CE videos, SS Plus identified 411 gastric and 422 SB lesions, whereas 400 gastric and 368 intestinal lesions were detected with CR. Moreover, junior endoscopists remarkably improved their CE image reading ability with SSP-AR.

CONCLUSIONS

Our study shows that the newly upgraded deep learning-based algorithm SS Plus can detect GI lesions and help improve the diagnostic performance of junior endoscopists in interpreting CE videos.

Vessel-aware aneurysm detection using multi-scale deformable 3D attention

Manual detection of intracranial aneurysms (IAs) in computed tomography (CT) scans is a complex, time-consuming task even for expert clinicians, and automating the process is no less challenging. Critical difficulties associated with detecting aneurysms include their small (yet varied) size compared to scans and a high potential for false positive (FP) predictions. To address these issues, we propose a 3D, multi-scale neural architecture that detects aneurysms via a deformable attention mechanism that operates on vessel distance maps derived from vessel segmentations and 3D features extracted from the layers of a convolutional network. Likewise, we reformulate aneurysm segmentation as bounding cuboid prediction using binary cross entropy and three localization losses (location, size, IoU). Given three validation sets comprised of 152/138/38 CT scans and containing 126/101/58 aneurysms, we achieved a Sensitivity of 91.3%/97.0%/74.1% @ FP rates 0.53/0.56/0.87, with Sensitivity around 80% on small aneurysms. Manual inspection of outputs by experts showed our model only tends to miss aneurysms located in unusual locations. Code and model weights are available online.

Evaluation of the clinical application value of artificial intelligence in diagnosing head and neck aneurysms

OBJECTIVE

To evaluate the performance of a semi-automated artificial intelligence (AI) software program (CerebralDoc® system) in aneurysm detection and morphological measurement.

METHODS

In this study, 354 cases of computed tomographic angiography (CTA) were retrospectively collected in our hospital. Among them, 280 cases were diagnosed with aneurysms by either digital subtraction angiography (DSA) and CTA (DSA group, n = 102), or CTA-only (non-DSA group, n = 178). The presence or absence of aneurysms, as well as their location and related morphological features determined by AI were evaluated using DSA and radiologist findings. Besides, post-processing image quality from AI and radiologists were also rated and compared.

RESULTS

In the DSA group, AI achieved a sensitivity of 88.24% and an accuracy of 81.97%, whereas radiologists achieved a sensitivity of 95.10% and an accuracy of 84.43%, using DSA results as the gold standard. The AI in the non-DSA group achieved 81.46% sensitivity and 76.29% accuracy, as per the radiologists' findings. The comparison of position consistency results showed better performance under loose criteria than strict criteria. In terms of morphological characteristics, both the DSA and the non-DSA groups agreed well with the diagnostic results for neck width and maximum diameter, demonstrating excellent ICC reliability exceeding 0.80. The AI-generated images exhibited superior quality compared to the standard software for post-processing, while also demonstrating a significantly reduced processing time.

CONCLUSIONS

The AI-based aneurysm detection rate demonstrates a commendable performance, while the extracted morphological parameters exhibit a remarkable consistency with those assessed by radiologists, thereby showcasing significant potential for clinical application.

Classification, detection, and segmentation performance of image-based AI in intracranial aneurysm: a systematic review

BACKGROUND

The detection and management of intracranial aneurysms (IAs) are vital to prevent life-threatening complications like subarachnoid hemorrhage (SAH). Artificial Intelligence (AI) can analyze medical images, like CTA or MRA, spotting nuances possibly overlooked by humans. Early detection facilitates timely interventions and improved outcomes. Moreover, AI algorithms offer quantitative data on aneurysm attributes, aiding in long-term monitoring and assessing rupture risks.

METHODS

We screened four databases (PubMed, Web of Science, IEEE and Scopus) for studies using artificial intelligence algorithms to identify IA. Based on algorithmic methodologies, we categorized them into classification, segmentation, detection and combined, and then their merits and shortcomings are compared. Subsequently, we elucidate potential challenges that contemporary algorithms might encounter within real-world clinical diagnostic contexts. Then we outline prospective research trajectories and underscore key concerns in this evolving field.

RESULTS

Forty-seven studies of IA recognition based on AI were included based on search and screening criteria. The retrospective results represent that current studies can identify IA in different modal images and predict their risk of rupture and blockage. In clinical diagnosis, AI can effectively improve the diagnostic accuracy of IA and reduce missed detection and false positives.

CONCLUSIONS

The AI algorithm can detect unobtrusive IA more accurately in communicating arteries and cavernous sinus arteries to avoid further expansion. In addition, analyzing aneurysm rupture and blockage before and after surgery can help doctors plan treatment and reduce the uncertainties in the treatment process.

Why Medical Students Pursue Radiology: A Current Longitudinal Survey on Motivations and Controversial Issues in Radiology

RATIONALE AND OBJECTIVES

Radiology is an increasingly competitive specialty. Various current factors influence medical students' decision to pursue a radiology career, including artificial intelligence (AI), remote reading, and COVID-19. This study seeks to determine the decision-making factors of all alumni from our medical school who matched into a radiology residency, and to gather opinions on emerging radiology topics.

MATERIALS AND METHODS

A survey querying decision-making factors and opinions on current radiology topics was distributed to all alumni from our medical school (first graduating class in 2011) who previously matched into a diagnostic or interventional radiology residency program (n = 57). Wilcoxon Rank-Sum and Fisher's Exact tests were used to determine statistical significance.

RESULTS

Forty-three of fifty-seven responses were received (75% response rate). The most influential factor that sparked respondents' interest in radiology was a radiology elective (25/43, 58%). Students who will finish radiology training in 2023 or later were more likely to be influenced by a mentor (15/23, 65%) than those who finished radiology training before 2023 (5/20, 25%) (p = 0.04). Respondents reported a 1.6/5 concern about AI negatively impacting their future career in radiology. There was 1.7/5 concern about performing radiology procedures on patients during the COVID-19 pandemic. Respondents predicted that remote reading would have a 3.2/5 positive impact on helping them achieve their preferred lifestyle. Job satisfaction among attending radiologists is rated at 4.3/5.

CONCLUSION

Radiology electives had the greatest influence in piquing students' interest in radiology, while mentorship is assuming increasing influence. AI is perceived as a relatively minimal threat to negatively impact radiologists' jobs. Respondents had little concern about performing radiology procedures during the COVID-19 pandemic. Remote reading is viewed as having a moderately positive impact on lifestyle. Responding radiologists enjoy notably high job satisfaction.

A Systematic Review of Deep-Learning Methods for Intracranial Aneurysm Detection in CT Angiography

Background: Subarachnoid hemorrhage resulting from cerebral aneurysm rupture is a significant cause of morbidity and mortality. Early identification of aneurysms on Computed Tomography Angiography (CTA), a frequently used modality for this purpose, is crucial, and artificial intelligence (AI)-based algorithms can improve the detection rate and minimize the intra- and inter-rater variability. Thus, a systematic review and meta-analysis were conducted to assess the diagnostic accuracy of deep-learning-based AI algorithms in detecting cerebral aneurysms using CTA. Methods: PubMed (MEDLINE), Embase, and the Cochrane Library were searched from January 2015 to July 2023. Eligibility criteria involved studies using fully automated and semi-automatic deep-learning algorithms for detecting cerebral aneurysms on the CTA modality. Eligible studies were assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines and the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool. A diagnostic accuracy meta-analysis was conducted to estimate pooled lesion-level sensitivity, size-dependent lesion-level sensitivity, patient-level specificity, and the number of false positives per image. An enhanced FROC curve was utilized to facilitate comparisons between the studies. Results: Fifteen eligible studies were assessed. The findings indicated that the methods exhibited high pooled sensitivity (0.87, 95% confidence interval: 0.835 to 0.91) in detecting intracranial aneurysms at the lesion level. Patient-level sensitivity was not reported due to the lack of a unified patient-level sensitivity definition. Only five studies involved a control group (healthy subjects), whereas two provided information on detection specificity. Moreover, the analysis of size-dependent sensitivity reported in eight studies revealed that the average sensitivity for small aneurysms (<3 mm) was rather low (0.56). Conclusions: The studies included in the analysis exhibited a high level of accuracy in detecting intracranial aneurysms larger than 3 mm in size. Nonetheless, there is a notable gap that necessitates increased attention and research focus on the detection of smaller aneurysms, the use of a common test dataset, and an evaluation of a consistent set of performance metrics.