Comparison of automated ASPECTS, large vessel occlusion detection and CTP analysis provided by Brainomix and RapidAI in patients with suspected ischaemic stroke

Assessing the clinical reasoning of ChatGPT for mechanical thrombectomy in patients with stroke

BACKGROUND

Artificial intelligence (AI) has become a promising tool in medicine. ChatGPT, a large language model AI Chatbot, shows promise in supporting clinical practice. We assess the potential of ChatGPT as a clinical reasoning tool for mechanical thrombectomy in patients with stroke.

METHODS

An internal validation of the abilities of ChatGPT was first performed using artificially created patient scenarios before assessment of real patient scenarios from the medical center's stroke database. All patients with large vessel occlusions who underwent mechanical thrombectomy at Tulane Medical Center between January 1, 2022 and December 31, 2022 were included in the study. The performance of ChatGPT in evaluating which patients should undergo mechanical thrombectomy was compared with the decisions made by board-certified stroke neurologists and neurointerventionalists. The interpretation skills, clinical reasoning, and accuracy of ChatGPT were analyzed.

RESULTS

102 patients with large vessel occlusions underwent mechanical thrombectomy. ChatGPT agreed with the physician's decision whether or not to pursue thrombectomy in 54.3% of the cases. ChatGPT had mistakes in 8.8% of the cases, consisting of mathematics, logic, and misinterpretation errors. In the internal validation phase, ChatGPT was able to provide nuanced clinical reasoning and was able to perform multi-step thinking, although with an increased rate of making mistakes.

CONCLUSION

ChatGPT shows promise in clinical reasoning, including the ability to factor a patient's underlying comorbidities when considering mechanical thrombectomy. However, ChatGPT is prone to errors as well and should not be relied on as a sole decision-making tool in its present form, but it has potential to assist clinicians with more efficient work flow.

Artificial intelligence-based decision support software to improve the efficacy of acute stroke pathway in the NHS: an observational study

Introduction

In a drip-and-ship model for endovascular thrombectomy (EVT), early identification of large vessel occlusion (LVO) and timely referral to a comprehensive center (CSC) are crucial when patients are admitted to an acute stroke center (ASC). Several artificial intelligence (AI) decision-aid tools are increasingly being used to facilitate the rapid identification of LVO. This retrospective cohort study aimed to evaluate the impact of deploying e-Stroke AI decision support software in the hyperacute stroke pathway on process metrics and patient outcomes at an ASC in the United Kingdom.

Methods

Except for the deployment of e-Stroke on 01 March 2020, there were no significant changes made to the stroke pathway at the ASC. The data were obtained from a prospective stroke registry between 01 January 2019 and 31 March 2021. The outcomes were compared between the 14 months before and 12 months after the deployment of AI (pre-e-Stroke cohort vs. post-e-Stroke cohort) on 01 March 2020. Time window analyses were performed using Welch's t-test. Cochran-Mantel-Haenszel test was used to compare changes in disability at 3 months assessed by modified Rankin Score (mRS) ordinal shift analysis, and Fisher's exact test was used for dichotomised mRS analysis.

Results

In the pre-e-Stroke cohort, 19 of 22 patients referred received EVT. In the post-e-Stroke cohort, 21 of the 25 patients referred were treated. The mean door-in-door-out (DIDO) and door-to-referral times in pre-e-Stroke vs. post-e-Stroke cohorts were 141 vs. 79 min (difference 62 min, 95% CI 96.9-26.8 min, p < 0.001) and 71 vs. 44 min (difference 27 min, 95% CI 47.4-5.4 min, p = 0.01), respectively. The adjusted odds ratio (age and NIHSS) for mRS ordinal shift analysis at 3 months was 3.14 (95% CI 0.99-10.51, p = 0.06) and the dichotomized mRS 0-2 at 3 months was 16% vs. 48% (p = 0.04) in the pre- vs. post-e-Stroke cohorts, respectively.

Conclusion

In this single-center study in the United Kingdom, the DIDO time significantly decreased since the introduction of e-Stroke decision support software into an ASC hyperacute stroke pathway. The reduction in door-in to referral time indicates faster image interpretation and referral for EVT. There was an indication of an increased proportion of patients regaining independent function after EVT. However, this should be interpreted with caution given the small sample size. Larger, prospective studies and further systematic real-world evaluation are needed to demonstrate the widespread generalisability of these findings.

Performance evaluation of two different software programs for automated ASPECTS scoring in patients with suspected stroke

BACKGROUND

The range of software available to radiologists has increased enormously with the advancement of AI. A good example of this is software to determine ASPECTS in the treatment of potential stroke patients.

OBJECTIVE

In this study, two software packages (eASPECTS from Brainomix and VIA_ASPECTS from Siemens) were tested and compared for their performance in the daily clinical routine of a maximum care provider with a 24/7 stroke unit.

METHODS

A total of 637 noncontrast CT images were obtained from consecutive patients with suspected stroke, of whom 73 were finally diagnosed with MCA infarction. Differences in agreement and quantification of agreement were analysed, as well as the correlation and sensitivity, specificity and accuracy compared to raters.

RESULTS

Compared to VIA_ASPECTS, eASPECTS shows good agreement and strong correlation with the raters. VIA_ASPECTS has lower accuracy and low specificity than eASPECTS but a higher sensitivity.

CONCLUSION

Both software products have the potential to be decision support tools for radiologists. There are, however, differences between the two software products in terms of their intended use.

Real-world evaluation of Brainomix e-Stroke software

BACKGROUND AND PURPOSE

Brainomix e-Stroke is an artificial intelligence-based decision support tool that aids the interpretation of CT imaging in the context of acute stroke. While e-Stroke has the potential to improve the speed and accuracy of diagnosis, real-world validation is essential. The aim of this study was to prospectively evaluate the performance of Brainomix e-Stroke in an unselected cohort of patients with suspected acute ischaemic stroke.

METHODS

The study cohort included all patients admitted to the University College London Hospital Hyperacute Stroke Unit between October 2021 and April 2022. For e-ASPECTS and e-CTA, the ground truth was determined by a neuroradiologist with access to all clinical and imaging data. For e-CTP, the values of the core infarct and ischaemic penumbra were compared with those derived from syngo.via, an alternate software used at our institution.

RESULTS

1163 studies were performed in 551 patients admitted during the study period. Of these, 1130 (97.2%) were successfully processed by e-Stroke in an average of 4 min. For identifying acute middle cerebral artery territory ischaemia, e-ASPECTS had an accuracy of 77.0% and was more specific (83.5%) than sensitive (58.6%). The accuracy for identifying hyperdense thrombus was lower (69.1%), which was mainly due to many false positives (positive predictive value of 22.9%). Identification of acute haemorrhage was highly accurate (97.8%) with a sensitivity of 100% and a specificity of 97.6%; false positives were typically caused by areas of calcification. The accuracy of e-CTA for large vessel occlusions was 91.5%. The core infarct and ischaemic penumbra volumes provided by e-CTP strongly correlated with those provided by syngo.via (ρ=0.804-0.979).

CONCLUSION

Brainomix e-Stroke software provides rapid and reliable analysis of CT imaging in the acute stroke setting although, in line with the manufacturer's guidance, it should be used as an adjunct to expert interpretation rather than a standalone decision-making tool.

Automated advanced imaging in acute ischemic stroke. Certainties and uncertainties

The purpose of this is study was to review pearls and pitfalls of advanced imaging, such as computed tomography perfusion and diffusion-weighed imaging and perfusion-weighted imaging in the selection of acute ischemic stroke (AIS) patients suitable for endovascular treatment (EVT) in the late time window (6-24 h from symptom onset). Advanced imaging can quantify infarct core and ischemic penumbra using specific threshold values and provides optimal selection parameters, collectively called target mismatch. More precisely, target mismatch criteria consist of core volume and/or penumbra volume and mismatch ratio (the ratio between total hypoperfusion and core volumes) with precise cut-off values. The parameters of target mismatch are automatically calculated with dedicated software packages that allow a quick and standardized interpretation of advanced imaging. However, this approach has several limitations leading to a misclassification of core and penumbra volumes. In fact, automatic software platforms are affected by technical artifacts and are not interchangeable due to a remarkable vendor-dependent variability, resulting in different estimate of target mismatch parameters. In addition, advanced imaging is not completely accurate in detecting infarct core, that can be under- or overestimated. Finally, the selection of candidates for EVT remains currently suboptimal due to the high rates of futile reperfusion and overselection caused by the use of very stringent inclusion criteria. For these reasons, some investigators recently proposed to replace advanced with conventional imaging in the selection for EVT, after the demonstration that non-contrast CT ASPECTS and computed tomography angiography collateral evaluation are not inferior to advanced images in predicting outcome in AIS patients treated with EVT. However, other authors confirmed that CTP and PWI/DWI postprocessed images are superior to conventional imaging in establishing the eligibility of patients for EVT. Therefore, the routine application of automatic assessment of advanced imaging remains a matter of debate. Recent findings suggest that the combination of conventional and advanced imaging might improving our selection criteria.

Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study

Background

The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence methods to automatically process non-contrast CT (NCCT) brain scans from patients with AIS affecting the middle cerebral artery (MCA) territory and generate an ASPECTS. This study aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth.

Methods

The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory. Each reader scored each scan on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard (expert consensus read with reference to follow-up imaging) was evaluated with and without software support.

Results

A comparison of the area under the curve (AUC) for each reader showed a significant improvement from 0.81 to 0.83 (p = 0.028) with the support of the e-ASPECTS tool. The agreement of reader ASPECTS scoring with the reference standard was improved with e-ASPECTS compared to unassisted reading of scans: Cohen's kappa improved from 0.60 to 0.65, and the case-based weighted Kappa improved from 0.70 to 0.81.

Conclusion

Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.

Accuracy of artificial intelligence software for CT angiography in stroke

OBJECTIVE

Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e-CTA by Brainomix™ Ltd by large-scale independent testing using expert reading as the reference standard.

METHODS

We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e-CTA results with masked expert interpretation of the same scans for the presence and location of laterality-matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e-CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation).

RESULTS

We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e-CTA for detecting arterial abnormality were each 72% (95% CI = 66-77%). Diagnostic accuracy was non-significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72-80%).

INTERPRETATION

Compared to experts, the diagnostic accuracy of e-CTA for identifying acute arterial abnormality was 72-76%. Users of e-CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.

Artificial Intelligence in Acute Ischemic Stroke Subtypes According to Toast Classification: A Comprehensive Narrative Review

The correct recognition of the etiology of ischemic stroke (IS) allows tempestive interventions in therapy with the aim of treating the cause and preventing a new cerebral ischemic event. Nevertheless, the identification of the cause is often challenging and is based on clinical features and data obtained by imaging techniques and other diagnostic exams. TOAST classification system describes the different etiologies of ischemic stroke and includes five subtypes: LAAS (large-artery atherosclerosis), CEI (cardio embolism), SVD (small vessel disease), ODE (stroke of other determined etiology), and UDE (stroke of undetermined etiology). AI models, providing computational methodologies for quantitative and objective evaluations, seem to increase the sensitivity of main IS causes, such as tomographic diagnosis of carotid stenosis, electrocardiographic recognition of atrial fibrillation, and identification of small vessel disease in magnetic resonance images. The aim of this review is to provide overall knowledge about the most effective AI models used in the differential diagnosis of ischemic stroke etiology according to the TOAST classification. According to our results, AI has proven to be a useful tool for identifying predictive factors capable of subtyping acute stroke patients in large heterogeneous populations and, in particular, clarifying the etiology of UDE IS especially detecting cardioembolic sources.