Automated versus manual imaging assessment of early ischemic changes in acute stroke: comparison of two software packages and expert consensus

A cortically-weighted versus total Alberta Stroke Program Early Computed Tomography Score in thrombectomy outcome models

OBJECTIVES

Individual subcortical infarct scoring for the Alberta Stroke Program Early Computed Tomography Score (ASPECTS) can be difficult and is subjected to higher inter-reader variability. This study compares performance of the 10-point ASPECTS with a new 7-point cortically-weighted score in predicting post-thrombectomy functional outcomes.

MATERIALS AND METHODS

Prospective registry data from two comprehensive stroke centers (Site 1 2016-2021; Site 2: 2019-2021) included patients with either M1 segment of middle cerebral artery or internal carotid artery occlusions who underwent thrombectomy. Two multivariate proportional odds training models utilizing either 10-point or 7-point ASPECTS predicting 90-day shift in modified Rankin score were generated using Site 1 data and validated with Site 2 data. Models were compared using multiclass receiver operator characteristics, corrected Akaike's Information Criterion, and likelihood ratio test.

RESULTS

Of 328 patients (Site 1 = 181, Site 2 = 147), median age was 71y (IQR 61-82), 119 (36%) had internal carotid artery occlusions, and median 10-point ASPECTS was 9 (IQR 8-10). There was no difference in performance between models using either total or cortically-weighted ASPECTS (p=0.14). Validation cohort data were correctly (i.e., predicting modified Rankin score within one point) classified 50% (cortically-weighted score model) and 56% (total score model) of the time.

CONCLUSIONS

The 7-point cortically-weighted ASPECTS was similarly predictive of post-thrombectomy functional outcome as 10-point ASPECTS. Given noninferior performance, the cortically-weighted score is a potentially reliable, but simplified, alternative to the traditional scoring paradigm, with potential implications in automated image analysis tool development.

Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis

Background

Artificial intelligence-derived software technologies have been developed that are intended to facilitate the review of computed tomography brain scans in patients with suspected stroke.

Objectives

To evaluate the clinical and cost-effectiveness of using artificial intelligence-derived software to support review of computed tomography brain scans in acute stroke in the National Health Service setting.

Methods

Twenty-five databases were searched to July 2021. The review process included measures to minimise error and bias. Results were summarised by research question, artificial intelligence-derived software technology and study type. The health economic analysis focused on the addition of artificial intelligence-derived software-assisted review of computed tomography angiography brain scans for guiding mechanical thrombectomy treatment decisions for people with an ischaemic stroke. The de novo model (developed in R Shiny, R Foundation for Statistical Computing, Vienna, Austria) consisted of a decision tree (short-term) and a state transition model (long-term) to calculate the mean expected costs and quality-adjusted life-years for people with ischaemic stroke and suspected large-vessel occlusion comparing artificial intelligence-derived software-assisted review to usual care.

Results

A total of 22 studies (30 publications) were included in the review; 18/22 studies concerned artificial intelligence-derived software for the interpretation of computed tomography angiography to detect large-vessel occlusion. No study evaluated an artificial intelligence-derived software technology used as specified in the inclusion criteria for this assessment. For artificial intelligence-derived software technology alone, sensitivity and specificity estimates for proximal anterior circulation large-vessel occlusion were 95.4% (95% confidence interval 92.7% to 97.1%) and 79.4% (95% confidence interval 75.8% to 82.6%) for Rapid (iSchemaView, Menlo Park, CA, USA) computed tomography angiography, 91.2% (95% confidence interval 77.0% to 97.0%) and 85.0 (95% confidence interval 64.0% to 94.8%) for Viz LVO (Viz.ai, Inc., San Fransisco, VA, USA) large-vessel occlusion, 83.8% (95% confidence interval 77.3% to 88.7%) and 95.7% (95% confidence interval 91.0% to 98.0%) for Brainomix (Brainomix Ltd, Oxford, UK) e-computed tomography angiography and 98.1% (95% confidence interval 94.5% to 99.3%) and 98.2% (95% confidence interval 95.5% to 99.3%) for Avicenna CINA (Avicenna AI, La Ciotat, France) large-vessel occlusion, based on one study each. These studies were not considered appropriate to inform cost-effectiveness modelling but formed the basis by which the accuracy of artificial intelligence plus human reader could be elicited by expert opinion. Probabilistic analyses based on the expert elicitation to inform the sensitivity of the diagnostic pathway indicated that the addition of artificial intelligence to detect large-vessel occlusion is potentially more effective (quality-adjusted life-year gain of 0.003), more costly (increased costs of £8.61) and cost-effective for willingness-to-pay thresholds of £3380 per quality-adjusted life-year and higher.

Limitations and conclusions

The available evidence is not suitable to determine the clinical effectiveness of using artificial intelligence-derived software to support the review of computed tomography brain scans in acute stroke. The economic analyses did not provide evidence to prefer the artificial intelligence-derived software strategy over current clinical practice. However, results indicated that if the addition of artificial intelligence-derived software-assisted review for guiding mechanical thrombectomy treatment decisions increased the sensitivity of the diagnostic pathway (i.e. reduced the proportion of undetected large-vessel occlusions), this may be considered cost-effective.

Future work

Large, preferably multicentre, studies are needed (for all artificial intelligence-derived software technologies) that evaluate these technologies as they would be implemented in clinical practice.

Study registration

This study is registered as PROSPERO CRD42021269609.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR133836) and is published in full in Health Technology Assessment; Vol. 28, No. 11. See the NIHR Funding and Awards website for further award information.

Real-world evaluation of Brainomix e-Stroke software

BACKGROUND AND PURPOSE

Brainomix e-Stroke is an artificial intelligence-based decision support tool that aids the interpretation of CT imaging in the context of acute stroke. While e-Stroke has the potential to improve the speed and accuracy of diagnosis, real-world validation is essential. The aim of this study was to prospectively evaluate the performance of Brainomix e-Stroke in an unselected cohort of patients with suspected acute ischaemic stroke.

METHODS

The study cohort included all patients admitted to the University College London Hospital Hyperacute Stroke Unit between October 2021 and April 2022. For e-ASPECTS and e-CTA, the ground truth was determined by a neuroradiologist with access to all clinical and imaging data. For e-CTP, the values of the core infarct and ischaemic penumbra were compared with those derived from syngo.via, an alternate software used at our institution.

RESULTS

1163 studies were performed in 551 patients admitted during the study period. Of these, 1130 (97.2%) were successfully processed by e-Stroke in an average of 4 min. For identifying acute middle cerebral artery territory ischaemia, e-ASPECTS had an accuracy of 77.0% and was more specific (83.5%) than sensitive (58.6%). The accuracy for identifying hyperdense thrombus was lower (69.1%), which was mainly due to many false positives (positive predictive value of 22.9%). Identification of acute haemorrhage was highly accurate (97.8%) with a sensitivity of 100% and a specificity of 97.6%; false positives were typically caused by areas of calcification. The accuracy of e-CTA for large vessel occlusions was 91.5%. The core infarct and ischaemic penumbra volumes provided by e-CTP strongly correlated with those provided by syngo.via (ρ=0.804-0.979).

CONCLUSION

Brainomix e-Stroke software provides rapid and reliable analysis of CT imaging in the acute stroke setting although, in line with the manufacturer's guidance, it should be used as an adjunct to expert interpretation rather than a standalone decision-making tool.

Localization of early infarction on non-contrast CT images in acute ischemic stroke with deep learning approach

Localization of early infarction on first-line Non-contrast computed tomogram (NCCT) guides prompt treatment to improve stroke outcome. Our previous study has shown a good performance in the identification of ischemic injury on NCCT. In the present study, we developed a deep learning (DL) localization model to help localize the early infarction sign on NCCT. This retrospective study included consecutive 517 ischemic stroke (IS) patients who received NCCT within 12 h after stroke onset. A total of 21,436 infarction patches and 20,391 non-infarction patches were extracted from the slice pool of 1,634 NCCT according to brain symmetricity property. The generated patches were fed into different pretrained convolutional neural network (CNN) models such as Visual Geometry Group 16 (VGG16), GoogleNet, Residual Networks 50 (ResNet50), Inception-ResNet-v2 (IR-v2), Inception-v3 and Inception-v4. The selected VGG16 model could detect the early infarction in both supratentorial and infratentorial regions to achieve an average area under curve (AUC) 0.73 after extensive customization. The properly tuned-VGG16 model could identify the early infarction in the cortical, subcortical and cortical plus subcortical areas of supratentorial region with the mean AUC > 0.70. Further, the model could attain 95.6% of accuracy on recognizing infarction lesion in 494 out of 517 IS patients.

Artificial intelligence-driven ASPECTS for the detection of early stroke changes in non-contrast CT: a systematic review and meta-analysis

BACKGROUND

Recent advances in machine learning have enabled development of the automated Alberta Stroke Program Early CT Score (ASPECTS) prediction algorithms using non-contrast enhanced computed tomography (NCCT) scans. The applicability of automated ASPECTS in daily clinical practice is yet to be established. The objective of this meta-analysis was to directly compare the performance of automated and manual ASPECTS predictions in recognizing early stroke changes on NCCT.

METHODS

The MEDLINE, Scopus, and Cochrane databases were searched. The last database search was performed on March 10, 2022. Studies reporting the diagnostic performance and validity of automated ASPECTS software compared with expert readers were included. The outcomes were the interobserver reliability of outputs between ASPECTS versus expert readings, experts versus reference standard, and ASPECTS versus reference standard by means of pooled Fisher's Z transformation of the interclass correlation coefficients (ICCs).

RESULTS

Eleven studies were included in the meta-analysis, involving 1976 patients. The meta-analyses showed good interobserver reliability between experts (ICC 0.72 (95% CI 0.63 to 0.79); p<0.001), moderate reliability in the correlation between automated and expert readings (ICC 0.54 (95% CI 0.40 to 0.67); p<0.001), good reliability between the total expert readings and the reference standard (ICC 0.62 (95% CI 0.52 to 0.71); p<0.001), and good reliability between the automated predictions and the reference standard (ICC 0.72 (95% CI 0.61 to 0.80); p<0.001).

CONCLUSIONS

Artificial intelligence-driven ASPECTS software has comparable or better performance than physicians in terms of recognizing early stroke changes on NCCT.

Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study

Background

The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence methods to automatically process non-contrast CT (NCCT) brain scans from patients with AIS affecting the middle cerebral artery (MCA) territory and generate an ASPECTS. This study aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth.

Methods

The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory. Each reader scored each scan on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard (expert consensus read with reference to follow-up imaging) was evaluated with and without software support.

Results

A comparison of the area under the curve (AUC) for each reader showed a significant improvement from 0.81 to 0.83 (p = 0.028) with the support of the e-ASPECTS tool. The agreement of reader ASPECTS scoring with the reference standard was improved with e-ASPECTS compared to unassisted reading of scans: Cohen's kappa improved from 0.60 to 0.65, and the case-based weighted Kappa improved from 0.70 to 0.81.

Conclusion

Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.

FDA-approved machine learning algorithms in neuroradiology: A systematic review of the current evidence for approval

Over the past decade, machine learning (ML) and artificial intelligence (AI) have become increasingly prevalent in the medical field. In the United States, the Food and Drug Administration (FDA) is responsible for regulating AI algorithms as "medical devices" to ensure patient safety. However, recent work has shown that the FDA approval process may be deficient. In this study, we evaluate the evidence supporting FDA-approved neuroalgorithms, the subset of machine learning algorithms with applications in the central nervous system (CNS), through a systematic review of the primary literature. Articles covering the 53 FDA-approved algorithms with applications in the CNS published in PubMed, EMBASE, Google Scholar and Scopus between database inception and January 25, 2022 were queried. Initial searches identified 1505 studies, of which 92 articles met the criteria for extraction and inclusion. Studies were identified for 26 of the 53 neuroalgorithms, of which 10 algorithms had only a single peer-reviewed publication. Performance metrics were available for 15 algorithms, external validation studies were available for 24 algorithms, and studies exploring the use of algorithms in clinical practice were available for 7 algorithms. Papers studying the clinical utility of these algorithms focused on three domains: workflow efficiency, cost savings, and clinical outcomes. Our analysis suggests that there is a meaningful gap between the FDA approval of machine learning algorithms and their clinical utilization. There appears to be room for process improvement by implementation of the following recommendations: the provision of compelling evidence that algorithms perform as intended, mandating minimum sample sizes, reporting of a predefined set of performance metrics for all algorithms and clinical application of algorithms prior to widespread use. This work will serve as a baseline for future research into the ideal regulatory framework for AI applications worldwide.

Performance of Automated ASPECTS Software and Value as a Computer-Aided Detection Tool

BACKGROUND AND PURPOSE

ASPECTS quantifies early ischemic changes in anterior circulation stroke on NCCT but has interrater variability. We examined the agreement of conventional and automated ASPECTS and studied the value of computer-aided detection.

MATERIALS AND METHODS

We retrospectively collected imaging data from consecutive patients with acute ischemic stroke with large-vessel occlusion undergoing thrombectomy. Five raters scored conventional ASPECTS on baseline NCCTs, which were also processed by RAPID software. Conventional and automated ASPECTS were compared with a consensus criterion standard. We determined the agreement over the full ASPECTS range as well as dichotomized, reflecting thrombectomy eligibility according to the guidelines (ASPECTS 0-5 versus 6-10). Raters subsequently scored ASPECTS on the same NCCTs with assistance of the automated ASPECTS outputs, and agreement was obtained.

RESULTS

For the total of 175 cases, agreement among raters individually and the criterion standard varied from fair to good (weighted κ = between 0.38 and 0.76) and was moderate (weighted κ = 0.59) for the automated ASPECTS. The agreement of all raters individually versus the criterion standard improved with software assistance, as did the interrater agreement (overall Fleiss κ = 0.15-0.23; P < .001 and .39 to .55; P = .01 for the dichotomized ASPECTS).

CONCLUSIONS

Automated ASPECTS had agreement with the criterion standard similar to that of conventional ASPECTS. However, including automated ASPECTS during the evaluation of NCCT in acute stroke improved the agreement with the criterion standard and improved interrater agreement, which could, therefore, result in more uniform scoring in clinical practice.

Validation of two automated ASPECTS software on non-contrast computed tomography scans of patients with acute ischemic stroke

Purpose

The Alberta Stroke Program Early Computed Tomography Score (ASPECTS) was designed for semi-quantitative assessment of early ischemic changes on non-contrast computed tomography (NCCT) for acute ischemic stroke (AIS). We evaluated two automated ASPECTS software in comparison with reference standard.

Methods

NCCT of 276 AIS patients were retrospectively reviewed (March 2018–June 2020). A three-radiologist consensus for ASPECTS was used as reference standard. Imaging data from both baseline and follow-up were evaluated for reference standard. Automated ASPECTS were calculated from baseline NCCT with 1-mm and 5-mm slice thickness, respectively. Agreement between automated ASPECTS and reference standard was assessed using intra-class correlation coefficient (ICC). Correlation of automated ASPECTS with baseline stroke severity (NIHSS) and follow-up ASPECTS were evaluated using Spearman correlation analysis.

Results

In score-based analysis, automated ASPECTS calculated from 5-mm slice thickness images agreed well with reference standard (software A: ICC = 0.77; software B: ICC = 0.65). Bland–Altman analysis revealed that the mean differences between automated ASPECTS and reference standard were ≤ 0.6. In region-based analysis, automated ASPECTS derived from 5-mm slice thickness images by software A showed higher sensitivity (0.60 vs. 0.54), lower specificity (0.91 vs. 0.94), and higher AUC (0.76 vs. 0.74) than those using 1-mm slice thickness images (p &lt; 0.05). Automated ASPECTS derived from 5-mm slice thickness images by software B showed higher sensitivity (0.56 vs. 0.51), higher specificity (0.87 vs. 0.81), higher accuracy (0.80 vs. 0.73), and higher AUC (0.71 vs. 0.66) than those using 1-mm slice thickness images (p &lt; 0.05). Automated ASPECTS were significantly associated with baseline NIHSS and follow-up ASPECTS.

Conclusion

Automated ASPECTS showed good reliability and 5 mm was the optimal slice thickness.

Machine learning and acute stroke imaging

BACKGROUND

In recent years, machine learning (ML) has had notable success in providing automated analyses of neuroimaging studies, and its role is likely to increase in the future. Thus, it is paramount for clinicians to understand these approaches, gain facility with interpreting ML results, and learn how to assess algorithm performance.

OBJECTIVE

To provide an overview of ML, present its role in acute stroke imaging, discuss methods to evaluate algorithms, and then provide an assessment of existing approaches.

METHODS

In this review, we give an overview of ML techniques commonly used in medical imaging analysis and methods to evaluate performance. We then review the literature for relevant publications. Searches were run in November 2021 in Ovid Medline and PubMed. Inclusion criteria included studies in English reporting use of artificial intelligence (AI), machine learning, or similar techniques in the setting of, and in applications for, acute ischemic stroke or mechanical thrombectomy. Articles that included image-level data with meaningful results and sound ML approaches were included in this discussion.

RESULTS

Many publications on acute stroke imaging, including detection of large vessel occlusion, detection and quantification of intracranial hemorrhage and detection of infarct core, have been published using ML methods. Imaging inputs have included non-contrast head CT, CT angiograph and MRI, with a range of performances. We discuss and review several of the most relevant publications.

CONCLUSIONS

ML in acute ischemic stroke imaging has already made tremendous headway. Additional applications and further integration with clinical care is inevitable. Thus, facility with these approaches is critical for the neurointerventional clinician.

Benefit and risk of intravenous alteplase in patients with acute large vessel occlusion stroke and low ASPECTS

BACKGROUND

The benefit of best medical treatment including intravenous alteplase (IVT) before mechanical thrombectomy (MT) in patients with acute ischemic stroke and extensive early ischemic changes on baseline CT remains uncertain. The purpose of this study was to evaluate the benefit of IVT for patients with low ASPECTS (Alberta Stroke Programme Early CT Score) compared with patients with or without MT.

METHODS

This multicenter study pooled consecutive patients with anterior circulation acute stroke and ASPECTS≤5 to analyze the impact of IVT on functional outcome, and to compare bridging IVT with direct MT. Functional endpoints were the rates of good (modified Rankin Scale (mRS) score ≤2) and very poor (mRS ≥5) outcome at day 90. Safety endpoint was the occurrence of symptomatic intracranial hemorrhage (sICH).

RESULTS

429 patients were included. 290 (68%) received IVT and 168 (39%) underwent MT. The rate of good functional outcome was 14.4% (95% CI 7.1% to 21.8%) for patients who received bridging IVT and 24.4% (95% CI 16.5% to 32.2%) for those who underwent direct MT. The rate of sICH was significantly higher in patients with bridging IVT compared with direct MT (17.8% vs 6.4%, p=0.004). In multivariable logistic regression analysis, IVT was significantly associated with very poor outcome (OR 2.22, 95% CI 1.05 to 4.73, p=0.04) and sICH (OR 3.44, 95% CI 1.18 to 10.07, p=0.02). Successful recanalization, age, and ASPECTS were associated with good functional outcome.

CONCLUSIONS

Bridging IVT in patients with low ASPECTS was associated with very poor functional outcome and an increased risk of sICH. The benefit of this treatment should therefore be carefully weighed in such scenarios. Further randomized controlled trials are required to validate our findings.

External Validation of e-ASPECTS Software for Interpreting Brain CT in Stroke

OBJECTIVE

The purpose of this study was to test e-ASPECTS software in patients with stroke. Marketed as a decision-support tool, e-ASPECTS may detect features of ischemia or hemorrhage on computed tomography (CT) imaging and quantify ischemic extent using Alberta Stroke Program Early CT Score (ASPECTS).

METHODS

Using CT from 9 stroke studies, we compared software with masked experts. As per indications for software use, we assessed e-ASPECTS results for patients with/without middle cerebral artery (MCA) ischemia but no other cause of stroke. In an analysis outside the intended use of the software, we enriched our dataset with non-MCA ischemia, hemorrhage, and mimics to simulate a representative "front door" hospital population. With final diagnosis as the reference standard, we tested the diagnostic accuracy of e-ASPECTS for identifying stroke features (ischemia, hyperattenuated arteries, and hemorrhage) in the representative population.

RESULTS

We included 4,100 patients (51% women, median age = 78 years, National Institutes of Health Stroke Scale [NIHSS] = 10, onset to scan = 2.5 hours). Final diagnosis was ischemia (78%), hemorrhage (14%), or mimic (8%). From 3,035 CTs with expert-rated ASPECTS, most (2084/3035, 69%) e-ASPECTS results were within one point of experts. In the representative population, the diagnostic accuracy of e-ASPECTS was 71% (95% confidence interval [CI] = 70-72%) for detecting ischemic features, 85% (83-86%) for hemorrhage. Software identified more false positive ischemia (12% vs 2%) and hemorrhage (14% vs <1%) than experts.

INTERPRETATION

On independent testing, e-ASPECTS provided moderate agreement with experts and overcalled stroke features. Therefore, future prospective trials testing impacts of artificial intelligence (AI) software on patient care and outcome are required before widespread implementation of stroke decision-support software. ANN NEUROL 2022;92:943-957.

Total and regional ASPECT score for non-contrast CT, CT angiography, and CT perfusion: inter-rater agreement and its association with the final infarction in acute ischemic stroke patients

BACKGROUND

Alberta Stroke Program Early Computed Tomography Score (ASPECTS) is a grading system to assess the extent and distribution of early ischemic changes.

PURPOSE

To assess inter-rater agreement for total and regional ASPECTS on non-contrast computed tomography (NCCT) images, CT angiography source images (CTA-SI), and CT-perfusion cerebral blood volume (CTP-CBV) maps, and their association with final infarction in patients with acute ischemic stroke (AIS).

MATERIAL AND METHODS

A total of 96 consecutive patients with AIS who underwent pre-treatment NCCT and CTP were retrospectively enrolled. CTA-SI was reconstructed using the raw data of CTP. Total and regional ASPECTS were assessed on baseline NCCT, CTA-SI, and CTP-CBV, and on follow-up NCCT or diffusion-weighted imaging. Follow-up ASPECTS served as the reference standard for final infarction.

RESULTS

CTP-CBV demonstrated higher concordance for total ASPECTS (interclass correlation coefficient, 0.895 vs. 0.771 vs. 0.777) and regional ASPECTS in internal capsule, lentiform, caudate nuclei, M5 and M6, compared with NCCT and CTA-SI. CTP-CBV showed a trend of stronger correlation with final ASPECTS than NCCT and CTA-SI (0.717 vs. 0.711 vs. 0.565; P > 0.05). ASPECTS in the internal capsule (ρ, 0.756 vs. 0.556; P = 0.016) and caudate nucleus (ρ, 0.717 vs. 0.476; P = 0.010) on CTP-CBV were more strongly correlated with follow-up ASPECTS than NCCT. CTP-CBV showed higher accuracy for predicting final infarction in the internal capsule (92.5% vs. 90.3% and 87.1%; P > 1.000, P = 0.125, respectively) and caudate nucleus (87.1% vs. 79.6% and 77.4%; P = 0.453, P = 0.039, respectively) than CTA-SI and NCCT.

CONCLUSION

CTP-CBV ASPECTS might be more reliable for delineating early ischemic changes and predicting final infarction.

Deep Learning Applications for Acute Stroke Management

Brain imaging is essential to the clinical care of patients with stroke, a leading cause of disability and death worldwide. Whereas advanced neuroimaging techniques offer opportunities for aiding acute stroke management, several factors, including time delays, inter-clinician variability, and lack of systemic conglomeration of clinical information, hinder their maximal utility. Recent advances in deep machine learning (DL) offer new strategies for harnessing computational medical image analysis to inform decision making in acute stroke. We examine the current state of the field for DL models in stroke triage. First, we provide a brief, clinical practice-focused primer on DL. Next, we examine real-world examples of DL applications in pixel-wise labeling, volumetric lesion segmentation, stroke detection, and prediction of tissue fate postintervention. We evaluate recent deployments of deep neural networks and their ability to automatically select relevant clinical features for acute decision making, reduce inter-rater variability, and boost reliability in rapid neuroimaging assessments, and integrate neuroimaging with electronic medical record (EMR) data in order to support clinicians in routine and triage stroke management. Ultimately, we aim to provide a framework for critically evaluating existing automated approaches, thus equipping clinicians with the ability to understand and potentially apply DL approaches in order to address challenges in clinical practice. ANN NEUROL 2022.

Automated ASPECTS in acute ischemic stroke: comparison of the overall scores and Hounsfield unit values of two software packages and radiologists with different levels of experience

BACKGROUND

ASPECTS is a simple, rapid, and semi-quantitative method for detecting early ischemic changes (EIC). However, the agreement between software applications and neuroradiologists varies greatly.

PURPOSE

To compare ASPECTS calculated by using automated software tools to neuroradiologists evaluation in patients with acute ischemic stroke (AIS).

MATERIAL AND METHODS

Retrospectively, 61 patients with large vessel occlusion (LVO) who underwent multimodal stroke computed tomography (CT) were evaluated using two automated ASPECTS software tools (NSK and RAPID) and three neuroradiologists with different experiences (two senior neuroradiologists and one junior neuroradiologist). Four weeks later, the same three neuroradiologists re-evaluated the ASPECTS in consensus using the baseline CT and follow-up non-contrast CT (NCCT). Interclass correlation coefficients (ICCs) and Pearson correlation coefficients were applied for statistical analysis.

RESULTS

The HU value exhibited the greatest correlation in the insular lobe (r = 0.81; P < 0.001) and the lowest correlation in the internal capsule (r = 0.65; P < 0.001) between NSK and RAPID. Software analysis and human readers showed excellent agreement with the consensus reading. Compared with the consensus reading, the correlation of the two senior radiologists (ICC = 0.975 and 0.969, respectively) were higher than that of junior radiologist (ICC = 0.869), and the consistency values of the NSK and RAPID software tools after 6 h of onset to imaging (ICC = 0.894 and 0.874, respectively) were greater than those within 6 h of onset (ICC = 0.746 and 0.828, respectively).

CONCLUSION

For patients experiencing AIS due to LVO, the ASPECTS calculated with automated software agrees well with the predefined consensus score but is inferior to that of senior radiologists.

Impact of Encephalomalacia and White Matter Hyperintensities on ASPECTS in Patients With Acute Ischemic Stroke: Comparison of Automated- and Radiologist-Derived Scores

Background: Automated software-based Alberta Stroke Program Early CT Score (ASPECTS) on unenhanced CT is associated with clinical outcomes after acute stroke. However, encephalomalacia or white matter hyperintensities (WMHs) may result in a falsely low automated ASPECTS if such findings are interpreted as early ischemia. Objective: To assess the impact of encephalomalacia and WMH on automated ASPECTS in patients with acute stroke, in comparison with radiologist-derived ASPECTS and clinical outcomes. Methods: This retrospective three-center study included 459 patients (322 men, 137 women; median age, 65 years) with acute ischemic stroke treated by IV thrombolysis who underwent baseline unenhanced CT within 6 hours after symptom onset and MRI within 24 hours after treatment. ASPECTS was determined by automated software and by three radiologists in consensus. Presence of encephalomalacia and extent of WMHs [categorized using the modified Scheltens scale (mSS)] were also determined using MRI. Kappa coefficients were used to compare ASPECTS between automated and radiologist-consensus methods. Multivariable logistic regression analyses and ROC analyses were performed to explore the predictive utility of baseline ASPECTS for unfavorable clinical outcome (90-day modified Rankin Scale score of 3-6) after thrombolysis. Results: Median automated ASPECTS was 9, and median radiologist-consensus ASPECTS was 10. Agreement between automated and radiologist-consensus ASPECTS, expressed as kappa, was 0.68, though was 0.76 in patients without encephalomalacia and 0.08 in patients with encephalomalacia. In patients without encephalomalacia, agreement decreased as the mSS score increased (e.g., 0.78 in subgroup with mSS score <10 vs 0.19 in subgroup with mSS >20). By anatomic region, agreement was highest for M5 (κ=0.52) and lowest for internal capsule (κ=0.18). In multivariable analyses, both automated (odds ratio=0.69) and radiologist-consensus (odds ratio=0.57) ASPECTS independently predicted unfavorable clinical outcome. For unfavorable outcome, automated ASPECTS had AUC of 0.70, sensitivity of 60.4%, and specificity of 71.0%, while radiologist-consensus ASPECTS had AUC of 0.72, sensitivity of 60.4%, and specificity of 80.5%. Conclusion: Presence of encephalomalacia or extensive WMH results in lower automated ASPECTS than radiologist-consensus ASPECTS, which may impact predictive utility of automated ASPECTS. Clinical Impact: When using automated ASPECTS, radiologists should manually confirm the score in patients with encephalomalacia or extensive leukoencephalopathy.

Quantitative Lesion Water Uptake as Stroke Imaging Biomarker: A Tool for Treatment Selection in the Extended Time Window?

BACKGROUND AND PURPOSE

Patients presenting in the extended time window may benefit from mechanical thrombectomy. However, selection for mechanical thrombectomy in this patient group has only been performed using specialized image processing platforms, which are not widely available. We hypothesized that quantitative lesion water uptake calculated in acute stroke computed tomography (CT) may serve as imaging biomarker to estimate ischemic lesion progression and predict clinical outcome in patients undergoing mechanical thrombectomy in the extended time window.

METHODS

All patients with ischemic anterior circulation stroke presenting within 4.5 to 24 hours after symptom onset who received initial multimodal CT between August 2014 and March 2020 and underwent mechanical thrombectomy were analyzed. Quantitative lesion net water uptake was calculated from the admission CT. Prediction of clinical outcome was assessed using univariable receiver operating characteristic curve analysis and logistic regression analyses.

RESULTS

One hundred two patients met the inclusion criteria. In the multivariable logistic regression analysis, net water uptake (odds ratio, 0.78 [95% CI, 0.64-0.95], P=0.01), age (odds ratio, 0.94 [95% CI, 0.88-0.99]; P=0.02), and National Institutes of Health Stroke Scale (odds ratio, 0.88 [95% CI, 0.79-0.99], P=0.03) were significantly and independently associated with favorable outcome (modified Rankin Scale score ≤1), adjusted for degree of recanalization and Alberta Stroke Program Early CT Score. A multivariable predictive model including the above parameters yielded the highest diagnostic ability in the classification of functional outcome, with an area under the curve of 0.88 (sensitivity 92.3%, specificity 82.9%).

CONCLUSIONS

The implementation of quantitative lesion water uptake as imaging biomarker in the diagnosis of patients with ischemic stroke presenting in the extended time window might improve clinical prognosis. Future studies could test this biomarker as complementary or even alternative tool to CT perfusion.

Emerging role of artificial intelligence in stroke imaging

Introduction: The recognition and therapy of patients with stroke is becoming progressively intricate as additional treatment choices become accessible and new associations between disease characteristics and treatment response are incessantly uncovered. Therefore, clinicians must regularly learn new skill, stay up to date with the literature and integrate advances into daily practice. The application of artificial intelligence (AI) to assist clinical decision making could diminish inter-rater variation in routine clinical practice and accelerate the mining of vital data that could expand recognition of patients with stroke, forecast of treatment responses and patient outcomes.Areas covered: In this review, the authors provide an up-to-date review of AI in stroke, analyzing the latest papers on this subject. These have been divided in two main groups: stroke diagnosis and outcome prediction.Expert opinion: The highest value of AI is its capability to merge, select and condense a large amount of clinical and imaging features of a single patient and to associate these with fitted models that have gone through robust assessment and optimization with large cohorts of data to support clinical decision making.

Artificial intelligence in clinical decision support and outcome prediction - applications in stroke

Artificial intelligence (AI) is making a profound impact in healthcare, with the number of AI applications in medicine increasing substantially over the past five years. In acute stroke, it is playing an increasingly important role in clinical decision-making. Contemporary advances have increased the amount of information - both clinical and radiological - which clinicians must consider when managing patients. In the time-critical setting of acute stroke, AI offers the tools to rapidly evaluate and consolidate available information, extracting specific predictions from rich, noisy data. It has been applied to the automatic detection of stroke lesions on imaging and can guide treatment decisions through the prediction of tissue outcomes and long-term functional outcomes. This review examines the current state of AI applications in stroke, exploring their potential to reform stroke care through clinical decision support, as well as the challenges and limitations which must be addressed to facilitate their acceptance and adoption for clinical use.

How to Improve the Management of Acute Ischemic Stroke by Modern Technologies, Artificial Intelligence, and New Treatment Methods

Stroke remains one of the leading causes of death and disability in Europe. The European Stroke Action Plan (ESAP) defines four main targets for the years 2018 to 2030. The COVID-19 pandemic forced the use of innovative technologies and created pressure to improve internet networks. Moreover, 5G internet network will be helpful for the transfer and collecting of extremely big databases. Nowadays, the speed of internet connection is a limiting factor for robotic systems, which can be controlled and commanded potentially from various places in the world. Innovative technologies can be implemented for acute stroke patient management soon. Artificial intelligence (AI) and robotics are used increasingly often without the exception of medicine. Their implementation can be achieved in every level of stroke care. In this article, all steps of stroke health care processes are discussed in terms of how to improve them (including prehospital diagnosis, consultation, transfer of the patient, diagnosis, techniques of the treatment as well as rehabilitation and usage of AI). New ethical problems have also been discovered. Everything must be aligned to the concept of “time is brain”.

Early Prediction of Malignant Cerebellar Edema in Posterior Circulation Stroke Using Quantitative Lesion Water Uptake

BACKGROUND

Malignant cerebellar edema (MCE) is a life-threatening complication of ischemic posterior circulation stroke that requires timely diagnosis and management. Yet, there is no established imaging biomarker that may serve as predictor of MCE. Early edematous water uptake can be determined using quantitative lesion water uptake, but this biomarker has only been applied in anterior circulation strokes.

OBJECTIVE

To test the hypothesis that lesion water uptake in early posterior circulation stroke predicts MCE.

METHODS

A total 179 patients with posterior circulation stroke and multimodal admission CT were included. A total of 35 (19.5%) patients developed MCE defined by using an established 10-point scale in follow-up CT, of which ≥4 points are considered malignant. Posterior circulation net water uptake (pcNWU) was quantified in admission CT based on CT densitometry and compared with posterior circulation Acute Stroke Prognosis Early CT Score (pc-ASPECTS) as predictor of MCE using receiver operating curve (ROC) analysis and logistic regression analysis.

RESULTS

Acute pcNWU within the early ischemic lesion was 24.6% (±8.4) for malignant and 7.2% (±7.4) for nonmalignant infarctions, respectively (P < .0001). Based on ROC analysis, pcNWU above 14.9% identified MCE with high discriminative power (area under the curve: 0.94; 95% CI: 0.89-0.97). Early pcNWU (odds ratio [OR]: 1.28; 95% CI: 1.15-1.42, P < .0001) and pc-ASPECTS (OR: 0.71, 95% CI: 0.53-0.95, P = .02) were associated with MCE, adjusted for age and recanalization status.

CONCLUSION

Quantitative pcNWU in early posterior circulation stroke is an important marker for MCE. Besides pc-ASPECTS, lesion water uptake measurements may further support identifying patients at risk for MCE at an early stage indicating stricter monitoring and consideration for further therapeutic measures.

Alberta Stroke Program Early CT Score Calculation Using the Deep Learning-Based Brain Hemisphere Comparison Algorithm

OBJECTIVES

The Alberta Stroke Program Early Computed Tomography Score (ASPECTS) is a promising tool for the evaluation of stroke expansion to determine suitability for reperfusion therapy. The aim of this study was to validate deep learning-based ASPECTS calculation software that utilizes a three-dimensional fully convolutional network-based brain hemisphere comparison algorithm (3D-BHCA).

MATERIALS AND METHODS

We retrospectively collected head non-contrast computed tomography (CT) data from 71 patients with acute ischemic stroke and 80 non-stroke patients. The results for ASPECTS on CT assessed by 5 stroke neurologists and by the 3D-BHCA model were compared with the ground truth by means of region-based and score-based analyses.

RESULTS

In total, 151 patients and 3020 (151 × 20) ASPECTS regions were investigated. Median time from onset to CT was 195 min in the stroke patients. In region-based analysis, the sensitivity (0.80), specificity (0.97), and accuracy (0.96) of the 3D-BHCA model were superior to those of stroke neurologists. The sensitivity (0.98), specificity (0.92), and accuracy (0.97) of dichotomized ASPECTS > 5 analysis and the intraclass correlation coefficient (0.90) in total score-based analysis of the 3D-BHCA model were superior to those of stroke neurologists overall. When patients with stroke were stratified by onset-to-CT time, the 3D-BHCA model exhibited the highest performance to calculate ASPECTS, even in the earliest time period.

CONCLUSIONS

The automated ASPECTS calculation software we developed using a deep learning-based algorithm was superior or equal to stroke neurologists in performing ASPECTS calculation in patients with acute stroke and non-stroke patients.

e-ASPECTS software improves interobserver agreement and accuracy of interpretation of aspects score

INTRODUCTION

There is increased interest in the use of artificial intelligence-based (AI) software packages in the evaluation of neuroimaging studies for acute ischemic stroke. We studied whether, compared to standard image interpretation without AI, Brainomix e-ASPECTS software improved interobserver agreement and accuracy in detecting ASPECTS regions affected in anterior circulation LVO.

METHODS

We included 60 consecutive patients with anterior circulation LVO who had TICI 3 revascularization within 60 minutes of their baseline CT. A total of 16 readers, including senior neuroradiologists, junior neuroradiologists and vascular neurologists participated. Readers interpreted CT scans on independent workstations and assessed final ASPECTS and evaluated whether each individual ASPECTS region was affected. Two months later, readers again evaluated the CT scans, but with assistance of e-ASPECTS software. We assessed interclass correlation coefficient for total ASPECTS and interobserver agreement with Fleiss' Kappa for each ASPECTS region with and without assistance of the e-ASPECTS. We also assessed accuracy for the readers with and without e-ASPECTS assistance. In our assessment of accuracy, ground truth was the 24 hour CT in this cohort of patients who had prompt and complete revascularization.

RESULTS

Interclass correlation coefficient for total ASPECTS without e-ASPECTS assistance was 0.395, indicating fair agreement compared, to 0.574 with e-ASPECTS assistance, indicating good agreement (P < 0.01). There was significant improvement in inter-rater agreement with e-ASPECTS assistance for each individual region with the exception of M6 and caudate. The e-ASPECTS software had higher accuracy than the overall cohort of readers (with and without e-ASPECTS assistance) for every region except the caudate.

CONCLUSIONS

Use of Brainomix e-ASPECTS software resulted in significant improvements in inter-rater agreement and accuracy of ASPECTS score evaluation in a large group of neuroradiologists and neurologists. e-ASPECTS software was more predictive of final infarct/ASPECTS than the overall group interpreting the CT scans with and without e-ASPECTS assistance.

Improved Reliability of Automated ASPECTS Evaluation Using Iterative Model Reconstruction from Head CT Scans

BACKGROUND AND PURPOSE

Iterative model reconstruction (IMR) has shown to improve computed tomography (CT) image quality compared to hybrid iterative reconstruction (HIR). Alberta Stroke Program Early CT Score (ASPECTS) assessment in early stroke is particularly dependent on high-image quality. Purpose of this study was to investigate the reliability of ASPECTS assessed by humans and software based on HIR and IMR, respectively.

METHODS

Forty-seven consecutive patients with acute anterior circulation large vessel occlusions (LVOs) and successful endovascular thrombectomy were included. ASPECTS was assessed by three neuroradiologists (one attending, two residents) and by automated software in noncontrast axial CT with HIR (iDose4; 5 mm) and IMR (5 and 0.9 mm). Two expert neuroradiologists determined consensus ASPECTS reading using all available image data including MRI. Agreement between four raters (three humans, one software) and consensus were compared using square-weighted kappa (κ).

RESULTS

Human raters achieved moderate to almost perfect agreement (κ = .557-.845) with consensus reading. The attending showed almost perfect agreement for 5 mm HIR (κ_HIR  = .845), while residents had mostly substantial agreements without clear trends across reconstructions. Software had substantial to almost perfect agreement with consensus, increasing with IMR 5 and 0.9 mm slice thickness (κ_HIR  = .751, κ_IMR  = .777, and κ_IMR0.9  = .814). Agreements inversely declined for these reconstructions for the attending (κ_HIR  = .845, κ_IMR  = .763, and κ_IMR0.9  = .681).

CONCLUSIONS

Human and software rating showed good reliability of ASPECTS across different CT reconstructions. Human raters performed best with the reconstruction algorithms they had most experience with (HIR for the attending). Automated software benefits from higher resolution with better contrasts in IMR with 0.9 mm slice thickness.

Artificial intelligence for decision support in acute stroke - current roles and potential

The identification and treatment of patients with stroke is becoming increasingly complex as more treatment options become available and new relationships between disease features and treatment response are continually discovered. Consequently, clinicians must constantly learn new skills (such as clinical evaluations or image interpretation), stay up to date with the literature and incorporate advances into everyday practice. The use of artificial intelligence (AI) to support clinical decision making could reduce inter-rater variation in routine clinical practice and facilitate the extraction of vital information that could improve identification of patients with stroke, prediction of treatment responses and patient outcomes. Such support systems would be ideal for centres that deal with few patients with stroke or for regional hubs, and could assist informed discussions with the patients and their families. Moreover, the use of AI for image processing and interpretation in stroke could provide any clinician with an imaging assessment equivalent to that of an expert. However, any AI-based decision support system should allow for expert clinician interaction to enable identification of errors (for example, in automated image processing). In this Review, we discuss the increasing importance of imaging in stroke management before exploring the potential and pitfalls of AI-assisted treatment decision support in acute stroke.

Region-specific agreement in ASPECTS estimation between neuroradiologists and e-ASPECTS software

BACKGROUND AND PURPOSE

The Alberta Stroke Program Early CT Score (ASPECTS) is a widely used measure of ischemic change on non-contrast CT. Although predictive of long-term outcome, ASPECTS is limited by its modest interobserver agreement. One potential solution to this is the use of machine learning strategies, such as e-ASPECTS, to detect ischemia. Here, we compared e-ASPECTS with manual scoring by experienced neuroradiologists for all 10 individual ASPECTS regions.

MATERIALS AND METHODS

We retrospectively reviewed 178 baseline non-contrast CT scans from patients with acute ischemic stroke undergoing endovascular thrombectomy. All scans were reviewed by two independent neuroradiologists with a third reader arbitrating disagreements for a consensus read. Each ASPECTS region was scored individually. All scans were then evaluated using a machine learning-based software package (e-ASPECTS, Brainomix). Interobserver agreement between readers and the software for each region was calculated with a kappa statistic.

RESULTS

The median ASPECTS was 9 for manual scoring and 8.5 for e-ASPECTS, with an overall agreement of κ=0.248. Regional agreement varied from κ=0.094 (M1) to κ=0.555 (lentiform), with better performance in subcortical regions. When corrected for the low number of infarcts in any given region, prevalence-adjusted bias-adjusted kappa ranged from 0.483 (insula) to 0.888 (M3), with greater agreement for cortical areas. Intraclass correlation coefficients were between 0.09 (M1) and 0.556 (lentiform).

CONCLUSION

Manual scoring and e-ASPECTS had fair agreement in our dataset on a per-region basis. This warrants further investigation using follow-up scans or MRI as the gold standard measure of true ASPECTS.

Validation of an automated ASPECTS method on non-contrast computed tomography scans of acute ischemic stroke patients

Background

The Alberta Stroke Program Early CT Score (ASPECTS) is a systematic method of assessing the extent of early ischemic change on non-contrast computed tomography in patients with acute ischemic stroke. Our objective was to validate an automated ASPECTS scoring method we recently developed on a large data set.

Materials and methods

We retrospectively collected 602 acute ischemic stroke patients’ non-contrast computed tomography scans. Expert ASPECTS readings on non-contrast computed tomography were compared to automated ASPECTS. Statistical analyses on the total ASPECTS, region level ASPECTS, and dichotomized ASPECTS (≤4 vs. >4) score were conducted.

Results

In total, 602 scans were evaluated and 6020 (602 × 10) ASPECTS regions were scored. Median time from stroke onset to computed tomography was 114 min (interquartile range: 73–183 min). Total ASPECTS for the 602 patients generated by the automated method agreed well with expert readings (intraclass correlation coefficient): 0.65 (95% confidence interval (CI): 0.60–0.69). Region level analysis showed that the automated method yielded accuracy of 81.25%, sensitivity of 61.13% (95% CI: 58.4%–63.8%), specificity of 86.56% (95% CI: 85.6%–87.5%), and area under curve of 0.74 (95% CI: 0.73–0.75). For dichotomized ASPECTS (≤4 vs. >4), the automated method demonstrated sensitivity 97.21% (95% CI: 95.4%–98.4%), specificity 57.81% (95% CI: 44.8%–70.1%), accuracy 93.02%, and area under the curve of 0.78 (95% CI: 0.74–0.81). For each individual region (M1–6, lentiform, insula, and caudate), the automated method demonstrated acceptable performance.

Conclusion

The automated system we developed approached the stroke expert in performance when scoring ASPECTS on non-contrast computed tomography scans of acute ischemic stroke patients.