Comparing the outcomes of two independent computed tomography perfusion softwares and their impact on therapeutic decisions in acute ischemic stroke

Automated advanced imaging in acute ischemic stroke. Certainties and uncertainties

The purpose of this is study was to review pearls and pitfalls of advanced imaging, such as computed tomography perfusion and diffusion-weighed imaging and perfusion-weighted imaging in the selection of acute ischemic stroke (AIS) patients suitable for endovascular treatment (EVT) in the late time window (6-24 h from symptom onset). Advanced imaging can quantify infarct core and ischemic penumbra using specific threshold values and provides optimal selection parameters, collectively called target mismatch. More precisely, target mismatch criteria consist of core volume and/or penumbra volume and mismatch ratio (the ratio between total hypoperfusion and core volumes) with precise cut-off values. The parameters of target mismatch are automatically calculated with dedicated software packages that allow a quick and standardized interpretation of advanced imaging. However, this approach has several limitations leading to a misclassification of core and penumbra volumes. In fact, automatic software platforms are affected by technical artifacts and are not interchangeable due to a remarkable vendor-dependent variability, resulting in different estimate of target mismatch parameters. In addition, advanced imaging is not completely accurate in detecting infarct core, that can be under- or overestimated. Finally, the selection of candidates for EVT remains currently suboptimal due to the high rates of futile reperfusion and overselection caused by the use of very stringent inclusion criteria. For these reasons, some investigators recently proposed to replace advanced with conventional imaging in the selection for EVT, after the demonstration that non-contrast CT ASPECTS and computed tomography angiography collateral evaluation are not inferior to advanced images in predicting outcome in AIS patients treated with EVT. However, other authors confirmed that CTP and PWI/DWI postprocessed images are superior to conventional imaging in establishing the eligibility of patients for EVT. Therefore, the routine application of automatic assessment of advanced imaging remains a matter of debate. Recent findings suggest that the combination of conventional and advanced imaging might improving our selection criteria.

FDA-approved machine learning algorithms in neuroradiology: A systematic review of the current evidence for approval

Over the past decade, machine learning (ML) and artificial intelligence (AI) have become increasingly prevalent in the medical field. In the United States, the Food and Drug Administration (FDA) is responsible for regulating AI algorithms as "medical devices" to ensure patient safety. However, recent work has shown that the FDA approval process may be deficient. In this study, we evaluate the evidence supporting FDA-approved neuroalgorithms, the subset of machine learning algorithms with applications in the central nervous system (CNS), through a systematic review of the primary literature. Articles covering the 53 FDA-approved algorithms with applications in the CNS published in PubMed, EMBASE, Google Scholar and Scopus between database inception and January 25, 2022 were queried. Initial searches identified 1505 studies, of which 92 articles met the criteria for extraction and inclusion. Studies were identified for 26 of the 53 neuroalgorithms, of which 10 algorithms had only a single peer-reviewed publication. Performance metrics were available for 15 algorithms, external validation studies were available for 24 algorithms, and studies exploring the use of algorithms in clinical practice were available for 7 algorithms. Papers studying the clinical utility of these algorithms focused on three domains: workflow efficiency, cost savings, and clinical outcomes. Our analysis suggests that there is a meaningful gap between the FDA approval of machine learning algorithms and their clinical utilization. There appears to be room for process improvement by implementation of the following recommendations: the provision of compelling evidence that algorithms perform as intended, mandating minimum sample sizes, reporting of a predefined set of performance metrics for all algorithms and clinical application of algorithms prior to widespread use. This work will serve as a baseline for future research into the ideal regulatory framework for AI applications worldwide.

Comparative analysis of core and perfusion lesion volumes between commercially available computed tomography perfusion software

Introduction:

Computed tomography perfusion (CTP) imaging has become an important tool in evaluating acute recanalization treatment candidates. Large clinical trials have successfully used RAPID automated imaging analysis software for quantifying ischemic core and penumbra, yet other commercially available software vendors are also on the market. We evaluated the possible difference in ischemic core and perfusion lesion volumes and the agreement rate of target mismatch between OLEA, MIStar, and Syngo.Via versus RAPID software in acute recanalization treatment candidates.

Patients and methods:

All consecutive stroke-code patients with baseline CTP RAPID imaging at Helsinki University Hospital during 8/2018–9/2021 were included. Ischemic core was defined as cerebral blood flow <30% than the contralateral hemisphere and within the area of delay time (DT) >3s with MIStar. Perfusion lesion volume was defined as DT > 3 s (MIStar) and Tmax > 6 s with all other software. A perfusion mismatch ratio of ⩾1.8, a perfusion lesion volume of ⩾15 mL, and ischemic core <70 mL was defined as target mismatch. The mean pairwise differences of the core and perfusion lesion volumes between software were calculated using the Bland-Altman method and the agreement of target mismatch between software using the Pearson correlation.

Results:

A total of 1606 patients had RAPID perfusion maps, 1222 of which had MIStar, 596 patients had OLEA, and 349 patients had Syngo.Via perfusion maps available. Each software was compared with simultaneously analyzed RAPID software. MIStar showed the smallest core difference compared with RAPID (−2 mL, confidence interval (CI) from −26 to 22), followed by OLEA (2 mL, CI from −33 to 38). Perfusion lesion volume differed least with MIStar (4 mL, CI from −62 to 71) in comparison with RAPID, followed by Syngo.Via (6 mL, CI from −94 to 106). MIStar had the best agreement rate with target mismatch of RAPID followed by OLEA and Syngo.Via.

Discussion and conclusion:

Comparison of RAPID with three other automated imaging analysis software showed variance in ischemic core and perfusion lesion volumes and in target mismatch.

Agreement of three CT perfusion software packages in patients with acute ischemic stroke: A comparison with RAPID

PURPOSE

To compare ischemic core volume (ICV) and penumbra volume (PV) measured by MIStar, F-STROKE, and Syngo.via with that measured by RAPID in acute ischemic stroke (AIS), and their concordance in selecting patients for endovascular thrombectomy (EVT).

METHODS

Computed tomography perfusion (CTP) data were processed with four software packages. Bland-Altman analysis and intraclass correlation coefficient (ICC) were performed to evaluate their agreement in quantifying ICV and PV. Kappa test was conducted to assess consistency in the selection of EVT candidates. The correlation between predicted ICV and segmented final infarct volume (FIV) on follow-up images was investigated.

RESULTS

A total of 91 patients were retrospectively included. F-STROKE had the best consistency with RAPID (ICV: ICC = 0.97; PV: ICC = 0.84) and Syngo.via had the worst consistency (ICV: ICC = 0.77; PV: ICC = 0.66). F-STROKE had the narrowest limits of agreements both in ICV (-27.02, 24.40 mL) and PV (-85.59, 101.80 mL). When selecting EVT candidates, MIStar (kappa = 0.71-0.88) and F-STROKE (kappa = 0.84-0.90) had good to excellent consistency with RAPID, while Syngo.via had poor consistency (kappa = 0.20-0.41). ICV predicted by MIStar was correlated strongest with FIV (r = 0.77).

CONCLUSIONS

F-STROKE is most consistent with RAPID in quantitative ICV and PV. F-STROKE and MIStar exhibit similar EVT candidate selection to RAPID. Syngo.via, for its part, seems to have overestimated ICV and underestimated PV, leading to an overly restrictive selection of EVT candidates.

Comparison of automated ASPECTS, large vessel occlusion detection and CTP analysis provided by Brainomix and RapidAI in patients with suspected ischaemic stroke

OBJECTIVES

The ischaemic core and penumbra volumes derived from CTP aid the selection of patients with an arterial occlusion for mechanical thrombectomy. Different post-processing software packages may give different CTP outputs, potentially causing variable patient selection for mechanical thrombectomy. The study aims were, firstly, to assess the correlation in CTP outputs from software packages provided by Brainomix and RapidAI. Secondly, the correlation between automated ASPECTS and neuroradiologist-derived ASPECTS and accuracy in detecting large vessel occlusion was assessed.

MATERIALS AND METHODS

This retrospective study included patients undergoing CTP for suspected anterior circulation large vessel occlusion. Pearson's correlation coefficient was used for testing the correlation in CTP outputs, ASPECTS/automated ASPECTS, and-in those with complete or near complete occlusion-final infarct volume. Diagnostic statistics were calculated for large vessel occlusion detection.

RESULTS

Correlation was high for ischaemic core and penumbra volumes (0.862 and 0.832, respectively) but lower for the mismatch ratio (0.477). Agreement in mechanical thrombectomy eligibility was achieved in 85% of cases (46/54). Correlation between ischaemic core and final infarct volume was higher for Brainomix (0.757) than for RapidAI (0.595). The correlation between ASPECTS and automated ASPECTS (0.738 and 0.659) and the accuracy of detecting large vessel occlusion (77% and 71%) was higher for Brainomix than for RapidAI.

CONCLUSION

There was high correlation between the CTP output from Brainomix and RapidAI. However, there was a difference in MT eligibility in 15% of cases, which highlights that the decision regarding MT should not be based on imaging parameters alone.

Comparison of Two Software Packages for Perfusion Imaging: Ischemic Core and Penumbra Estimation and Patient Triage in Acute Ischemic Stroke

Purpose: Automated postprocessing packages have been developed for managing acute ischemic stroke (AIS). These packages identify ischemic core and penumbra using either computed tomographic perfusion imaging (CTP) data or magnetic resonance imaging (MRI) data. Measurements of abnormal tissues and treatment decisions derived from different vendors can vary. The purpose of this study is to investigate the agreement of volumetric and decision-making outcomes derived from two software packages. Methods: A total of 594 AIS patients (174 underwent CTP and 420 underwent MRI) were included. Imaging data were accordingly postprocessed by two software packages: RAPID and RealNow. Volumetric outputs were compared between packages by performing intraclass correlation coefficient (ICC), Wilcoxon paired test and Bland–Altman analysis. Concordance of selecting patients eligible for mechanical thrombectomy (MT) was assessed based on neuroimaging criteria proposed in DEFUSE3. Results: In the group with CTP data, mean ischemic core volume (ICV)/penumbral volume (PV) was 14.9/81.1 mL via RAPID and 12.6/83.2 mL via RealNow. Meanwhile, in the MRI group, mean ICV/PV were 52.4/68.4 mL and 48.9/61.6 mL via RAPID and RealNow, respectively. Reliability, which was measured by ICC of ICV and PV in CTP and MRI groups, ranged from 0.87 to 0.99. The bias remained small between measurements (CTP ICV: 0.89 mL, CTP PV: −2 mL, MRI ICV: 3.5 mL and MRI PV: 6.8 mL). In comparison with CTP ICV with follow-up DWI, the ICC was 0.92 and 0.94 for RAPID and Realnow, respectively. The bias remained small between CTP ICV and follow-up DWI measurements (Rapid: −4.65 mL, RealNow: −3.65 mL). Wilcoxon paired test showed no significant difference between measurements. The results of patient triage were concordant in 159/174 cases (91%, ICC: 0.90) for CTP and 400/420 cases (95%, ICC: 0.93) for MRI. Conclusion: The CTP ICV derived from RealNow was more accurate than RAPID. The similarity in volumetric measurement between packages did not necessarily relate to equivalent patient triage. In this study, RealNow showed excellent agreement with RAPID in measuring ICV and PV as well as patient triage.

Comparison of Two Automated Computed Tomography Perfusion Applications to Predict the Final Infarct Volume After Thrombolysis in Cerebral Infarction 3 Recanalization

BACKGROUND AND PURPOSE

Several automated computed tomography perfusion software applications have been developed to provide support in the definition of ischemic core and penumbra in acute ischemic stroke. However, the degree of interchangeability between software packages is not yet clear. Our study aimed to evaluate 2 commonly used automated perfusion software applications (Syngo.via and RAPID) for the indication of ischemic core with respect to the follow-up infarct volume (FIV) after successful recanalization and with consideration of the clinical impact.

METHODS

Retrospectively, 154 patients with large vessel occlusion of the middle cerebral artery or the internal carotid artery, who underwent endovascular therapy with a consequent Thrombolysis in Cerebral Infarction 3 result within 2 hours after computed tomography perfusion, were included. Computed tomography perfusion core volumes were assessed with both software applications with different thresholds for relative cerebral blood flow (rCBF). The results were compared with the FIV on computed tomography within 24 to 36 hours after recanalization. Bland-Altman was applied to display the levels of agreement and to evaluate systematic differences.

RESULTS

Highest correlation between ischemic core volume and FIV without significant differences was found at a threshold of rCBF<38% for the RAPID software (r=0.89, P<0.001) and rCBF<25% for the Syngo software (r=0.87, P<0.001). Bland-Altman analysis revealed best agreement in these settings. In the vendor default settings (rCBF<30% for RAPID and rCBF<20% for Syngo) correlation between ischemic core volume and FIV was also high (RAPID: r=0.88, Syngo: r=0.86, P<0.001), but mean differences were significant (P<0.001). The risk of critical overestimation of the FIV was higher with rCBF<38% (RAPID) and rCBF<25% (Syngo) than in the default settings.

CONCLUSIONS

By adjusting the rCBF thresholds, comparable results with reliable information on the FIV after complete recanalization can be obtained both with the RAPID and Syngo software. Keeping the software specific default settings means being more inclusive in patient selection, but forgo the highest possible accuracy in the estimation of the FIV.

Computed Tomography Perfusion-Based Prediction of Core Infarct and Tissue at Risk: Can Artificial Intelligence Help Reduce Radiation Exposure?

BACKGROUND AND PURPOSE

We explored the feasibility of automated, arterial input function independent, vendor neutral prediction of core infarct, and penumbral tissue using complete and partial computed tomographic perfusion data sets through neural networks.

METHODS

Using retrospective computed tomographic perfusion data from 57 patients, split as training/validation (60%/40%), we developed and validated separate 2-dimensional U-net models for cerebral blood flow (CBF) and time to maximum (Tmax) maps calculation to predict core infarct and tissue at risk, respectively. Once trained, the full sets of 28 input images were sequentially reduced to equitemporal 14, 10, and 7 time points. The averaged structural similarity index measure between the model-derived images and ground truth perfusion maps was compared. Volumes for core infarct and Tmax were compared using the Pearson correlation coefficient.

RESULTS

Both CBF and Tmax maps derived using 28 and 14 time points had similar structural similarity index measure (0.80-0.81; P>0.05) when compared with ground truth images. The Pearson correlation for the CBF and Tmax volumes derived from the model using 28-tp with ground truth volumes derived from the RAPID software was 0.69 for CBF and 0.74 for Tmax. The predicted maps were fully concordant in terms of laterality to the commercial perfusion maps. The mean Dice scores were 0.54 for the core infarct and 0.63 for the hypoperfusion maps.

CONCLUSIONS

Artificial intelligence model-derived volumes show good correlation with RAPID-derived volumes for CBF and Tmax. Within the constraints of a small sample size, the perfusion map quality is similar when using 14-tp instead of 28-tp. Our findings provide proof of concept that vendor neutral artificial intelligence models for computed tomographic perfusion processing using complete or partial image data sets appear feasible. The model accuracy could be further optimized using larger data sets.

Positive predictive value and stroke workflow outcomes using automated vessel density (RAPID-CTA) in stroke patients: One year experience

PURPOSE

Several new techniques have emerged for detecting anterior circulation large vessel occlusion by quantifying relative vessel density including RAPID-CTA, potentially allowing for faster triage and decreased time to mechanical thrombectomy. We present our one-year experience on positive predictive value of RAPID-CTA for the detection of large vessel occlusion in patients presenting with stroke symptoms and its effect on treatment time and clinical outcomes.

MATERIALS AND METHODS

Three hundred and ten patients presenting with stroke symptoms with relative vessel density <60% on RAPID-CTA were included (average age 70 years, 145 male, 165 female). Examinations were considered positive if there was evidence of large vessel occlusion or high grade stenosis. Computed tomography angiography to groin puncture time was calculated during one-year time intervals before and after RAPID-CTA installation. Ninety-day Modified Rankin Scale scores were obtained for patients in each cohort.

RESULTS

Of the 310 patients, 270 had large vessel occlusion or high grade stenosis (87% positive predictive value), with 161 having large vessel occlusion. Using 45% relative vessel density threshold, 129/161 large vessel occlusion were detected (80% sensitivity) and 163/172 examinations were positive (95% positive predictive value). Computed tomography angiography to groin puncture time was significantly lower after deployment of RAPID-CTA (93 min vs 68 min, p<0.05). Average 90 day modified Rankin Scale score was lower in the RAPID-CTA group with a higher percentage of patients with functional independence, although the data was not statistically significant.

CONCLUSION

RAPID-CTA had high positive predictive value for large vessel occlusion with a 45% relative vessel density threshold, which could facilitate active worklist reprioritization. Time to treatment was significantly lower and clinical outcomes were improved after deployment of RAPID-CTA.

Hypoperfusion Index Ratio as a Surrogate of Collateral Scoring on CT Angiogram in Large Vessel Stroke

Background: This study was to evaluate the correlation of the hypoperfusion intensity ratio (HIR) with the collateral score from multiphase computed tomography angiography (mCTA) among patients with large vessel stroke. Method: From February 2019 to May 2020, we retrospectively reviewed the patients with large vessel strokes (intracranial carotid artery or proximal middle cerebral artery occlusion). HIR was defined as a Tmax > 10 s lesion volume divided by a Tmax > 6 s lesion volume, which was calculated by automatic software (Syngo.via, Siemens). The correlation between the HIR and mCTA score was evaluated by Pearson’s correlation. The cutoff value predicting the mCTA score was evaluated by receiver operating characteristic analysis. Result: Ninety-four patients were enrolled in the final analysis. The patients with good collaterals had a smaller core volume (37.3 ± 24.7 vs. 116.5 ± 70 mL, p < 0.001) and lower HIR (0.51 ± 0.2 vs. 0.73 ± 0.13, p < 0.001) than those with poor collaterals. A higher HIR was correlated with a poorer collateral score by Pearson’s correlation. (r = −0.64, p < 0.001). The receiver operating characteristic (ROC) analysis suggested that the best HIR value for predicting a good collateral score was 0.68 (area under curve: 0.82). Conclusion: HIR is a good surrogate of collateral circulation in patients with acute large artery occlusion.