Prediction models for clinical outcome after a carotid revascularisation procedure: A systematic review

Predicting Major Adverse Cardiovascular Events Following Carotid Endarterectomy Using Machine Learning

Background Carotid endarterectomy (CEA) is a major vascular operation for stroke prevention that carries significant perioperative risks; however, outcome prediction tools remain limited. The authors developed machine learning algorithms to predict outcomes following CEA. Methods and Results The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent CEA between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse cardiovascular events (composite of stroke, myocardial infarction, or death). The data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary metric for evaluating model performance was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Overall, 38 853 patients underwent CEA during the study period. Thirty-day major adverse cardiovascular events occurred in 1683 (4.3%) patients. The best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve of 0.91 (95% CI, 0.90-0.92). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.62 (95% CI, 0.60-0.64), and existing tools in the literature demonstrate area under the receiver operating characteristic curve values ranging from 0.58 to 0.74. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The strongest predictive feature in our algorithm was carotid symptom status. Conclusions The machine learning models accurately predicted 30-day outcomes following CEA using preoperative data and performed better than existing tools. They have potential for important utility in guiding risk-mitigation strategies to improve outcomes for patients being considered for CEA.

Using machine learning to predict outcomes following carotid endarterectomy

OBJECTIVE

Prediction of outcomes following carotid endarterectomy (CEA) remains challenging, with a lack of standardized tools to guide perioperative management. We used machine learning (ML) to develop automated algorithms that predict outcomes following CEA.

METHODS

The Vascular Quality Initiative (VQI) database was used to identify patients who underwent CEA between 2003 and 2022. We identified 71 potential predictor variables (features) from the index hospitalization (43 preoperative [demographic/clinical], 21 intraoperative [procedural], and 7 postoperative [in-hospital complications]). The primary outcome was stroke or death at 1 year following CEA. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, insurance status, symptom status, and urgency of surgery.

RESULTS

Overall, 166,369 patients underwent CEA during the study period. In total, 7749 patients (4.7%) had the primary outcome of stroke or death at 1 year. Patients with an outcome were older with more comorbidities, had poorer functional status, and demonstrated higher risk anatomic features. They were also more likely to undergo intraoperative surgical re-exploration and have in-hospital complications. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.90 (95% confidence interval [CI], 0.89-0.91). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. Our XGBoost models maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.90 (95% CI, 0.89-0.91) and 0.94 (95% CI, 0.93-0.95), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.15 (preoperative), 0.14 (intraoperative), and 0.11 (postoperative). Of the top 10 predictors, eight were preoperative features, including comorbidities, functional status, and previous procedures. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

We developed ML models that accurately predict outcomes following CEA. Our algorithms perform better than logistic regression and existing tools, and therefore, have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes.

Characteristics of peripheral blood cells are independently related to major adverse cardiovascular events after carotid endarterectomy

Background and aims

Patients who underwent carotid endarterectomy (CEA) still have a residual risk of 13% of developing a major adverse cardiovascular event (MACE) within 3 years. Inflammatory processes leading up to MACE are not fully understood. Therefore, we examined blood cell characteristics (BCCs), possibly reflecting inflammatory processes, in relation to MACE to identify BCCs that may contribute to an increased risk.

Methods

We analyzed 75 pretreatment BCCs from the Sapphire analyzer, and clinical data from the Athero-Express biobank in relation to MACE after CEA using Random Survival Forests, and a Generalized Additive Survival Model. To understand biological mechanisms, we related the identified variables to intraplaque hemorrhage (IPH).

Results

Of 783 patients, 97 (12%) developed MACE within 3 years after CEA. Red blood cell distribution width (RDW) (HR 1.23 [1.02, 1.68], p = 0.022), CV of lymphocyte size (LACV) (HR 0.78 [0.63, 0.99], p = 0.043), neutrophil complexity of the intracellular structure (NIMN) (HR 0.80 [0.64, 0.98], p = 0.033), mean neutrophil size (NAMN) (HR 0.67 [0.55, 0.83], p < 0.001), mean corpuscular volume (MCV) (HR 1.35 [1.09, 1.66], p = 0.005), eGFR (HR 0.65 [0.52, 0.80], p < 0.001); and HDL-cholesterol (HR 0.62 [0.45, 0.85], p = 0.003) were related to MACE. NAMN was related to IPH (OR 0.83 [0.71-0.98], p = 0.02).

Conclusions

This is the first study to present a higher RDW and MCV and lower LACV, NIMN and NAMN as biomarkers reflecting inflammatory processes that may contribute to an increased risk of MACE after CEA.

Intraplaque neovascularisation is associated with ischaemic events after carotid artery stenting: an observational prospective study

Background

Intraplaque neovascularisation (IPN) is a component of vulnerable atherosclerotic plaque, which is a biomarker of cardiovascular events. However, the identification of patients with high probability of ischaemic events after carotid artery stenting (CAS) is mainly based on vascular risk factors. Prospective studies on the development of plaques are lacking.

Objectives

The purpose of this study was to investigate whether IPN detected by contrast-enhanced ultrasound is related to the occurrence of ischaemic events after CAS.

Methods

Sixty consecutive patients receiving CAS were prospectively enrolled in our centre. The patients were evaluated using contrast-enhanced ultrasound before CAS. According to the degree of microbubble enhancement, IPN was graded from 0 to 2. Endpoint events, including ischaemic stroke and other cardiovascular events, were recorded during follow-up. Kaplan-Meier survival curves and Cox proportional-hazards models were used to evaluate the risk factors for endpoint events. At a median follow-up of 30 months, 13 patients (28.9%) experienced endpoint events. Kaplan-Meier survival curves showed that patients with grade 2 IPN had a higher risk of future ischaemic events than those with grade 0 or 1 IPN (p < 0.05). Cox proportional-hazards models showed that grade 2 IPN [adjusted hazard ratio (HR), 4.049; 95% confidence interval (CI), 1.078-15.202] was a significant predictor of endpoint events (p < 0.05).

Conclusion

Grade 2 IPN evaluated by contrast-enhanced ultrasound has predictive value for ischaemic events in patients after CAS and may help clinicians identify high-risk patients who need close follow-up.

Plain Language Summary

Neovascularisation and carotid artery stenting Introduction: Introduction: It is unclear whether intraplaque neovascularisation (IPN) can be used as an biomarker of high probability ischemic events after carotid artery stenting (CAS).Materials and methods: We enrolled 60 patients who underwent CAS, all of whom underwent CEUS before CAS. We recorded ischaemic events during follow-up. Cox proportional-hazards models were used to evaluate the risk factors for ischaemic events.Results: We found that grade 2 IPN was an independent predictor (hazard ratio, 4.049; 95% confidence interval, 1.078-15.202; p < 0.05) of ischaemic events in patients after CAS.Conclusion: This may help clinicians identify high-risk patients who need close follow-up.

Guideline on carotid surgery for stroke prevention: updates from the Italian Society of Vascular and Endovascular Surgery. A trend towards personalized medicine

BACKGROUND

This guideline (GL) on carotid surgery as updating of "Stroke: Italian guidelines for Prevention and Treatment" of the ISO-SPREAD Italian Stroke Organization-Group, has recently been published in the National Guideline System and shared with the Italian Society of Vascular and Endovascular Surgery (SICVE) and other Scientific Societies and Patient's Association.

METHODS

GRADE-SIGN version, AGREE quality of reporting checklist. Clinical questions formulated according to the PICO model. Recommendations developed based on clinical questions by a multidisciplinary experts' panel and patients' representatives. Systematic reviews performed for each PICO question. Considered judgements filled by assessing the evidence level, direction, and strength of the recommendations.

RESULTS

The panel provided indications and recommendations for appropriate, comprehensive, and individualized management of patients with carotid stenosis. Diagnostic and therapeutic processes of the best medical therapy, carotid endarterectomy (CEA), carotid stenting (CAS) according to the evidences and the judged opinions were included. Symptomatic carotid stenosis in elective and emergency, asymptomatic carotid stenosis, association with ischemic heart disease, preoperative diagnostics, types of anesthesia, monitoring in case of CEA, CEA techniques, comparison between CEA and CAS, post-surgical carotid restenosis, and medical therapy are the main topics, even with analysis of uncertainty areas for risk-benefit assessments in the individual patient (personalized medicine [PM]).

CONCLUSIONS

This GL updates on the main recommendations for the most appropriate diagnostic and medical-surgical management of patients with atherosclerotic carotid artery stenosis to prevent ischemic stroke. This GL also provides useful elements for the application of PM in good clinical practice.

Ceramides and phospholipids in plasma extracellular vesicles are associated with high risk of major cardiovascular events after carotid endarterectomy

Ceramides and phosphatidylcholines (PCs) are bioactive lipids and lipid bilayer membrane components. Distinct ceramides/PCs (ratios) predict cardiovascular outcome in patients with coronary artery disease. Extracellular vesicles (EVs) are proposed biomarkers for cardiovascular disease and contain ceramides/PCs. Ceramides/PCs have not been studied in patients undergoing carotid endarterectomy (CEA) nor in EVs. We therefore investigated whether levels of ceramides/PCs in plasma and EVs are associated with postoperative risk of major adverse cardiovascular events (MACE) following CEA. In 873 patients undergoing CEA of the Athero-Express biobank, we quantitatively measured seven ceramides/PCs in preoperative blood samples: Cer(d18:1/16:0), Cer(d18:1/18:0), Cer(d18:1/24:0), Cer(d18:1/24:1), PC(14:0/22:6), PC(16:0/16:0) and PC(16:0/22:5) in plasma and two plasma EV-subfractions (LDL and TEX). We analyzed the association of ceramides, PCs and their predefined ratios with the three-year postoperative risk of MACE (including stroke, myocardial infarction and cardiovascular death). A total of 138 patients (16%) developed MACE during the three-year follow-up. In the LDL-EV subfraction, higher levels of Cer(d18:1/24:1) and Cer(d18:1/16:0)/PC(16:0/22:5) ratio were significantly associated with an increased risk of MACE (adjusted HR per SD [95% CI] 1.24 [1.01–1.53] and 1.26 [1.04–1.52], respectively). In the TEX-EV subfraction, three ratios Cer(d18:1/16:0)/Cer(d18:1/24:0), Cer(d18:1/18:0)/Cer(d18:1/24:0) and Cer(d18:1/24:1)/Cer(d18:1/24:0) were positively associated with MACE (adjusted HR per SD 1.34 [1.06–1.70], 1.24 [1.01–1.51] and 1.31 [1.08–1.58], respectively). In conclusion, distinct ceramides and PCs in plasma EVs determined in preoperative blood were independently associated with an increased 3-year risk of MACE after CEA. These lipids are therefore potential markers to identify high-risk CEA patients qualifying for secondary preventive add-on therapy.

External Validation of Risk Prediction Models to Improve Selection of Patients for Carotid Endarterectomy

BACKGROUND AND PURPOSE

The net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if 30-day risks are <6% for symptomatic stenosis and <3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA and to externally validate these models in a large registry of patients from the United States.

METHODS

We conducted a systematic search in MEDLINE and EMBASE for prediction models of procedural outcomes after CEA. We validated these models with data from patients who underwent CEA in the American College of Surgeons National Surgical Quality Improvement Program (2011-2017). We assessed discrimination using C statistics and calibration graphically. We determined the number of patients with predicted risks that exceeded recommended thresholds of procedural risks to perform CEA.

RESULTS

After screening 788 reports, 15 studies describing 17 prediction models were included. Nine were developed in populations including both asymptomatic and symptomatic patients, 2 in symptomatic and 5 in asymptomatic populations. In the external validation cohort of 26 293 patients who underwent CEA, 702 (2.7%) developed a stroke or died within 30-days. C statistics varied between 0.52 and 0.64 using all patients, between 0.51 and 0.59 using symptomatic patients, and between 0.49 to 0.58 using asymptomatic patients. The Ontario Carotid Endarterectomy Registry model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors, had C statistic of 0.64 and the best concordance between predicted and observed risks. This model identified 4.5% of symptomatic and 2.1% of asymptomatic patients with procedural risks that exceeded recommended thresholds.

CONCLUSIONS

Of the 17 externally validated prediction models, the Ontario Carotid Endarterectomy Registry risk model had most reliable predictions of procedural stroke or death after CEA and can inform patients about procedural hazards and help focus CEA toward patients who would benefit most from it.

Pre-Operative Plasma Extracellular Vesicle Proteins are Associated with a High Risk of Long Term Secondary Major Cardiovascular Events in Patients Undergoing Carotid Endarterectomy

OBJECTIVE

Patients undergoing carotid endarterectomy (CEA) maintain a substantial residual risk of major cardiovascular events (MACE). Improved risk stratification is warranted to select high risk patients qualifying for secondary add on therapy. Plasma extracellular vesicles (EVs) are involved in atherothrombotic processes and their content has been related to the presence and recurrence of cardiovascular events. The association between pre-operative levels of five cardiovascular disease related proteins in plasma EVs and the post-operative risk of MACE was assessed.

METHODS

In 864 patients undergoing CEA from 2002 to 2016 included in the Athero-Express biobank, three plasma EV subfractions (low density lipoprotein [LDL], high density lipoprotein [HDL], and tiny extracellular vesicles [TEX]) were isolated from pre-operative blood samples. Using an electrochemiluminescence immunoassay, five proteins were quantified in each EV subfraction: cystatin C, serpin C1, serpin G1, serpin F2, and CD14. The association between EV protein levels and the three year post-operative risk of MACE (any stroke, myocardial infarction, or cardiovascular death) was evaluated using multivariable Cox proportional hazard regression analyses.

RESULTS

During a median follow up of three years (interquartile range 2.2 - 3.0), 137 (16%) patients developed MACE. In the HDL-EV subfraction, increased levels of CD14, cystatin C, serpin F2, and serpin C1 were associated with an increased risk of MACE (adjusted hazard ratios per one standard deviation increase of 1.30, 95% confidence interval [CI] 1.15-1.48; 1.22, 95% CI 1.06-1.42; 1.36, 95% CI 1.16-1.61; and 1.29, 95% CI 1.10-1.51; respectively), independently of cardiovascular risk factors. No significant associations were found for serpin G1. CD14 improved the predictive value of the clinical model encompassing cardiovascular risk factors (net re-classification index = 0.16, 95% CI 0.08-0.21).

CONCLUSION

EV derived pre-operative plasma levels of cystatin C, serpin C1, CD14, and serpin F2 were independently associated with an increased long term risk of MACE after CEA and are thus markers for residual cardiovascular risk. EV derived CD14 levels could improve the identification of high risk patients who may benefit from secondary preventive add on therapy in order to reduce future risk of MACE.

Comparison of Long Term Prognosis between Carotid Endarterectomy versus Stenting; A Korean Population-Based Study Using National Insurance Data

Purpose

Although carotid endarterectomy (CEA) is recommended as a treatment for carotid stenosis rather than carotid artery stenting (CAS), CAS has been preferred in Korea. The aim of this study was to analyze long-term outcomes after CAS compared with CEA using Korean nationwide insurance data.Materials and

Methods

We obtained all data from the nationwide database of the Health Insurance Review & Assessment Service (HIRA) during the study period using several codes regarding the procedure or operation. We included the HIRA data, which included at least one-year follow-up after the procedures. The outcomes associated with both procedures were death, recurrence of ischemic stroke, and admission for cerebral hemorrhage.

Results

A total of 16,065 eligible patients who were treated with CAS or CEA between 1 January 2007 and 31 December 2016 were analyzed. The number of patients with CAS and CEA was 12,173 (75.8%) and 3,892 (24.2%), respectively. 8,976 patients (55.9%) were classified as symptomatic patients. CAS was associated with a higher risk of all-cause mortality (adjusted hazard ratio [HR], 1.282; 95% confidence interval [CI], 1.173–1.400). The adjusted rates for recurrent ischemic stroke and cerebral hemorrhage between CAS versus CEA were 24.9% versus 15.9% (HR, 1.474; 95% CI, 1.325–1.639) and 1.5% versus 0.9% (HR, 2.026; 95% CI, 1.322–3.106), respectively. In young symptomatic patients, there was no statistically significant difference in all-cause mortality and cardiovascular death between CAS and CEA.

Conclusion

Our study using Korean nationwide insurance data demonstrated similar results to previous studies. Until further evidence of CAS is established through prospective studies, CAS should be performed in selected patients according to current guidelines.

Prediction Models for Clinical Outcome After a Carotid Revascularization Procedure

Supplemental Digital Content is available in the text.

Background and Purpose—

Prediction models may help physicians to stratify patients with high and low risk for periprocedural complications or long-term stroke risk after carotid artery stenting or carotid endarterectomy. We aimed to evaluate external performance of previously published prediction models for short- and long-term outcome after carotid revascularization in patients with symptomatic carotid artery stenosis.

Methods—

From a literature review, we selected all prediction models that used only readily available patient characteristics known before procedure initiation. Follow-up data from 2184 carotid artery stenting and 2261 carotid endarterectomy patients from 4 randomized trials (EVA-3S [Endarterectomy Versus Angioplasty in Patients With Symptomatic Severe Carotid Stenosis], SPACE [Stent-Protected Angioplasty Versus Carotid Endarterectomy], ICSS [International Carotid Stenting Study], and CREST [Carotid Revascularization Endarterectomy Versus Stenting Trial]) were used to validate 23 short-term outcome models to estimate stroke or death risk ≤30 days after the procedure and the original outcome measure for which the model was developed. Additionally, we validated 7 long-term outcome models for the original outcome measure. Predictive performance of the models was assessed with C statistics and calibration plots.

Results—

Stroke or death ≤30 days after the procedure occurred in 158 (7.2%) patients after carotid artery stenting and in 84 (3.7%) patients after carotid endarterectomy. Most models for short-term outcome after carotid artery stenting (n=4) or carotid endarterectomy (n=19) had poor discriminative performance (C statistics ranging from 0.49–0.64) and poor calibration with small absolute risk differences between the lowest and highest risk groups and overestimation of risk in the highest risk groups. Long-term outcome models (n=7) had a slightly better performance with C statistics ranging from 0.59 to 0.67 and reasonable calibration.

Conclusions—

Current models did not reliably predict outcome after carotid revascularization in a trial population of patients with symptomatic carotid stenosis. In particular, prediction of short-term outcome seemed to be difficult. Further external validation of existing prediction models or development of new prediction models is needed before such models can be used to support treatment decisions in individual patients.