Enhancing the Value of Population-Based Risk Scores for Institutional-Level Use

Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients

Background

The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning-based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models.

Methods

All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures.

Results

A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set.

Conclusions

Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.

Redefining "low risk": Outcomes of surgical aortic valve replacement in low-risk patients in the transcatheter aortic valve replacement era

OBJECTIVES

Guidelines suggest aortic valve replacement (AVR) for low-risk asymptomatic patients. Indications for transcatheter AVR now include low-risk patients, making it imperative to understand state-of-the-art surgical AVR (SAVR) in this population. Therefore, we compared SAVR outcomes in low-risk patients with those expected from Society of Thoracic Surgeons (STS) models and assessed their intermediate-term survival.

METHODS

From January 2005 to January 2017, 3493 isolated SAVRs were performed in 3474 patients with STS predicted risk of mortality <4%. Observed operative mortality and composite major morbidity or mortality were compared with STS-expected outcomes according to calendar year of surgery. Logistic regression analysis was used to identify risk factors for these outcomes. Patients were followed for time-related mortality.

RESULTS

With 15 observed operative deaths (0.43%) compared with 55 expected (1.6%), the observed:expected ratio was 0.27 for mortality (95% confidence interval [CI], 0.14-0.42), stroke 0.65 (95% CI, 0.41-0.89), and reoperation 0.50 (95% CI, 0.42-0.60). Major morbidity or mortality steadily declined, with probabilities of 8.6%, 6.7%, and 5.2% in 2006, 2011, and 2016, respectively, while STS-expected risk remained at approximately 12%. Mitral valve regurgitation, ventricular hypertrophy, pulmonary, renal, and hepatic failure, coronary artery disease, and earlier surgery date were residual risk factors. Survival was 98%, 91%, and 82% at 1, 5, and 9 years, respectively, superior to that predicted for the US age-race-sex-matched population.

CONCLUSIONS

STS risk models overestimate contemporary SAVR risk at a high-volume center, supporting efforts to create a more agile quality assessment program. SAVR in low-risk patients provides durable survival benefit, supporting early surgery and providing a benchmark for transcatheter AVR.

Trends in Coronary Revascularization and Ischemic Heart Disease-Related Mortality in Israel

BACKGROUND

We investigated national trends in volume and outcomes of percutaneous coronary angioplasty (PCI), coronary artery bypass grafting (CABG), and ischemic heart disease-related mortality in Israel.

METHODS AND RESULTS

Using International Classification of Diseases 9th and 10th revision codes, we linked 5 Israeli national databases, including the Israel Center for Disease Control National PCI and CABG Registries, the Ministry of Health Hospitalization Report, the Center of Bureau of Statistics, and the Ministry of Interior Mortality Report, to assess the annual PCI and CABG volume, procedural mortality, comorbidities, and ischemic heart disease-related mortality between 2002 and 2014. Trends over time were analyzed using linear regression, assuming a Poisson distribution. A total of 298 390 revascularization procedures (PCI: 255 724, CABG: 42 666) were performed during the study period. PCI volume increased by 9% from 2002 to 2008 (387.4/100 000 to 423.2/100 000), steadily decreasing by 10.5% to 378.5/100 000 in 2014 (P=0.70 for the trend). CABG volume decreased by 59% (109.0/100 000 to 45.2/100 000) from 2002 to 2013, leveling at 46.4/100 000 (P<0.0001). PCI/CABG ratio increased from 3.6 in 2002 to 8.5 in 2013, slightly decreasing to 8.2 by 2014 (P<0.0001). In-hospital procedural mortality remained stable (PCI: 1.2-1.6%, P=0.34, CABG: 3.7-4.4%, P=0.29) despite a significant change in patient clinical profile. During the course of the study, ischemic heart disease-related mortality decreased by 46% (84.6-46/100 000, P<0.001).

CONCLUSIONS

We observed a dramatic change in coronary revascularization procedures type and volume, and a marked decrease in ischemic heart disease-related mortality in Israel. The reasons for the observed changes remain unclear and need to be further investigated.