Temporal Shift and Predictive Performance of Machine Learning for Heart Transplant Outcomes

Predicting Organ Rejections for Pediatric Heart Transplantations with a Combined Use of Transplant Registry Data and Electronic Health Records

Objective

Pediatric heart transplantation is challenged by limited donor organ availability, prolonged waitlist times, and elevated risks of late acute rejection (LAR) and hospitalization. Current predictive models for post-transplant outcomes lack high accuracy due to reliance on registry data without integrating dynamic clinical and social factors. This study aimed to improve predictive performance and model interpretability by incorporating electronic health records (EHR), social determinants of health (SDoH), and United Network for Organ Sharing (UNOS) data.

Materials and Methods

We used EHR and UNOS data from 111 pediatric heart transplant patients (ages 0-18) at the University of Florida Health Shands Children's Hospital to build predictive models for organ rejection at 1-, 3-, and 5-year intervals post-transplant. UNOS data includes pre- and post-transplant health and medical records, encompassing procedures, clinical evaluations, and post-transplant follow-up information, EHR data included evolving clinical parameters (e.g., comorbidities, medication adherence, and laboratory results), while SDoH encompassed socioeconomic status, living conditions, and healthcare access. Feature importance was assessed using Shapley Variable Importance Cloud (ShapleyVIC), which integrates Shapley Additive Explanations (SHAP) to provide robust, interpretable insights across nearly optimal models.

Results

Models integrating EHR, SDoH, and UNOS data outperformed those using UNOS data alone, with AUROC of 0.743 (0.607-0.879), 0.798 (0.725-0.871), and 0.760 (0.692-0.828). Key predictors of rejection included severe pre-transplant conditions (e.g., life support, prolonged waitlist times), elevated bilirubin and creatinine levels, and social factors (e.g., transportation barriers, BMI, insurance type).

Discussion

Findings reveal the importance of integrating clinical and social data to address multisystem dysfunction, disparities in healthcare access, and adherence challenges. ShapleyVIC enhanced model interpretability, providing actionable insights for improving post-transplant care.

Conclusion

Holistic, data-driven approaches that combine EHR, SDoH, and registry data significantly enhance predictive accuracy and interpretability, supporting improved long-term outcomes for pediatric heart transplant patients.

Mortality prediction of heart transplantation using machine learning models: a systematic review and meta-analysis

Introduction

Machine learning (ML) models have been increasingly applied to predict post-heart transplantation (HT) mortality, aiming to improve decision-making and optimize outcomes. This systematic review and meta-analysis evaluates the performance of ML algorithms in predicting mortality and explores factors contributing to model accuracy.

Method

A systematic search of PubMed, Scopus, Web of Science, and Embase identified relevant studies, with 17 studies included in the review and 12 in the meta-analysis. The algorithms assessed included random forests, CatBoost, neural networks, and others. Model performance was evaluated using pooled area under the curve (AUC) values, with subgroup analyses for algorithm type, validation methods, and prediction timeframes. The risk of bias was assessed using the QUADAS-2 tool.

Results

The pooled AUC of all ML algorithms was 0.65 (95% CI: 0.64, 0.67), with no significant difference between machine learning and deep learning models (p = 0.67). Among the algorithms, CatBoost demonstrated the highest accuracy (AUC 0.80, 95% CI: 0.74, 0.86), while K-nearest neighbor had the lowest accuracy (AUC 0.53, 95% CI: 0.50, 0.55). A meta-regression indicated improved model performance with longer post-transplant periods (p = 0.008). When pooling only the best-performing models, the AUC improved to 0.73 (95% CI: 0.68, 0.78). The risk of bias was high in eight studies, with the flow and timing domains most commonly contributing to bias.

Conclusion

ML models demonstrate moderate accuracy in predicting post-HT mortality, with CatBoost achieving the best performance. While ML shows potential for improving predictive precision, significant heterogeneity and biases highlight the need for standardized methods and further external validations to enhance clinical applicability.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO/view/CRD42024509630, CRD42024509630.

Machine learning enhanced immunologic risk assessments for solid organ transplantation

The purpose of this study was to enhance the prediction of solid-organ recipient and donor crossmatch compatibility by applying machine learning (ML). Prediction of crossmatch compatibility is complex and requires an understanding of the recipient and donor human leukocyte antigen (HLA) alleles and recipient HLA antibodies. An HLA allele imputation system that converts HLA antigens to alleles was developed to enhance the prediction's performance. The imputed and known HLA alleles were combined for recipient and donor with a recipient's HLA antibody profile. After processing, donor-specific antibodies were input into various ML models. Next, an ML model was developed and characterized based on determining donor-specific antibodies using the full HLA antibody profile of the recipient without laboratory interpretation. The models achieved an ROC-AUC of 0.975. These results demonstrate that the models can predict crossmatch reactivity and yield insight into the importance of specific HLA antibodies in the transplant-matching process. These data represent our understanding of personalized histocompatibility risk assessments.

Tailoring Risk Prediction Models to Local Populations

Importance

Risk estimation is an integral part of cardiovascular care. Local recalibration of guideline-recommended models could address the limitations of existing tools.

Objective

To provide a machine learning (ML) approach to augment the performance of the American Heart Association's Predicting Risk of Cardiovascular Disease Events (AHA-PREVENT) equations when applied to a local population while preserving clinical interpretability.

Design, Setting, and Participants

This cohort study used a New England-based electronic health record cohort of patients without prior atherosclerotic cardiovascular disease (ASCVD) who had the data necessary to calculate the AHA-PREVENT 10-year risk of developing ASCVD in the event period (2007-2016). Patients with prior ASCVD events, death prior to 2007, or age 79 years or older in 2007 were subsequently excluded. The final study population of 95 326 patients was split into 3 nonoverlapping subsets for training, testing, and validation. The AHA-PREVENT model was adapted to this local population using the open-source ML model (MLM) Extreme Gradient Boosting model (XGBoost) with minimal predictor variables, including age, sex, and AHA-PREVENT. The MLM was monotonically constrained to preserve known associations between risk factors and ASCVD risk. Along with sex, race and ethnicity data from the electronic health record were collected to validate the performance of ASCVD risk prediction in subgroups. Data were analyzed from August 2021 to February 2024.

Main Outcomes and Measures

Consistent with the AHA-PREVENT model, ASCVD events were defined as the first occurrence of either nonfatal myocardial infarction, coronary artery disease, ischemic stroke, or cardiovascular death. Cardiovascular death was coded via government registries. Discrimination, calibration, and risk reclassification were assessed using the Harrell C index, a modified Hosmer-Lemeshow goodness-of-fit test and calibration curves, and reclassification tables, respectively.

Results

In the test set of 38 137 patients (mean [SD] age, 64.8 [6.9] years, 22 708 [59.5]% women and 15 429 [40.5%] men; 935 [2.5%] Asian, 2153 [5.6%] Black, 1414 [3.7%] Hispanic, 31 400 [82.3%] White, and 2235 [5.9%] other, including American Indian, multiple races, unspecified, and unrecorded, consolidated owing to small numbers), MLM-PREVENT had improved calibration (modified Hosmer-Lemeshow P > .05) compared to the AHA-PREVENT model across risk categories in the overall cohort (χ23 = 2.2; P = .53 vs χ23 > 16.3; P < .001) and sex subgroups (men: χ23 = 2.1; P = .55 vs χ23 > 16.3; P < .001; women: χ23 = 6.5; P = .09 vs. χ23 > 16.3; P < .001), while also surpassing a traditional recalibration approach. MLM-PREVENT maintained or improved AHA-PREVENT's calibration in Asian, Black, and White individuals. Both MLM-PREVENT and AHA-PREVENT performed equally well in discriminating risk (approximate ΔC index, ±0.01). Using a clinically significant 7.5% risk threshold, MLM-PREVENT reclassified a total of 11.5% of patients. We visualize the recalibration through MLM-PREVENT ASCVD risk charts that highlight preserved risk associations of the original AHA-PREVENT model.

Conclusions and Relevance

The interpretable ML approach presented in this article enhanced the accuracy of the AHA-PREVENT model when applied to a local population while still preserving the risk associations found by the original model. This method has the potential to recalibrate other established risk tools and is implementable in electronic health record systems for improved cardiovascular risk assessment.

LightGBM outperforms other machine learning techniques in predicting graft failure after liver transplantation: Creation of a predictive model through large-scale analysis

BACKGROUND

The incidence of graft failure following liver transplantation (LTx) is consistent. While traditional risk scores for LTx have limited accuracy, the potential of machine learning (ML) in this area remains uncertain, despite its promise in other transplant domains. This study aims to determine ML's predictive limitations in LTx by replicating methods used in previous heart transplant research.

METHODS

This study utilized the UNOS STAR database, selecting 64,384 adult patients who underwent LTx between 2010 and 2020. Gradient boosting models (XGBoost and LightGBM) were used to predict 14, 30, and 90-day graft failure compared to conventional logistic regression model. Models were evaluated using both shuffled and rolling cross-validation (CV) methodologies. Model performance was assessed using the AUC across validation iterations.

RESULTS

In a study comparing predictive models for 14-day, 30-day and 90-day graft survival, LightGBM consistently outperformed other models, achieving the highest AUC of.740,.722, and.700 in shuffled CV methods. However, in rolling CV the accuracy of the model declined across every ML algorithm. The analysis revealed influential factors for graft survival prediction across all models, including total bilirubin, medical condition, recipient age, and donor AST, among others. Several features like donor age and recipient diabetes history were important in two out of three models.

CONCLUSIONS

LightGBM enhances short-term graft survival predictions post-LTx. However, due to changing medical practices and selection criteria, continuous model evaluation is essential. Future studies should focus on temporal variations, clinical implications, and ensure model transparency for broader medical utility.

The transformative potential of artificial intelligence in solid organ transplantation

Solid organ transplantation confronts numerous challenges ranging from donor organ shortage to post-transplant complications. Here, we provide an overview of the latest attempts to address some of these challenges using artificial intelligence (AI). We delve into the application of machine learning in pretransplant evaluation, predicting transplant rejection, and post-operative patient outcomes. By providing a comprehensive overview of AI's current impact, this review aims to inform clinicians, researchers, and policy-makers about the transformative power of AI in enhancing solid organ transplantation and facilitating personalized medicine in transplant care.

Developing machine learning models to predict primary graft dysfunction after lung transplantation

Primary graft dysfunction (PGD) is the leading cause of morbidity and mortality in the first 30 days after lung transplantation. Risk factors for the development of PGD include donor and recipient characteristics, but how multiple variables interact to impact the development of PGD and how clinicians should consider these in making decisions about donor acceptance remain unclear. This was a single-center retrospective cohort study to develop and evaluate machine learning pipelines to predict the development of PGD grade 3 within the first 72 hours of transplantation using donor and recipient variables that are known at the time of donor offer acceptance. Among 576 bilateral lung recipients, 173 (30%) developed PGD grade 3. The cohort underwent a 75% to 25% train-test split, and lasso regression was used to identify 11 variables for model development. A K-nearest neighbor's model showing the best calibration and performance with relatively small confidence intervals was selected as the final predictive model with an area under the receiver operating characteristics curve of 0.65. Machine learning models can predict the risk for development of PGD grade 3 based on data available at the time of donor offer acceptance. This may improve donor-recipient matching and donor utilization in the future.

Predicting need for heart failure advanced therapies using an interpretable tropical geometry-based fuzzy neural network

BACKGROUND

Timely referral for advanced therapies (i.e., heart transplantation, left ventricular assist device) is critical for ensuring optimal outcomes for heart failure patients. Using electronic health records, our goal was to use data from a single hospitalization to develop an interpretable clinical decision-making system for predicting the need for advanced therapies at the subsequent hospitalization.

METHODS

Michigan Medicine heart failure patients from 2013-2021 with a left ventricular ejection fraction ≤ 35% and at least two heart failure hospitalizations within one year were used to train an interpretable machine learning model constructed using fuzzy logic and tropical geometry. Clinical knowledge was used to initialize the model. The performance and robustness of the model were evaluated with the mean and standard deviation of the area under the receiver operating curve (AUC), the area under the precision-recall curve (AUPRC), and the F1 score of the ensemble. We inferred membership functions from the model for continuous clinical variables, extracted decision rules, and then evaluated their relative importance.

RESULTS

The model was trained and validated using data from 557 heart failure hospitalizations from 300 patients, of whom 193 received advanced therapies. The mean (standard deviation) of AUC, AUPRC, and F1 scores of the proposed model initialized with clinical knowledge was 0.747 (0.080), 0.642 (0.080), and 0.569 (0.067), respectively, showing superior predictive performance or increased interpretability over other machine learning methods. The model learned critical risk factors predicting the need for advanced therapies in the subsequent hospitalization. Furthermore, our model displayed transparent rule sets composed of these critical concepts to justify the prediction.

CONCLUSION

These results demonstrate the ability to successfully predict the need for advanced heart failure therapies by generating transparent and accessible clinical rules although further research is needed to prospectively validate the risk factors identified by the model.

Prediction of Outcomes After Heart Transplantation in Pediatric Patients Using National Registry Data: Evaluation of Machine Learning Approaches

BACKGROUND

The prediction of posttransplant health outcomes for pediatric heart transplantation is critical for risk stratification and high-quality posttransplant care.

OBJECTIVE

The purpose of this study was to examine the use of machine learning (ML) models to predict rejection and mortality for pediatric heart transplant recipients.

METHODS

Various ML models were used to predict rejection and mortality at 1, 3, and 5 years after transplantation in pediatric heart transplant recipients using United Network for Organ Sharing data from 1987 to 2019. The variables used for predicting posttransplant outcomes included donor and recipient as well as medical and social factors. We evaluated 7 ML models-extreme gradient boosting (XGBoost), logistic regression, support vector machine, random forest (RF), stochastic gradient descent, multilayer perceptron, and adaptive boosting (AdaBoost)-as well as a deep learning model with 2 hidden layers with 100 neurons and a rectified linear unit (ReLU) activation function followed by batch normalization for each and a classification head with a softmax activation function. We used 10-fold cross-validation to evaluate model performance. Shapley additive explanations (SHAP) values were calculated to estimate the importance of each variable for prediction.

RESULTS

RF and AdaBoost models were the best-performing algorithms for different prediction windows across outcomes. RF outperformed other ML algorithms in predicting 5 of the 6 outcomes (area under the receiver operating characteristic curve [AUROC] 0.664 and 0.706 for 1-year and 3-year rejection, respectively, and AUROC 0.697, 0.758, and 0.763 for 1-year, 3-year, and 5-year mortality, respectively). AdaBoost achieved the best performance for prediction of 5-year rejection (AUROC 0.705).

CONCLUSIONS

This study demonstrates the comparative utility of ML approaches for modeling posttransplant health outcomes using registry data. ML approaches can identify unique risk factors and their complex relationship with outcomes, thereby identifying patients considered to be at risk and informing the transplant community about the potential of these innovative approaches to improve pediatric care after heart transplantation. Future studies are required to translate the information derived from prediction models to optimize counseling, clinical care, and decision-making within pediatric organ transplant centers.

A machine learning model for prediction of 30-day primary graft failure after heart transplantation

Background

Primary graft failure (PGF) remains the most common cause of short-term mortality after heart transplantation. The main objective was to develop and validate a risk model for prediction of short-term mortality due to PGF after heart transplantation using the ISHLT Heart Transplant Registry.

Methods

We developed a non-linear artificial neural networks (ANN) model to evaluate the association between recipient-donor variables and post-transplant PGF. Patients in the ISHLT registry were randomly divided into derivation and an independent internal validation cohort. The primary endpoint was PGF defined as death within 30 days due to Graft failure or Cardiovascular causes or retransplant within 30 days for causes other than rejection.

Results

Among 64,964 adult recipients transplanted between 1994 and 2013, mean age was 51 years and 22% were female. The incidence of PGF up to 30 days was 3.7%. The ANN model selected 33 of 77 risk variables as relevant for PGF prediction. The C-index in the test cohort was 0.70 (95% CI: 0.68-0.71). The risk variables which most influenced the PGF were underlying HF diagnosis, ischemia time and sex, while renal function had a lower influence.

Conclusion

An ANN model to predict primary graft dysfunction was derived and independently validated. The good discrimination of the ANN model likely results from its flexibility to model potentially non-linear relationships and interactions. Whether this model with improved discrimination can assist in clinical decisions at the time of transplant should be tested.

JM, Shickel B, Rashidi P, Bihorac A, Ahmed MM, Mardini MT. Artificial intelligence guidance of advanced heart failure therapies: A systematic scoping review

Introduction

Artificial intelligence can recognize complex patterns in large datasets. It is a promising technology to advance heart failure practice, as many decisions rely on expert opinions in the absence of high-quality data-driven evidence.

Methods

We searched Embase, Web of Science, and PubMed databases for articles containing "artificial intelligence," "machine learning," or "deep learning" and any of the phrases "heart transplantation," "ventricular assist device," or "cardiogenic shock" from inception until August 2022. We only included original research addressing post heart transplantation (HTx) or mechanical circulatory support (MCS) clinical care. Review and data extraction were performed in accordance with PRISMA-Scr guidelines.

Results

Of 584 unique publications detected, 31 met the inclusion criteria. The majority focused on outcome prediction post HTx (n = 13) and post durable MCS (n = 7), as well as post HTx and MCS management (n = 7, n = 3, respectively). One study addressed temporary mechanical circulatory support. Most studies advocated for rapid integration of AI into clinical practice, acknowledging potential improvements in management guidance and reliability of outcomes prediction. There was a notable paucity of external data validation and integration of multiple data modalities.

Conclusion

Our review showed mounting innovation in AI application in management of MCS and HTx, with the largest evidence showing improved mortality outcome prediction.

Artificial Intelligence in Nuclear Cardiology

Artificial intelligence (AI) encompasses a variety of computer algorithms that have a wide range of potential clinical applications in nuclear cardiology. This article will introduce core terminology and concepts for AI including classifications of AI as well as training and testing regimens. We will then highlight the potential role for AI to improve image registration and image quality. Next, we will discuss methods for AI-driven image attenuation correction. Finally, we will review advancements in machine learning and deep-learning applications for disease diagnosis and risk stratification, including efforts to improve clinical translation of this valuable technology with explainable AI models.

Enhanced survival prediction using explainable artificial intelligence in heart transplantation

The most limiting factor in heart transplantation is the lack of donor organs. With enhanced prediction of outcome, it may be possible to increase the life-years from the organs that become available. Applications of machine learning to tabular data, typical of clinical decision support, pose the practical question of interpretation, which has technical and potential ethical implications. In particular, there is an issue of principle about the predictability of complex data and whether this is inherent in the data or strongly dependent on the choice of machine learning model, leading to the so-called accuracy-interpretability trade-off. We model 1-year mortality in heart transplantation data with a self-explaining neural network, which is benchmarked against a deep learning model on the same development data, in an external validation study with two data sets: (1) UNOS transplants in 2017-2018 (n = 4750) for which the self-explaining and deep learning models are comparable in their AUROC 0.628 [0.602,0.654] cf. 0.635 [0.609,0.662] and (2) Scandinavian transplants during 1997-2018 (n = 2293), showing good calibration with AUROCs of 0.626 [0.588,0.665] and 0.634 [0.570, 0.698], respectively, with and without missing data (n = 982). This shows that for tabular data, predictive models can be transparent and capture important nonlinearities, retaining full predictive performance.

Improving predictive performance in incident heart failure using machine learning and multi-center data

Objective

To mitigate the burden associated with heart failure (HF), primary prevention is of the utmost importance. To improve early risk stratification, advanced computational methods such as machine learning (ML) capturing complex individual patterns in large data might be necessary. Therefore, we compared the predictive performance of incident HF risk models in terms of (a) flexible ML models and linear models and (b) models trained on a single cohort (single-center) and on multiple heterogeneous cohorts (multi-center).

Design and methods

In our analysis, we used the meta-data consisting of 30,354 individuals from 6 cohorts. During a median follow-up of 5.40 years, 1,068 individuals experienced a non-fatal HF event. We evaluated the predictive performance of survival gradient boosting (SGB), CoxNet, the PCP-HF risk score, and a stacking method. Predictions were obtained iteratively, in each iteration one cohort serving as an external test set and either one or all remaining cohorts as a training set (single- or multi-center, respectively).

Results

Overall, multi-center models systematically outperformed single-center models. Further, c-index in the pooled population was higher in SGB (0.735) than in CoxNet (0.694). In the precision-recall (PR) analysis for predicting 10-year HF risk, the stacking method, combining the SGB, CoxNet, Gaussian mixture and PCP-HF models, outperformed other models with PR/AUC 0.804, while PCP-HF achieved only 0.551.

Conclusion

With a greater number and variety of training cohorts, the model learns a wider range of specific individual health characteristics. Flexible ML algorithms can be used to capture these diverse distributions and produce more precise prediction models.