Predictive modeling for 14-day unplanned hospital readmission risk by using machine learning algorithms

Explainable machine learning to identify risk factors for unplanned hospital readmissions in Nova Scotian hospitals

OBJECTIVE

A report from the Canadian Institute for Health Information found unplanned hospital readmissions (UHR) common, costly, and potentially avoidable, estimating a $1.8 billion cost to the Canadian healthcare system associated with inpatient readmissions within 30 days of discharge for the studied period (11 months). The first step towards addressing this costly problem is enabling early detection of patients at risk through detecting UHR risk factors.

METHODOLOGY

We utilized Machine Learning and explainability tools to examine risk factors for UHR within 30 days of discharge, utilizing data from Nova Scotian (Canada) healthcare institutions (2015-2022). To the best of our knowledge, our research constitutes the most comprehensive study on UHR risk factors for the province.

RESULTS

We found that predicting UHR solely from healthcare data has limitations, as discharge information often falls short of accurately predicting readmission occurrences. However, despite this inherent limitation, integrating explainability tools offers insights into the underlying factors contributing to readmission risk, empowering medical personnel with information to improve patient care and outcomes. As part of this work, we identify and report risk factors for UHR and build a guideline to support medical personnel's decision-making regarding targeted post-discharge follow-ups. We found that conditions such as heart failure and Chronic Obstructive Pulmonary Disease (COPD) are associated with a higher likelihood of readmission. Patients admitted for procedures related to childbirth have a lower probability of readmission. We studied the impact of the admission type, patient characteristics, and patient stay characteristics on UHR. For example, we found that new and elective admission patients are less likely to be readmitted, while patients who received a transfusion are more likely to be readmitted.

CONCLUSIONS

We validated the risk factors and the guidelines using real-world data. Our results suggested that our proposal correctly identifies risk factors and effectively produces valuable guidelines for medical personnel. The guideline evaluation suggests we can screen half the patients while capturing more than 72% of the readmission episodes. Our study contributes insights into the challenge of identifying risk factors for UHR while providing a practical guideline for healthcare professionals to identify factors influencing patient readmission, particularly within Nova Scotia.

Development and validation of interpretable machine learning models to predict distant metastasis and prognosis of muscle-invasive bladder cancer patients

Muscle-Invasive Bladder Cancer (MIBC) is a more aggressive disease than non-muscle-invasive bladder cancer (NMIBC), with greater chances of metastasis. We sought to develop machine learning (ML) models to predict metastasis and prognosis in MIBC patients. Clinical data of MIBC cases from 2000 to 2020 were sourced from the Surveillance, Epidemiology, and End Results (SEER) database. Clinical variables used to predict DM were identified through univariate and multivariate logistic regression, and Recursive Feature Elimination (RFE). Thirteen ML models predicting DM were evaluated based on AUC, PRAUC, accuracy, sensitivity, specificity, precision, cross-entropy, Brier score, balanced accuracy, and F-beta score. SHapley Additive exPlanations (SHAP) framework helped interpret the best model. Additionally, we utilized ML algorithm combinations to predict prognosis in MIBC patients with metastasis. A total of 43,951 T2-T4 MIBC patients aged over 18 years old from the SEER database were enrolled consecutively. Nine clinical variables were selected to predict DM. The CatBoost model was identified as the optimal predictor, with AUC values of 0.956 [0.933, 0.969] for the training set, 0.882 [0.857, 0.919] for the internal test set, and 0.839 [0.723, 0.936] for the external test set. The model achieved an accuracy of 0.875 [0.854, 0.896], sensitivity of 0.869 [0.851, 0.889], specificity of 0.883 [0.823, 0.912], and precision of 0.917 [0.885, 0.944]. SHAP analysis revealed that tumor size was the most influential factor in predicting distant metastasis. For prognosis, the "RSF + Enet[alpha = 0.8]" model emerged as the top performer, with C-index values of 0.683 in training, 0.688 in the internal test, and 0.666 in the external test sets. Our ML models provide high accuracy and dependability, delivering refined, individualized predictions for metastasis risk and prognosis in MIBC patients.

Individual risk and prognostic value prediction by interpretable machine learning for distant metastasis in neuroblastoma: A population-based study and an external validation

PURPOSE

Neuroblastoma (NB) is a childhood malignancy with a poor prognosis and a propensity for distant metastasis (DM). We aimed to establish machine learning (ML) based model to accurately predict risk of DM and prognosis of NB patients with DM.

METHODS

We analyzed NB patients from the Surveillance, Epidemiology, and End Results (SEER) database between 2000 and 2020. Univariate and multivariate logistic analysis were employed to select meaning variables. Recursive Feature Elimination (RFE) method based on 6 ML algorithms was utilized in feature selection. To construct predictive model, 13 ML algorithms were evaluated by area under the operating characteristic curve (AUC), accuracy, sensitivity, specificity, precision, cross-entropy, Brier scores, Balanced Accuracy and F-beta score. An optimal ML model was constructed to predict DM, and the predictive results were explained by SHapley Additive exPlanations (SHAP) framework. Meanwhile, 101 ML algorithm combinations were developed to select the best model with highest C-index to predict prognosis of NB patients with DM.

RESULTS

A total of 1,668 NB patients from SEER database was consecutively enrolled. We identified that tumor primary site, grade, surgery type, regional lymph nodes, radiotherapy and chemotherapy are significant risk factors for DM. CatBoost model was selected as the best prediction model, and AUC was 0.846 (95 %CI: [0.804,0.899]), 0.834 (95 %CI: [0.796,0.873]) and 0.813 (95 %CI: [0.776,0.852]) in training, internal test and external test sets, with 0.777 accuracy, 0.839 sensitivity, 0.72 specificity and 0.731 precision in training set. Grade, chemotherapy and radiotherapy had the greatest effects on DM according to SHAP results. For prognosis prediction, "RSF + GBM" algorithm was the best prognostic model with C-index of 0.656, 0.611 and 0.629 in training, internal test and external test sets.

CONCLUSIONS

Our ML models demonstrate excellent accuracy and reliability, offering more precise personalized metastasis diagnosis and prognostic prediction to NB patients.

Predicting Readmission Among High-Risk Discharged Patients Using a Machine Learning Model With Nursing Data: Retrospective Study

Background

Unplanned readmissions increase unnecessary health care costs and reduce the quality of care. It is essential to plan the discharge care from the beginning of hospitalization to reduce the risk of readmission. Machine learning-based readmission prediction models can support patients' preemptive discharge care services with improved predictive power.

Objective

This study aimed to develop a readmission early prediction model utilizing nursing data for high-risk discharge patients.

Methods

This retrospective study included the electronic medical records of 12,977 patients with 1 of the top 6 high-risk readmission diseases at a tertiary hospital in Seoul from January 2018 to January 2020. We used demographic, clinical, and nursing data to construct a prediction model. We constructed unplanned readmission prediction models by dividing them into Model 1 and Model 2. Model 1 used early hospitalization data (up to 1 day after admission), and Model 2 used all the data. To improve the performance of the machine learning method, we performed 5-fold cross-validation and utilized adaptive synthetic sampling to address data imbalance. The 6 algorithms of logistic regression, random forest, decision tree, XGBoost, CatBoost, and multiperceptron layer were employed to develop predictive models. The analysis was conducted using Python Language Reference, version 3.11.3. (Python Software Foundation).

Results

In Model 1, among the 6 prediction model algorithms, the random forest model had the best result, with an area under the receiver operating characteristic (AUROC) curve of 0.62. In Model 2, the CatBoost model had the best result, with an AUROC of 0.64. BMI, systolic blood pressure, and age consistently emerged as the most significant predictors of readmission risk across Models 1 and 2. Model 1, which enabled early readmission prediction, showed a higher proportion of nursing data variables among its important predictors compared to Model 2.

Conclusions

Machine learning-based readmission prediction models utilizing nursing data provide basic data for evidence-based clinical decision support for high-risk discharge patients with complex conditions and facilitate early intervention. By integrating nursing data containing diverse patient information, these models can provide more comprehensive risk assessment and improve patient outcomes.

Personalized three-year survival prediction and prognosis forecast by interpretable machine learning for pancreatic cancer patients: a population-based study and an external validation

Purpose

The overall survival of patients with pancreatic cancer is extremely low. We aimed to establish machine learning (ML) based model to accurately predict three-year survival and prognosis of pancreatic cancer patients.

Methods

We analyzed pancreatic cancer patients from the Surveillance, Epidemiology, and End Results (SEER) database between 2000 and 2021. Univariate and multivariate logistic analysis were employed to select variables. Recursive Feature Elimination (RFE) method based on 6 ML algorithms was utilized in feature selection. To construct predictive model, 13 ML algorithms were evaluated by area under the curve (AUC), area under precision-recall curve (PRAUC), accuracy, sensitivity, specificity, precision, cross-entropy, Brier scores and Balanced Accuracy (bacc) and F Beta Score (fbeta). An optimal ML model was constructed to predict three-year survival, and the predictive results were explained by SHapley Additive exPlanations (SHAP) framework. Meanwhile, 101 ML algorithm combinations were developed to select the best model with highest C-index to predict prognosis of pancreatic cancer patients.

Results

A total of 20,064 pancreatic cancer patients from SEER database was consecutively enrolled. We utilized eight clinical variables to establish prediction model for three-year survival. CatBoost model was selected as the best prediction model, and AUC was 0.932 [0.924, 0.939], 0.899 [0.873, 0.934] and 0.826 [0.735, 0.919] in training, internal test and external test sets, with 0.839 [0.831, 0.847] accuracy, 0.872 [0.858, 0.887] sensitivity, 0.803 [0.784, 0.825] specificity and 0.832 [0.821, 0.853] precision. Surgery type had the greatest effects on three-year survival according to SHAP results. For prognosis prediction, "RSF+GBM" algorithm was the best prognostic model with C-index of 0.774, 0.722 and 0.674 in training, internal test and external test sets.

Conclusions

Our ML models demonstrate excellent accuracy and reliability, offering more precise personalized prognostic prediction to pancreatic cancer patients.

Development, evaluation and comparison of machine learning algorithms for predicting in-hospital patient charges for congestive heart failure exacerbations, chronic obstructive pulmonary disease exacerbations and diabetic ketoacidosis

BACKGROUND

Hospitalizations for exacerbations of congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD) and diabetic ketoacidosis (DKA) are costly in the United States. The purpose of this study was to predict in-hospital charges for each condition using machine learning (ML) models.

RESULTS

We conducted a retrospective cohort study on national discharge records of hospitalized adult patients from January 1st, 2016, to December 31st, 2019. We constructed six ML models (linear regression, ridge regression, support vector machine, random forest, gradient boosting and extreme gradient boosting) to predict total in-hospital cost for admission for each condition. Our models had good predictive performance, with testing R-squared values of 0.701-0.750 (mean of 0.713) for CHF; 0.694-0.724 (mean 0.709) for COPD; and 0.615-0.729 (mean 0.694) for DKA. We identified important key features driving costs, including patient age, length of stay, number of procedures, and elective/nonelective admission.

CONCLUSIONS

ML methods may be used to accurately predict costs and identify drivers of high cost for COPD exacerbations, CHF exacerbations and DKA. Overall, our findings may inform future studies that seek to decrease the underlying high patient costs for these conditions.

Using machine learning methods to predict all-cause somatic hospitalizations in adults: A systematic review

AIM

In this review, we investigated how Machine Learning (ML) was utilized to predict all-cause somatic hospital admissions and readmissions in adults.

METHODS

We searched eight databases (PubMed, Embase, Web of Science, CINAHL, ProQuest, OpenGrey, WorldCat, and MedNar) from their inception date to October 2023, and included records that predicted all-cause somatic hospital admissions and readmissions of adults using ML methodology. We used the CHARMS checklist for data extraction, PROBAST for bias and applicability assessment, and TRIPOD for reporting quality.

RESULTS

We screened 7,543 studies of which 163 full-text records were read and 116 met the review inclusion criteria. Among these, 45 predicted admission, 70 predicted readmission, and one study predicted both. There was a substantial variety in the types of datasets, algorithms, features, data preprocessing steps, evaluation, and validation methods. The most used types of features were demographics, diagnoses, vital signs, and laboratory tests. Area Under the ROC curve (AUC) was the most used evaluation metric. Models trained using boosting tree-based algorithms often performed better compared to others. ML algorithms commonly outperformed traditional regression techniques. Sixteen studies used Natural language processing (NLP) of clinical notes for prediction, all studies yielded good results. The overall adherence to reporting quality was poor in the review studies. Only five percent of models were implemented in clinical practice. The most frequently inadequately addressed methodological aspects were: providing model interpretations on the individual patient level, full code availability, performing external validation, calibrating models, and handling class imbalance.

CONCLUSION

This review has identified considerable concerns regarding methodological issues and reporting quality in studies investigating ML to predict hospitalizations. To ensure the acceptability of these models in clinical settings, it is crucial to improve the quality of future studies.

Machine Learning-Based Prediction of Readmission Risk in Cardiovascular and Cerebrovascular Conditions Using Patient EMR Data

The primary objective of this study was to develop a risk-based readmission prediction model using the EMR data available at discharge. This model was then validated with the LACE plus score. The study cohort consisted of about 310,000 hospital admissions of patients with cardiovascular and cerebrovascular conditions. The EMR data of the patients consisted of lab results, vitals, medications, comorbidities, and admit/discharge settings. These data served as the input to an XGBoost model v1.7.6, which was then used to predict the number of days until the next readmission. Our model achieved remarkable results, with a precision score of 0.74 (±0.03), a recall score of 0.75 (±0.02), and an overall accuracy of approximately 82% (±5%). Notably, the model demonstrated a high accuracy rate of 78.39% in identifying the patients readmitted within 30 days and 80.81% accuracy for those with readmissions exceeding six months. The model was able to outperform the LACE plus score; of the people who were readmitted within 30 days, only 47.70 percent had a LACE plus score greater than 70, and, for people with greater than 6 months, only 10.09 percent had a LACE plus score less than 30. Furthermore, our analysis revealed that the patients with a higher comorbidity burden and lower-than-normal hemoglobin levels were associated with increased readmission rates. This study opens new doors to the world of differential patient care, helping both clinical decision makers and healthcare providers make more informed and effective decisions. This model is comparatively more robust and can potentially substitute the LACE plus score in cardiovascular and cerebrovascular settings for predicting the readmission risk.

Forecasting hospital discharges for respiratory conditions in Costa Rica using climate and pollution data

Respiratory diseases represent one of the most significant economic burdens on healthcare systems worldwide. The variation in the increasing number of cases depends greatly on climatic seasonal effects, socioeconomic factors, and pollution. Therefore, understanding these variations and obtaining precise forecasts allows health authorities to make correct decisions regarding the allocation of limited economic and human resources. We aimed to model and forecast weekly hospitalizations due to respiratory conditions in seven regional hospitals in Costa Rica using four statistical learning techniques (Random Forest, XGboost, Facebook's Prophet forecasting model, and an ensemble method combining the above methods), along with 22 climate change indices and aerosol optical depth as an indicator of pollution. Models were trained using data from 2000 to 2018 and were evaluated using data from 2019 as testing data. During the training period, we set up 2-year sliding windows and a 1-year assessment period, along with the grid search method to optimize hyperparameters for each model. The best model for each region was selected using testing data, based on predictive precision and to prevent overfitting. Prediction intervals were then computed using conformal inference. The relative importance of all climatic variables was computed for the best model, and similar patterns in some of the seven regions were observed based on the selected model. Finally, reliable predictions were obtained for each of the seven regional hospitals.

Development and Optimization of Machine Learning Algorithms for Predicting In-hospital Patient Charges for Congestive Heart Failure Exacerbations, Chronic Obstructive Pulmonary Disease Exacerbations and Diabetic Ketoacidosis

Background

Hospitalizations for exacerbations of congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD) and diabetic ketoacidosis (DKA) are costly in the United States. The purpose of this study was to predict in-hospital charges for each condition using machine learning (ML) models.

Results

We conducted a retrospective cohort study on national discharge records of hospitalized adult patients from January 1st, 2016, to December 31st, 2019. We used numerous ML techniques to predict in-hospital total cost. We found that linear regression (LM), gradient boosting (GBM) and extreme gradient boosting (XGB) models had good predictive performance and were statistically equivalent, with training R-square values ranging from 0.49-0.95 for CHF, 0.56-0.95 for COPD, and 0.32-0.99 for DKA. We identified important key features driving costs, including patient age, length of stay, number of procedures. and elective/nonelective admission.

Conclusions

ML methods may be used to accurately predict costs and identify drivers of high cost for COPD exacerbations, CHF exacerbations and DKA. Overall, our findings may inform future studies that seek to decrease the underlying high patient costs for these conditions.

Comparison of Acute Health Care Utilization Between Patients Receiving In-Center Hemodialysis and the General Population: A Population-Based Matched Cohort Study From Ontario, Canada

Background

Patients receiving maintenance hemodialysis have multiple comorbidities and are at high risk of presenting to the hospital. However, the incidence and cost of acute health care utilization in the in-center hemodialysis population and how this compares with other populations is poorly understood.

Objective

To determine the rate, pattern, and cost of emergency department visits and hospitalizations in patients receiving in-center hemodialysis compared with a matched general population.

Design

Population-based matched cohort study.

Setting

We used linked administrative health care databases from Ontario, Canada.

Patients

We included 25 379 patients (incident and prevalent) receiving in-center hemodialysis between January 1, 2010, and December 31, 2018. Patients were matched on birth date (±2 years), sex, and cohort entry date using a 1:4 ratio to 101 516 individuals from the general population.

Measurements

Our primary outcomes were emergency department visits (allowing for multiple visits per individual) and hospital admissions from the emergency department. We also assessed all-cause hospitalizations, all-cause readmissions within 30 days of discharge from the original hospitalization, length of stay for hospital admissions (including multiple visits per individual), and the financial cost of these admissions.

Methods

We presented the rate, percentage, median (25th, 75th percentiles), and incidence rate per 1000 person-years for emergency department visits and hospitalizations. Individual-level health care costs for emergency department visits and all-cause hospitalization were estimated using resource intensity weights multiplied by the cost per weighted case.

Results

Patients receiving in-center hemodialysis had substantially more comorbidities (eg, diabetes) than the matched general population. Eighty percent (n = 20 309) of patients receiving in-center hemodialysis had at least 1 emergency department visit compared with 56% (n = 56 452) of individuals in the matched general population, over a median follow-up of 1.8 years (25th, 75th percentiles: 0.7, 3.6) and 5.2 (2.5, 8.4) years, respectively. The incidence rate of emergency department visits, allowing for multiple visits per individual, was 2274 per 1000 person-years (95% confidence interval [CI]: 2263, 2286) for patients receiving in-center hemodialysis, which was almost 5 times as high as the matched general population (471 per 1000 person-years; 95% CI: 469, 473). The rate of hospital admissions from the emergency department and the rate of all-cause hospital admissions in the in-center hemodialysis population was more than 7 times as high as the matched general population (hospital admissions from the emergency department: 786 vs 101 per 1000 person-years; all-cause hospital admissions: 1056 vs 139 per 1000 person-years). The median number of all-cause hospitalization days per patient year was 4.0 (0, 16.5) in the in-center hemodialysis population compared with 0 (0, 0.5) in the matched general population. The cost per patient-year for emergency department visits in the in-center hemodialysis population was approximately 5.5 times as high as the matched general population while the cost of hospitalizations in the in-center hemodialysis population was approximately 11 times as high as the matched general population (emergency department visits: CAN$ 1153 vs CAN$ 209; hospitalizations: CAN$ 21 151 vs CAN$ 1873 [all costs in 2023 CAN$]).

Limitations

External generalizability and we could not determine whether emergency department visits and hospitalizations were preventable.

Conclusions

Patients receiving in-center hemodialysis have high acute health care utilization. These results improve our understanding of the burden of disease and the associated costs in the in-center hemodialysis population, highlight the need to improve acute outcomes, and can aid health care capacity planning. Additional research is needed to address the risk of hospitalization after controlling for patient comorbidities.

Trial registration

This is not applicable as this is a population-based matched cohort study and not a clinical trial.

Machine learning-based prediction model for emergency department visits using prescription information in community-dwelling non-cancer older adults

Older adults are more likely to require emergency department (ED) visits than others, which might be attributed to their medication use. Being able to predict the likelihood of an ED visit using prescription information and readily available data would be useful for primary care. This study aimed to predict the likelihood of ED visits using extensive medication variables generated according to explicit clinical criteria for elderly people and high-risk medication categories by applying machine learning (ML) methods. Patients aged ≥ 65 years were included, and ED visits were predicted with 146 variables, including demographic and comprehensive medication-related factors, using nationwide claims data. Among the eight ML models, the final model was developed using LightGBM, which showed the best performance. The final model incorporated 93 predictors, including six sociodemographic, 28 comorbidity, and 59 medication-related variables. The final model had an area under the receiver operating characteristic curve of 0.689 in the validation cohort. Approximately half of the top 20 strong predictors were medication-related variables. Here, an ED visit risk prediction model for older people was developed and validated using administrative data that can be easily applied in clinical settings to screen patients who are likely to visit an ED.

Development of a prediction model for the risk of 30-day unplanned readmission in older patients with heart failure: A multicenter retrospective study

BACKGROUND AND AIM

Heart failure (HF) imposes significant global health costs due to its high incidence, readmission, and mortality rate. Accurate assessment of readmission risk and precise interventions have become important measures to improve health for patients with HF. Therefore, this study aimed to develop a machine learning (ML) model to predict 30-day unplanned readmissions in older patients with HF.

METHODS AND RESULTS

This study collected data on hospitalized older patients with HF from the medical data platform of Chongqing Medical University from January 1, 2012, to December 31, 2021. A total of 5 candidate algorithms were selected from 15 ML algorithms with excellent performance, which was evaluated by area under the operating characteristic curve (AUC) and accuracy. Then, the 5 candidate algorithms were hyperparameter tuned by 5-fold cross-validation grid search, and performance was evaluated by AUC, accuracy, sensitivity, specificity, and recall. Finally, an optimal ML model was constructed, and the predictive results were explained using the SHapley Additive exPlanations (SHAP) framework. A total of 14,843 older patients with HF were consecutively enrolled. CatBoost model was selected as the best prediction model, and AUC was 0.732, with 0.712 accuracy, 0.619 sensitivity, and 0.722 specificity. NT.proBNP, length of stay (LOS), triglycerides, blood phosphorus, blood potassium, and lactate dehydrogenase had the greatest effect on 30-day unplanned readmission in older patients with HF, according to SHAP results.

CONCLUSIONS

The study developed a CatBoost model to predict the risk of unplanned 30-day special-cause readmission in older patients with HF, which showed more significant performance compared with the traditional logistic regression model.

Guidelines on perioperative optimization protocol for the adult patient 2023

OBJECTIVE

The French Society of Anesthesiology and Intensive Care Medicine [Société Française d'Anesthésie et de Réanimation (SFAR)] aimed at providing guidelines for the implementation of perioperative optimization programs.

DESIGN

A consensus committee of 29 experts from the SFAR was convened. A formal conflict-of-interest policy was developed at the outset of the process and enforced throughout. The entire guidelines process was conducted independently of any industry funding. The authors were advised to follow the principles of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system to guide assessment of quality of evidence.

METHODS

Four fields were defined: 1) Generalities on perioperative optimization programs; 2) Preoperative measures; 3) Intraoperative measures and; 4) Postoperative measures. For each field, the objective of the recommendations was to answer a number of questions formulated according to the PICO model (population, intervention, comparison, and outcomes). Based on these questions, an extensive bibliographic search was carried out using predefined keywords according to PRISMA guidelines and analyzed using the GRADE® methodology. The recommendations were formulated according to the GRADE® methodology and then voted on by all the experts according to the GRADE grid method. As the GRADE® methodology could have been fully applied for the vast majority of questions, the recommendations were formulated using a "formalized expert recommendations" format.

RESULTS

The experts' work on synthesis and application of the GRADE® method resulted in 30 recommendations. Among the formalized recommendations, 19 were found to have a high level of evidence (GRADE 1±) and ten a low level of evidence (GRADE 2±). For one recommendation, the GRADE methodology could not be fully applied, resulting in an expert opinion. Two questions did not find any response in the literature. After two rounds of rating and several amendments, strong agreement was reached for all the recommendations.

CONCLUSIONS

Strong agreement among the experts was obtained to provide 30 recommendations for the elaboration and/or implementation of perioperative optimization programs in the highest number of surgical fields.

Development and Internal Validation of an Interpretable Machine Learning Model to Predict Readmissions in a United States Healthcare System

(1) One in four hospital readmissions is potentially preventable. Machine learning (ML) models have been developed to predict hospital readmissions and risk-stratify patients, but thus far they have been limited in clinical applicability, timeliness, and generalizability. (2) Methods: Using deidentified clinical data from the University of California, San Francisco (UCSF) between January 2016 and November 2021, we developed and compared four supervised ML models (logistic regression, random forest, gradient boosting, and XGBoost) to predict 30-day readmissions for adults admitted to a UCSF hospital. (3) Results: Of 147,358 inpatient encounters, 20,747 (13.9%) patients were readmitted within 30 days of discharge. The final model selected was XGBoost, which had an area under the receiver operating characteristic curve of 0.783 and an area under the precision-recall curve of 0.434. The most important features by Shapley Additive Explanations were days since last admission, discharge department, and inpatient length of stay. (4) Conclusions: We developed and internally validated a supervised ML model to predict 30-day readmissions in a US-based healthcare system. This model has several advantages including state-of-the-art performance metrics, the use of clinical data, the use of features available within 24 h of discharge, and generalizability to multiple disease states.

Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011-2022)

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) has branched out to various applications in healthcare, such as health services management, predictive medicine, clinical decision-making, and patient data and diagnostics. Although AI models have achieved human-like performance, their use is still limited because they are seen as a black box. This lack of trust remains the main reason for their low use in practice, especially in healthcare. Hence, explainable artificial intelligence (XAI) has been introduced as a technique that can provide confidence in the model's prediction by explaining how the prediction is derived, thereby encouraging the use of AI systems in healthcare. The primary goal of this review is to provide areas of healthcare that require more attention from the XAI research community.

METHODS

Multiple journal databases were thoroughly searched using PRISMA guidelines 2020. Studies that do not appear in Q1 journals, which are highly credible, were excluded.

RESULTS

In this review, we surveyed 99 Q1 articles covering the following XAI techniques: SHAP, LIME, GradCAM, LRP, Fuzzy classifier, EBM, CBR, rule-based systems, and others.

CONCLUSION

We discovered that detecting abnormalities in 1D biosignals and identifying key text in clinical notes are areas that require more attention from the XAI research community. We hope this is review will encourage the development of a holistic cloud system for a smart city.

Explainable machine learning models for predicting 30-day readmission in pediatric pulmonary hypertension: A multicenter, retrospective study

Background

Short-term readmission for pediatric pulmonary hypertension (PH) is associated with a substantial social and personal burden. However, tools to predict individualized readmission risk are lacking. This study aimed to develop machine learning models to predict 30-day unplanned readmission in children with PH.

Methods

This study collected data on pediatric inpatients with PH from the Chongqing Medical University Medical Data Platform from January 2012 to January 2019. Key clinical variables were selected by the least absolute shrinkage and the selection operator. Prediction models were selected from 15 machine learning algorithms with excellent performance, which was evaluated by area under the operating characteristic curve (AUC). The outcome of the predictive model was interpreted by SHapley Additive exPlanations (SHAP).

Results

A total of 5,913 pediatric patients with PH were included in the final cohort. The CatBoost model was selected as the predictive model with the greatest AUC for 0.81 (95% CI: 0.77–0.86), high accuracy for 0.74 (95% CI: 0.72–0.76), sensitivity 0.78 (95% CI: 0.69–0.87), and specificity 0.74 (95% CI: 0.72–0.76). Age, length of stay (LOS), congenital heart surgery, and nonmedical order discharge showed the greatest impact on 30-day readmission in pediatric PH, according to SHAP results.

Conclusions

This study developed a CatBoost model to predict the risk of unplanned 30-day readmission in pediatric patients with PH, which showed more significant performance compared with traditional logistic regression. We found that age, LOS, congenital heart surgery, and nonmedical order discharge were important factors for 30-day readmission in pediatric PH.

Comparison of machine learning models for predicting the risk of breast cancer-related lymphedema in Chinese women

Objective

Predictive models for the occurrence of cancer symptoms by using machine learning (ML) algorithms could be used to aid clinical decision-making in order to enhance the quality of cancer care. This study aimed to develop and validate a selection of classification models that used ML algorithms to predict the occurrence of breast cancer-related lymphedema (BCRL) among Chinese women.

Methods

This was a retrospective cohort study of consecutive cases that had been diagnosed with breast cancer, stages I-IV. Forty-eight variables were grouped into five feature sets. Five classification models with ML algorithms were developed, and the models' performance and the variables’ relative importance were assessed accordingly.

Results

Of 370 eligible female participants, 91 had BCRL (24.6%). The mean age of this study sample was 49.89 (SD ​= ​7.45). All participants had had breast cancer surgery, and more than half of them had had a modified radical mastectomy (n ​= ​206, 55.5%). The mean follow-up time after breast cancer surgery was 28.73 months (SD ​= ​11.71). Most of the tumors were either stage I (n ​= ​49, 31.2%) or stage II (n ​= ​252, 68.1%). More than half of the sample had had postoperative chemotherapy (n ​= ​227, 61.4%). Overall, the logistic regression model achieved the best performance in terms of accuracy (91.6%), precision (82.1%), and recall (91.4%) for BCRL. Although this study included 48 predicting variables, we found that the five models required only 22 variables to achieve predictive performance. The most important variable was the number of positive lymph nodes, followed in descending order by the BCRL occurring on the same side as the surgery, a history of sentinel lymph node biopsy, a dietary preference for meat and fried food, and an exercise frequency of less than three times per week. These factors were the most influential predictors for enhancing the ML models’ performance.

Conclusions

This study found that in the ML training dataset, the multilayer perceptron model and the logistic regression model were the best discrimination models for predicting the outcome of BCRL, and the k-nearest neighbors and support vector machine models demonstrated good calibration performance in the ML validation dataset. Future research will need to use large-sample datasets to establish a more robust ML model for predicting BCRL deeply and reliably.