Decision curve analysis confirms higher clinical utility of multi-domain versus single-domain prediction models in patients with open abdomen treatment for peritonitis

Predicting the risk of ibrutinib in combination with R-ICE in patients with relapsed or refractory DLBCL using explainable machine learning algorithms

Relapsed or refractory diffuse large B-cell lymphoma (DLBCL) poses significant therapeutic challenges due to heterogeneous patient outcomes. This study aimed to evaluate the efficacy of the ibrutinib plus R-ICE regimen and to leverage explainable machine learning models (ML) for predicting treatment risks and outcomes. Retrospective data from 28 patients treated between March 2019 and July 2022 were analyzed. Machine learning models, including CoxBoost + StepCox, were developed and validated using bootstrap methods. Synthetic minority over-sampling combined with propensity score matching (SMOTE-PSM) addressed class imbalances. Prognostic performance was compared against the Cox proportional hazards model using decision curve and calibration analysis, as well as time-dependent ROC curves. The CoxBoost + StepCox model achieved an average C-index of 0.955 for overall survival (OS) and progression-free survival (PFS). Key prognostic indicators included elevated lactate dehydrogenase (LDH), initial treatment response, time to relapse > 12 months, and CD5 + expression. Calibration curves showed a C-index of 0.932 for OS and 0.972 for PFS in the training set. CD5 + was most predictive for OS and LDH for PFS. Machine learning models demonstrated high accuracy and clinical utility, indicating potential for data-driven treatment decisions in DLBCL.

Development of a machine learning prediction model for loss to follow-up in HIV care using routine electronic medical records in a low-resource setting

BACKGROUND

Despite the global commitment to ending AIDS by 2030, the loss of follow-up (LTFU) in HIV care remains a significant challenge. To address this issue, a data-driven clinical decision tool is crucial for identifying patients at greater risk of LTFU and facilitating personalized and proactive interventions. This study aimed to develop a prediction model to assess the future risk of LTFU in HIV care in Ethiopia.

METHODS

The study used a retrospective design in which machine learning (ML) methods were applied to the electronic medical records (EMRs) data of adult HIV-positive individuals who were newly enrolled in antiretroviral therapy between July 2019 and April 2024. The data were collected across eight randomly selected high-volume healthcare facilities. Six supervised ML classifiers-J48 decision tree, random forest, K-nearest neighbors, support vector machine, logistic regression, and naïve Bayes-were utilized for training via Weka 3.8.6 software. The performance of each algorithm was evaluated through a 10-fold cross-validation approach. Algorithm performance was compared via the corrected resampled t test (p < 0.05), and decision curve analysis (DCA) was used to assess the model's clinical utility.

RESULTS

A total of 3,720 individuals' EMR data were analyzed, with 2,575 (69.2%) classified as not LTFU and 1,145 (30.8%) classified as LTFU. On the basis of the ML feature selection process, six strong predictors of LTFU were identified: differentiated service delivery model, adherence, tuberculosis preventive therapy, follow-up period, nutritional status, and address information. The random forest algorithm showed superior performance, with an accuracy of 84.2%, a sensitivity of 82.4%, a specificity of 85.7%, a precision of 83.7%, an F1 score of 83.1%, and an area under the curve of 89.5%. The model demonstrated greater clinical utility, offering greater net benefit than both the 'intervention for all' approach and the 'intervention for none' approach, particularly at threshold probabilities of 10% and above.

CONCLUSIONS

This study developed a machine learning-based predictive model for assessing the future risk of LTFU in HIV care within low-resource settings. Notably, the model built via the random forest algorithm exhibited high accuracy and strong discriminative performance, highlighting its positive net benefit for clinical applications. Furthermore, ongoing external validation across diverse populations is important to ensure the model's reliability and generalizability.

Epidemiology and reporting characteristics of systematic reviews of clinical prediction models: a scoping review

OBJECTIVES

This study aimed to explore research trends and areas of interest in systematic reviews (SRs) and meta-analysis of clinical prediction models (CPMs), while summarizing their conduct and reporting characteristics.

STUDY DESIGN AND SETTING

A scoping review was conducted, with searches performed in PubMed, Embase, and Cochrane Library from inception to January 7, 2023. Pairs of reviewers independently screened potentially eligible studies. Data on bibliographic and methodological characteristics were collected and analyzed descriptively.

RESULTS

A total of 1004 SRs published between 2001 and 2023 were included. The number of SRs increased significantly after 2020, with the majority originating from Europe (44.1%) and Asia (26.7%). Populations and outcomes were categorized into 19 and 34 classifications, respectively. The general population was the most frequently targeted (38.7%), and mortality was the most common outcome (18.9%). The prediction or diagnosis of neoplasms in the general population was the most prevalent focus (7.2%). Prognostic models were included only in 69.6% of SRs, while diagnostic models were included in 16.8%; 13.6% included both. The number of primary studies included in SRs ranged from 1 to 495, and the models ranged from 1 to 731. Most SRs lacked standardized reporting: 88.3% did not frame their review questions using established frameworks, and 79.8% did not follow standardized checklists for data extraction. Quality and risk of bias assessments were reported in 76.5% of SRs, with the Prediction model Risk of Bias Assessment Tool (27.9%) and the Quality Assessment of Diagnostic Accuracy Studies-2 tool (17.0%) being the most common. Narrative synthesis was the predominant method for evidence summarization (63.5%), while meta-analysis was conducted in 36.5%. Measures of model performance were summarized in 80.5% of SRs, with discrimination being the most frequently reported (67.7%). Only 5.2% assessed the certainty of evidence. Moreover, 42.2% of SRs published a protocol, 76.0% clearly stated support, and 91.1% stated competing interests.

CONCLUSION

The number of SRs of CPMs has grown substantially, with increasing diversity in populations and outcomes. However, significant variability in conduct and reporting was observed. Future SRs should strictly follow well-developed guidelines, and a dedicated study assessing the reporting quality and risk of bias in SRs of CPMs is warranted.

Choice of Machine Learning Models Is Important to Predict Post-Operative Infections in Surgical Patients

Background: Surgical quality datasets are critical to decision-making tools including surgical infection (SI). Machine learning models (MLMs), a branch of artificial intelligence (AI), are increasingly being ingrained within surgical decision-making algorithms. However, given the unique and distinct functioning of individual models, not all models may be suitable for acutely ill surgical patients. Patients and Methods: This is a 5-year retrospective review of National Surgical Quality Improvement Program (NSQIP) patients who underwent an operation. The data were reviewed for demographics, medical comorbidities, rates, and sites of infection. To generate the MLMs, data were imported into Python, and four common MLMs, extreme gradient boosting, K-nearest neighbor (KNN), random forest, and logistic regression, as well as two novel models (flexible discriminant analysis and generalized additive model) and ensemble modeling, were generated to predict post-operative SIs. Outputs included area under the receiver-operating characteristic curve (AUC ROC) including recall curves. Results: Overall, 624,625 urgent and emergent NSQIP patients were included. The overall infection rate was 8.6%. Patients who sustained a post-operative infection were older, more likely geriatric, male, diabetic, had chronic obstructive pulmonary disease, were smokers, and were less likely White race. With respect to MLMs, all four MLMs had reasonable accuracy. However, a hierarchy of MLMs was noted with predictive abilities (XGB AUC = 0.85 and logistic regression = 0.82), wherein KNN has the lowest performance (AUC = 0.62). With respect to the ability to detect an infection, precision recall of XGB performed well (AUC = 0.73), whereas KNN performed poorly (AUC = 0.16). Conclusions: MLMs are not created nor function similarly. We identified differences with MLMs to predict post-operative infections in surgical patients. Before MLMs are incorporated into surgical decision making, it is critical that surgeons are at the fore of understanding the role and functioning of MLMs.

Construction and validation of a clinical prediction model for sepsis using peripheral perfusion index to predict in-hospital and 28-day mortality risk

Sepsis is a clinical syndrome caused by infection, leading to organ dysfunction due to a dysregulated host response. In recent years, its high mortality rate has made it a significant cause of death and disability worldwide. The pathophysiological process of sepsis is related to the body's dysregulated response to infection, with microcirculatory changes serving as early warning signals that guide clinical treatment. The Peripheral Perfusion Index (PI), as an indicator of peripheral microcirculation, can effectively evaluate patient prognosis. This study aims to develop two new prediction models using PI and other common clinical indicators to assess the mortality risk of sepsis patients during hospitalization and within 28 days post-ICU admission. This retrospective study analyzed data from sepsis patients treated in the Intensive Care Unit of Peking Union Medical College Hospital between December 2019 and June 2023, ultimately including 645 patients. LASSO regression and logistic regression analyses were used to select predictive factors from 35 clinical indicators, and two clinical prediction models were constructed to predict in-hospital mortality and 28-day mortality. The models' performance was then evaluated using ROC curve, calibration curve, and decision curve analyses. The two prediction models performed excellently in distinguishing patient mortality risk. The AUC for the in-hospital mortality prediction model was 0.82 in the training set and 0.73 in the validation set; for the 28-day mortality prediction model, the AUC was 0.79 in the training set and 0.73 in the validation set. The calibration curves closely aligned with the ideal line, indicating consistency between predicted and actual outcomes. Decision curve analysis also demonstrated high net benefits for the clinical utility of both models. The study shows that these two prediction models not only perform excellently statistically but also hold high practical value in clinical applications. The models can help physicians accurately assess the mortality risk of sepsis patients, providing a scientific basis for personalized treatment.

Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making

BACKGROUND

Clinical prediction modeling plays a pivotal part in modern clinical care, particularly in predicting the risk of in-hospital mortality. Recent modeling efforts have focused on leveraging intraoperative data sources to improve model performance. However, the individual and collective benefit of pre- and intraoperative data for clinical decision-making remains unknown. We hypothesized that pre- and intraoperative predictors contribute equally to the net benefit in a decision curve analysis (DCA) of in-hospital mortality prediction models that include pre- and intraoperative predictors.

METHODS

Data from the VitalDB database featuring a subcohort of 6043 patients were used. A total of 141 predictors for in-hospital mortality were grouped into preoperative (demographics, intervention characteristics, and laboratory measurements) and intraoperative (laboratory and monitor data, drugs, and fluids) data. Prediction models using either preoperative, intraoperative, or all data were developed with multiple methods (logistic regression, neural network, random forest, gradient boosting machine, and a stacked learner). Predictive performance was evaluated by the area under the receiver-operating characteristic curve (AUROC) and under the precision-recall curve (AUPRC). Clinical utility was examined with a DCA in the predefined risk preference range (denoted by so-called treatment threshold probabilities) between 0% and 20%.

RESULTS

AUROC performance of the prediction models ranged from 0.53 to 0.78. AUPRC values ranged from 0.02 to 0.25 (compared to the incidence of 0.09 in our dataset) and high AUPRC values resulted from prediction models based on preoperative laboratory values. A DCA of pre- and intraoperative prediction models highlighted that preoperative data provide the largest overall benefit for decision-making, whereas intraoperative values provide only limited benefit for decision-making compared to preoperative data. While preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for low treatment thresholds up to 5% to 10%, preoperative laboratory measurements become the dominant source for decision support for higher thresholds.

CONCLUSIONS

When it comes to predicting in-hospital mortality and subsequent decision-making, preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for clinicians with risk-averse preferences, whereas preoperative laboratory values provide the largest benefit for decision-makers with more moderate risk preferences. Our decision-analytic investigation of different predictor categories moves beyond the question of whether certain predictors provide a benefit in traditional performance metrics (eg, AUROC). It offers a nuanced perspective on for whom these predictors might be beneficial in clinical decision-making. Follow-up studies requiring larger datasets and dedicated deep-learning models to handle continuous intraoperative data are essential to examine the robustness of our results.