A tutorial on calibration measurements and calibration models for clinical prediction models

Differential gene expression analysis pipelines and bioinformatic tools for the identification of specific biomarkers: A review

In recent years, the role of bioinformatics and computational biology together with omics techniques and transcriptomics has gained tremendous importance in biomedicine and healthcare, particularly for the identification of biomarkers for precision medicine and drug discovery. Differential gene expression (DGE) analysis is one of the most used techniques for RNA-sequencing (RNA-seq) data analysis. This tool, which is typically used in various RNA-seq data processing applications, allows the identification of differentially expressed genes across two or more sample sets. Functional enrichment analyses can then be performed to annotate and contextualize the resulting gene lists. These studies provide valuable information about disease-causing biological processes and can help in identifying molecular targets for novel therapies. This review focuses on differential gene expression (DGE) analysis pipelines and bioinformatic techniques commonly used to identify specific biomarkers and discuss the advantages and disadvantages of these techniques.

Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction

BACKGROUND

Ex-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting.

METHODS

Our cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: "Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior." The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score.

RESULTS

In total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856-0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels.

CONCLUSION

Our machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.

Using Machine Learning to Evaluate the Value of Genetic Liabilities in the Classification of Hypertension within the UK Biobank

Background and Objective: Hypertension increases the risk of cardiovascular diseases (CVD) such as stroke, heart attack, heart failure, and kidney disease, contributing to global disease burden and premature mortality. Previous studies have utilized statistical and machine learning techniques to develop hypertension prediction models. Only a few have included genetic liabilities and evaluated their predictive values. This study aimed to develop an effective hypertension classification model and investigate the potential influence of genetic liability for multiple risk factors linked to CVD on hypertension risk using the random forest and the neural network. Materials and Methods: The study involved 244,718 European participants, who were divided into training and testing sets. Genetic liabilities were constructed using genetic variants associated with CVD risk factors obtained from genome-wide association studies (GWAS). Various combinations of machine learning models before and after feature selection were tested to develop the best classification model. The models were evaluated using area under the curve (AUC), calibration, and net reclassification improvement in the testing set. Results: The models without genetic liabilities achieved AUCs of 0.70 and 0.72 using the random forest and the neural network methods, respectively. Adding genetic liabilities improved the AUC for the random forest but not for the neural network. The best classification model was achieved when feature selection and classification were performed using random forest (AUC = 0.71, Spiegelhalter z score = 0.10, p-value = 0.92, calibration slope = 0.99). This model included genetic liabilities for total cholesterol and low-density lipoprotein (LDL). Conclusions: The study highlighted that incorporating genetic liabilities for lipids in a machine learning model may provide incremental value for hypertension classification beyond baseline characteristics.

Generalizability of polygenic prediction models: how is the R2 defined on test data?

BACKGROUND

Polygenic risk scores (PRS) quantify an individual's genetic predisposition for different traits and are expected to play an increasingly important role in personalized medicine. A crucial challenge in clinical practice is the generalizability and transferability of PRS models to populations with different ancestries. When assessing the generalizability of PRS models for continuous traits, the R 2 is a commonly used measure to evaluate prediction accuracy. While the R 2 is a well-defined goodness-of-fit measure for statistical linear models, there exist different definitions for its application on test data, which complicates interpretation and comparison of results.

METHODS

Based on large-scale genotype data from the UK Biobank, we compare three definitions of the R 2 on test data for evaluating the generalizability of PRS models to different populations. Polygenic models for several phenotypes, including height, BMI and lipoprotein A, are derived based on training data with European ancestry using state-of-the-art regression methods and are evaluated on various test populations with different ancestries.

RESULTS

Our analysis shows that the choice of the R 2  definition can lead to considerably different results on test data, making the comparison of R 2  values from the literature problematic. While the definition as the squared correlation between predicted and observed phenotypes solely addresses the discriminative performance and always yields values between 0 and 1, definitions of the R 2 based on the mean squared prediction error (MSPE) with reference to intercept-only models assess both discrimination and calibration. These MSPE-based definitions can yield negative values indicating miscalibrated predictions for out-of-target populations. We argue that the choice of the most appropriate definition depends on the aim of PRS analysis - whether it primarily serves for risk stratification or also for individual phenotype prediction. Moreover, both correlation-based and MSPE-based definitions of R 2 can provide valuable complementary information.

CONCLUSIONS

Awareness of the different definitions of the R 2 on test data is necessary to facilitate the reporting and interpretation of results on PRS generalizability. It is recommended to explicitly state which definition was used when reporting R 2 values on test data. Further research is warranted to develop and evaluate well-calibrated polygenic models for diverse populations.

Validation of the European Society of Cardiology pretest probability models for obstructive coronary artery disease in high-risk population

BACKGROUND

The pre-test probability (PTP) model for obstructive coronary artery disease (CAD) was updated in 2019 by the European Society of Cardiology (ESC). To our knowledge, this model was never externally validated in population with high incidence of CAD. The aim of this study is to validate the new PTP ESC model in our population which has a high CAD incidence and to compare it with previous PTP ESC model from 2013.

METHODS

We retrospectively analysed 1294 symptomatic patients with suspected CAD referred to our centre between 2015 and 2019. In all patients, the PTP score was calculated based on age, gender and symptoms according to the ESC model from 2013 (2013-ESC-PTP) and 2019 (2019-ESC-PTP). All patients underwent invasive coronary angiography (ICA).

RESULTS

Of the 1294 patients, obstructive CAD was diagnosed in 533 patients (41.2%). The 2019-ESC-PTP model categorised significantly more patients into the low probability group (PTP < 15%) than the 2013-ESC-PTP model (39.8% vs. 5.6%, P< 0.001). Obstructive CAD prevalence was underestimated using 2019-ESC-PTP at all PTP levels (calibration intercept 1.15, calibration slope 0.96). The 2013-ESC-PTP overestimated obstructive CAD prevalence (calibration intercept -0.24, calibration slope 0.73). The discrimination measured with an area under the curve was similar for both models, indicating moderate accuracy of the models.

CONCLUSIONS

In high-risk Serbian population, both the 2013 and 2019 ESC-PTP models had moderate accuracy in diagnosing CAD, with the 2019-ESC-PTP underestimating the prevalence of CAD, while the 2013-ESC-PTP overestimating it. Further studies are warranted to establish PTP models for high-risk countries.

Association of Blood Copper With the Subclinical Carotid Atherosclerosis: An Observational Study

BACKGROUND

Copper exposure is reported to be associated with increased risk of stroke. However, the association of copper exposure with subclinical carotid atherosclerosis remains unclear.

METHODS AND RESULTS

This observational study included consecutive participants from Xinqiao Hospital between May 2020 and August 2021. Blood metals were measured using inductively coupled plasma mass spectrometry and carotid atherosclerosis was assessed using ultrasound. Modified Poisson regression was performed to evaluate the associations of copper and other metals with subclinical carotid plaque presence. Blood metals were analyzed as categorical according to the quartiles. Multivariable models were adjusted for age, sex, body mass index, education, smoking, drinking, hypertension, diabetes, dyslipidemia, estimated glomerular filtration rate, and coronary artery disease history. Bayesian Kernel Machine Regression was conducted to evaluate the overall association of metal mixture with subclinical carotid plaque presence. One thousand five hundred eighty-five participants were finally enrolled in our study, and carotid plaque was found in 1091 subjects. After adjusting for potential confounders, metal-progressively-adjusted models showed that blood copper was positively associated with subclinical carotid plaque (relative risk according to comparing quartile 4 to quartile 1 was 1.124 [1.021-1.238], relative risk according to per interquartile increment was 1.039 [1.008-1.071]). Blood cadmium and lead were also significantly associated with subclinical carotid plaque. Bayesian Kernel Machine Regression analyses suggested a synergistic effect of copper-cadmium-lead mixture on subclinical carotid plaque presence.

CONCLUSIONS

Our findings identify copper as a novel risk factor of subclinical carotid atherosclerosis and show the potential synergistic proatherogenic effect of copper, cadmium, and lead mixture.

Prediction Models for Forecasting Risk of Development of Surgical Site Infection after Lower Limb Revascularization Surgery: A Systematic Review

BACKGROUND

Surgical site infections (SSIs) are a common and potentially preventable complication of lower limb revascularization surgery associated with increased healthcare resource utilization and patient morbidity. We conducted a systematic review to evaluate multivariable prediction models designed to forecast risk of SSI development after these procedures.

METHODS

After protocol registration (CRD42022331292), we searched MEDLINE, EMBASE, CENTRAL, and Evidence-Based Medicine Reviews (inception to April 4th, 2023) for studies describing multivariable prediction models designed to forecast risk of SSI in adults after lower limb revascularization surgery. Two investigators independently screened abstracts and full-text articles, extracted data, and assessed risk of bias. A narrative synthesis was performed to summarize predictors included in the models and their calibration and discrimination, validation status, and clinical applicability.

RESULTS

Among the 6,671 citations identified, we included 5 studies (n = 23,063 patients). The included studies described 5 unique multivariable prediction models generated through forward selection, backward selection, or Akaike Information Criterion-based methods. Two models were designed to predict any SSI and 3 Szyilagyi grade II (extending into subcutaneous tissue) SSI. Across the 5 models, 18 adjusted predictors (10 of which were preoperative, 3 intraoperative, and 5 postoperative) significantly predicted any SSI and 14 adjusted predictors significantly predict Szilagyi grade II SSI. Female sex, obesity, and chronic obstructive pulmonary disease significantly predicted SSI in more than one model. All models had a "good fit" according to the Hosmer-Lemeshow test (P > 0.05). Model discrimination was quantified using the area under the curve, which ranged from 0.66 to 0.75 across models. Two models were internally validated using non-exhaustive twofold cross-validation and bootstrap resampling. No model was externally validated. Three studies had a high overall risk of bias according to the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

CONCLUSIONS

Five multivariable prediction models with moderate discrimination have been developed to forecast risk of SSI development after lower limb revascularization surgery. Given the frequency and consequences of SSI after these procedures, development and external validation of novel prediction models and comparison of these models to the existing models evaluated in this systematic review is warranted.

Early prediction of long hospital stay for Intensive Care units readmission patients using medication information

OBJECTIVE

Predicting Intensive Care Unit (ICU) Length of Stay (LOS) accurately can improve patient wellness, hospital operations, and the health system's financial status. This study focuses on predicting the prolonged ICU LOS (≥3 days) of the 2nd admission, utilizing short historical data (1st admission only) for early-stage prediction, as well as incorporating medication information.

MATERIALS AND METHODS

We selected 18,572 ICU patients' records from the MIMIC-IV database for this study. We applied five machine learning classifiers: Logistic regression (LR), Random Forest (RF), Support Vector Machine (SVM), AdaBoost (AB) and XGBoost (XGB). We computed both the sum dose and the average dose for the medication and included them in our model.

RESULTS

The performance of the RF model demonstrates the highest level of accuracy compared to other models, as indicated by an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.716 and an Expected Calibration Error (ECE) of 0.023.

DISCUSSION

The calibration improved all five classifiers (LR, RF, SVC, AB, XGB) in terms of ECE. The most important two features for RF are the length of 1st admission and the patient's age when they visited the hospital. The most important medication features are Phytonadione and Metoprolol Succinate XL. Also, both the sum and the average dose for the medication features contributed to the prediction task.

CONCLUSION

Our model showed the capability to predict the prolonged ICU LOS of the 2nd admission by utilizing the demographic, diagnosis, and medication information from the 1st admission. This method can potentially support the prevention of patient complications and enhance resource allocation in hospitals.

T cell-mediated tumor killing sensitivity gene signature-based prognostic score for acute myeloid leukemia

BACKGROUND AND OBJECTIVE

Acute myeloid leukemia (AML) is an aggressive, heterogenous hematopoetic malignancies with poor long-term prognosis. T-cell mediated tumor killing plays a key role in tumor immunity. Here, we explored the prognostic performance and functional significance of a T-cell mediated tumor killing sensitivity gene (GSTTK)-based prognostic score (TTKPI).

METHODS

Publicly available transcriptomic data for AML were obtained from TCGA and NCBI-GEO. GSTTK were identified from the TISIDB database. Signature GSTTK for AML were identified by differential expression analysis, COX proportional hazards and LASSO regression analysis and a comprehensive TTKPI score was constructed. Prognostic performance of the TTKPI was examined using Kaplan-Meier survival analysis, Receiver operating curves, and nomogram analysis. Association of TTKPI with clinical phenotypes, tumor immune cell infiltration patterns, checkpoint expression patterns were analysed. Drug docking was used to identify important candidate drugs based on the TTKPI-component genes.

RESULTS

From 401 differentially expressed GSTTK in AML, 24 genes were identified as signature genes and used to construct the TTKPI score. High-TTKPI risk score predicted worse survival and good prognostic accuracy with AUC values ranging from 75 to 96%. Higher TTKPI scores were associated with older age and cancer stage, which showed improved prognostic performance when combined with TTKPI. High TTKPI was associated with lower naïve CD4 T cell and follicular helper T cell infiltrates and higher M2 macrophages/monocyte infiltration. Distinct patterns of immune checkpoint expression corresponded with TTKPI score groups. Three agents; DB11791 (Capmatinib), DB12886 (GSK-1521498) and DB14773 (Lifirafenib) were identified as candidates for AML.

CONCLUSION

A T-cell mediated killing sensitivity gene-based prognostic score TTKPI showed good accuracy in predicting survival in AML. TTKPI corresponded to functional and immunological features of the tumor microenvironment including checkpoint expression patterns and should be investigated for precision medicine approaches.

Predictive model for non-malignant portal vein thrombosis associated with cirrhosis based on inflammatory biomarkers

BACKGROUND

Portal vein thrombosis (PVT), a complication of liver cirrhosis, is a major public health concern. PVT prediction is the most effective method for PVT diagnosis and treatment.

AIM

To develop and validate a nomogram and network calculator based on clinical indicators to predict PVT in patients with cirrhosis.

METHODS

Patients with cirrhosis hospitalized between January 2016 and December 2021 at the First Hospital of Lanzhou University were screened and 643 patients with cirrhosis who met the eligibility criteria were retrieved. Following a 1:1 propensity score matching 572 patients with cirrhosis were screened, and relevant clinical data were collected. PVT risk factors were identified using the least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression analysis. Variance inflation factors and correlation matrix plots were used to analyze multicollinearity among the variables. A nomogram was constructed to predict the probability of PVT based on independent risk factors for PVT, and its predictive performance was verified using a receiver operating characteristic curve (ROC), calibration curves, and decision curve analysis (DCA). Finally, a network calculator was constructed based on the nomograms.

RESULTS

This study enrolled 286 cirrhosis patients with PVT and 286 without PVT. LASSO analysis revealed 13 variables as strongly associated with PVT occurrence. Multivariate logistic regression analysis revealed nine indicators as independent PVT risk factors, including etiology, ascites, gastroesophageal varices, platelet count, D-dimer, portal vein diameter, portal vein velocity, aspartate transaminase to neutrophil ratio index, and platelet-to-lymphocyte ratio. LASSO and correlation matrix plot results revealed no significant multicollinearity or correlation among the variables. A nomogram was constructed based on the screened independent risk factors. The nomogram had excellent predictive performance, with an area under the ROC curve of 0.821 and 0.829 in the training and testing groups, respectively. Calibration curves and DCA revealed its good clinical performance. Finally, the optimal cutoff value for the total nomogram score was 0.513. The sensitivity and specificity of the optimal cutoff values were 0.822 and 0.706, respectively.

CONCLUSION

A nomogram for predicting PVT occurrence was successfully developed and validated, and a network calculator was constructed. This can enable clinicians to rapidly and easily identify high PVT risk groups.

Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores

Theoretical guarantees for causal inference using propensity scores are partially based on the scores behaving like conditional probabilities. However, scores between zero and one do not necessarily behave like probabilities, especially when output by flexible statistical estimators. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established postprocessing method to calibrate the propensity scores. We observe that postcalibration reduces the error in effect estimation and that larger improvements in calibration result in larger improvements in effect estimation. Specifically, we find that expressive tree-based estimators, which are often less calibrated than logistic regression-based models initially, tend to show larger improvements relative to logistic regression-based models. Given the improvement in effect estimation and that postcalibration is computationally cheap, we recommend its adoption when modeling propensity scores with expressive models.

Development and Validation of Machine Learning Algorithms to Predict 1-Year Ischemic Stroke and Bleeding Events in Patients with Atrial Fibrillation and Cancer

In this study, we leveraged machine learning (ML) approach to develop and validate new assessment tools for predicting stroke and bleeding among patients with atrial fibrillation (AFib) and cancer. We conducted a retrospective cohort study including patients who were newly diagnosed with AFib with a record of cancer from the 2012-2018 Surveillance, Epidemiology, and End Results (SEER)-Medicare database. The ML algorithms were developed and validated separately for each outcome by fitting elastic net, random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and neural network models with tenfold cross-validation (train:test = 7:3). We obtained area under the curve (AUC), sensitivity, specificity, and F2 score as performance metrics. Model calibration was assessed using Brier score. In sensitivity analysis, we resampled data using Synthetic Minority Oversampling Technique (SMOTE). Among 18,388 patients with AFib and cancer, 523 (2.84%) had ischemic stroke and 221 (1.20%) had major bleeding within one year after AFib diagnosis. In prediction of ischemic stroke, RF significantly outperformed other ML models [AUC (0.916, 95% CI 0.887-0.945), sensitivity 0.868, specificity 0.801, F2 score 0.375, Brier score = 0.035]. However, the performance of ML algorithms in prediction of major bleeding was low with highest AUC achieved by RF (0.623, 95% CI 0.554-0.692). RF models performed better than CHA2DS2-VASc and HAS-BLED scores. SMOTE did not improve the performance of the ML algorithms. Our study demonstrated a promising application of ML in stroke prediction among patients with AFib and cancer. This tool may be leveraged in assisting clinicians to identify patients at high risk of stroke and optimize treatment decisions.

Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study

While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.

Early prediction of mortality at sepsis diagnosis time in critically ill patients by using interpretable machine learning

This study applied machine learning for the early prediction of 30-day mortality at sepsis diagnosis time in critically ill patients. Retrospective study using data collected from the Medical Information Mart for Intensive Care IV database. The data of the patient cohort was divided on the basis of the year of hospitalization, into training (2008-2013), validation (2014-2016), and testing (2017-2019) datasets. 24,377 patients with the sepsis diagnosis time < 24 h after intensive care unit (ICU) admission were included. A gradient boosting tree-based algorithm (XGBoost) was used for training the machine learning model to predict 30-day mortality at sepsis diagnosis time in critically ill patients. Model performance was measured in both discrimination and calibration aspects. The model was interpreted using the SHapley Additive exPlanations (SHAP) module. The 30-day mortality rate of the testing dataset was 17.9%, and 39 features were selected for the machine learning model. Model performance on the testing dataset achieved an area under the receiver operating characteristic curve (AUROC) of 0.853 (95% CI 0.837-0.868) and an area under the precision-recall curves of 0.581 (95% CI 0.541-0.619). The calibration plot for the model revealed a slope of 1.03 (95% CI 0.94-1.12) and intercept of 0.14 (95% CI 0.04-0.25). The SHAP revealed the top three most significant features, namely age, increased red blood cell distribution width, and respiratory rate. Our study demonstrated the feasibility of using the interpretable machine learning model to predict mortality at sepsis diagnosis time.

A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines

Background: There are algorithms to predict the risk of SARS-CoV-2-related complications. Given the spread of anti-COVID vaccination, which sensibly modified the burden of risk of the infection, these tools need to be re-calibrated. Therefore, we updated our vulnerability index, namely, the Health Search (HS)-CoVulnerabiltyIndex (VI)d (HS-CoVId), to predict the risk of SARS-CoV-2-related hospitalization/death in the primary care setting. Methods: We formed a cohort of individuals aged ≥15 years and diagnosed with COVID-19 between 1 January and 31 December 2021 in the HSD. The date of COVID-19 diagnosis was the study index date. These patients were eligible if they had received an anti-COVID vaccine at least 15 days before the index date. Patients were followed up from the index date until one of the following events, whichever came first: COVID-19-related hospitalization/death (event date), end of registration with their GPs, and end of the study period (31 December 2022). To calculate the incidence rate of COVID-19-related hospitalization/death, a patient-specific score was derived through linear combination of the coefficients stemming from a multivariate Cox regression model. Its prediction performance was evaluated by obtaining explained variation, discrimination, and calibration measures. Results: We identified 2192 patients who had received an anti-COVID vaccine from 1 January to 31 December 2021. With this cohort, we re-calibrated the HS-CoVId by calculating optimism-corrected pseudo-R2, AUC, and calibration slope. The final model reported a good predictive performance by explaining 58% (95% CI: 48-71%) of variation in the occurrence of hospitalizations/deaths, the AUC was 83 (95% CI: 77-93%), and the calibration slope did not reject the equivalence hypothesis (p-value = 0.904). Conclusions: Two versions of HS-CoVId need to be differentially adopted to assess the risk of COVID-19-related complications among vaccinated and unvaccinated subjects. Therefore, this functionality should be operationalized in related patient- and population-based informatic tools intended for general practitioners.

Emergency Department Utilization for Pediatric Gastrostomy Tubes Across the United States

INTRODUCTION

Emergency Department (ED) visits for gastrostomy tube complications in children represent a substantial health-care burden, and many ED visits are potentially preventable. The number and nature of ED visits to community hospitals for pediatric gastrostomy tube complications is unknown.

METHODS

Using the 2019 Nationwide Emergency Department Sample, we performed a retrospective cross-sectional analysis of pediatric patients (<18 y) with a primary diagnosis of gastrostomy tube complication. Our primary outcome was a potentially preventable ED visit, defined as an encounter that did not result in any imaging, procedures, or an inpatient admission. Univariate and multivariable logistic regression analyses were used to determine the associations between patient factors and our primary outcome.

RESULTS

We observed 32,036 ED visits at 535 hospitals and 15,165 (47.3%) were potentially preventable. The median (interquartile range) age was 2 (1, 6) years and 17,707 (55%) were male. Compared to White patients, patients with higher odds of potentially preventable visits were Black (adjusted odds ratio (aOR) [95% confidence interval {CI}]: 1.07 [1.05-1.11], P < 0.001) and Hispanic (aOR [95% CI]: 1.05 [1.02-1.08], P = 0.004). Patients with residential zip codes in the first (aOR [95% CI]: 1.08 [1.04, 1.12], P < 0.001), second (aOR [95% CI]: 1.07 [1.03, 1.11], P < 0.001), and third (aOR [95% CI]: 1.09 [1.05, 1.13], P < 0.001) median household income quartiles had higher odds of potentially preventable visits compared to the highest.

CONCLUSIONS

In a nationally representative sample of EDs, 47.3% of visits for pediatric gastrostomy tubes were potentially preventable. Efforts to improve outpatient management are warranted to reduce health-care utilization for these patients.

Nomogram for predicting neutropenia in patients with esophageal, gastric, or colorectal cancer treated by chemotherapy in the first cycle

OBJECTIVES

Development and validation of a predictive model including serum vitamin concentration to estimate the risk of chemotherapy-induced grade 3/4 neutropenia in esophageal cancer, gastric cancer, or colorectal cancer patients who receive the first cycle of chemotherapy.

METHODS

Data from 535 patients treated at the Affiliated Fuyang People's Hospital of Anhui Medical University from January 1, 2020, to March 2, 2022, were used to derive the predictive model. Least absolute shrinkage and selection operator regression analysis was performed to screen potential risk characteristics, and multivariate logistic regression was utilized to investigate efficient factors associated with chemotherapy-induced neutropenia. A nomogram was constructed using this logistic model. This nomogram was then tested on a temporal validation cohort containing 212 consecutive patients.

RESULTS

In the cohort of all 747 eligible patients, grade 3/4 neutropenia incidence was 45.2%. Age, Eastern Cooperative Oncology Group-performance status, neutrophil count, serum albumin, and hemoglobin data were entered into the final model. The performance of the final predictive nomogram was assessed by the area under the receiver operating characteristic curve in both the development and validation datasets. The calibration curves indicated that the estimated risks were accurate. Decision curve analysis for the predictive model exhibited improved clinical practicality.

CONCLUSION

In the present study, we established an accessible risk predictive model and identified valuable serum vitamin concentration parameters associated with chemotherapy-induced neutropenia. The predictive model may improve the grade 3/4 neutropenia risk prediction in patients with gastrointestinal malignancies who receive oxaliplatin- and fluoropyrimidine-based chemotherapy and help physicians make appropriate decisions for disease management.

A review on statistical and machine learning competing risks methods

When modeling competing risks (CR) survival data, several techniques have been proposed in both the statistical and machine learning literature. State-of-the-art methods have extended classical approaches with more flexible assumptions that can improve predictive performance, allow high-dimensional data and missing values, among others. Despite this, modern approaches have not been widely employed in applied settings. This article aims to aid the uptake of such methods by providing a condensed compendium of CR survival methods with a unified notation and interpretation across approaches. We highlight available software and, when possible, demonstrate their usage via reproducible R vignettes. Moreover, we discuss two major concerns that can affect benchmark studies in this context: the choice of performance metrics and reproducibility.

Leveraging artificial intelligence for perioperative cancer risk assessment of oral potentially malignant disorders

Oral potentially malignant disorders (OPMDs) are mucosal conditions with an inherent disposition to develop oral squamous cell carcinoma. Surgical management is the most preferred strategy to prevent malignant transformation in OPMDs, and surgical approaches to treatment include conventional scalpel excision, laser surgery, cryotherapy, and photodynamic therapy. However, in reality, since all patients with OPMDs will not develop oral squamous cell carcinoma in their lifetime, there is a need to stratify patients according to their risk of malignant transformation to streamline surgical intervention for patients with the highest risks. Artificial intelligence (AI) has the potential to integrate disparate factors influencing malignant transformation for robust, precise, and personalized cancer risk stratification of OPMD patients than current methods to determine the need for surgical resection, excision, or re-excision. Therefore, this article overviews existing AI models and tools, presents a clinical implementation pathway, and discusses necessary refinements to aid the clinical application of AI-based platforms for cancer risk stratification of OPMDs in surgical practice.

Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study

INTRODUCTION

Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models.

MATERIAL AND METHODS

We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed.

RESULTS

A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods.

CONCLUSIONS

The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.

Semi-supervised ROC analysis for reliable and streamlined evaluation of phenotyping algorithms

OBJECTIVE

High-throughput phenotyping will accelerate the use of electronic health records (EHRs) for translational research. A critical roadblock is the extensive medical supervision required for phenotyping algorithm (PA) estimation and evaluation. To address this challenge, numerous weakly-supervised learning methods have been proposed. However, there is a paucity of methods for reliably evaluating the predictive performance of PAs when a very small proportion of the data is labeled. To fill this gap, we introduce a semi-supervised approach (ssROC) for estimation of the receiver operating characteristic (ROC) parameters of PAs (eg, sensitivity, specificity).

MATERIALS AND METHODS

ssROC uses a small labeled dataset to nonparametrically impute missing labels. The imputations are then used for ROC parameter estimation to yield more precise estimates of PA performance relative to classical supervised ROC analysis (supROC) using only labeled data. We evaluated ssROC with synthetic, semi-synthetic, and EHR data from Mass General Brigham (MGB).

RESULTS

ssROC produced ROC parameter estimates with minimal bias and significantly lower variance than supROC in the simulated and semi-synthetic data. For the 5 PAs from MGB, the estimates from ssROC are 30% to 60% less variable than supROC on average.

DISCUSSION

ssROC enables precise evaluation of PA performance without demanding large volumes of labeled data. ssROC is also easily implementable in open-source R software.

CONCLUSION

When used in conjunction with weakly-supervised PAs, ssROC facilitates the reliable and streamlined phenotyping necessary for EHR-based research.

Development and validation of a risk calculator for postoperative diplopia following orbital fracture repair in adults

Postoperative diplopia is the most common complication following orbital fracture repair (OFR). Existing evidence on its risk factors is based on single-institution studies and small sample sizes. Our study is the first multi-center study to develop and validate a risk calculator for the prediction of postoperative diplopia following OFR. We reviewed trauma patients who underwent OFR at two high-volume trauma centers (2015-2019). Excluded were patients < 18 years old and those with postoperative follow-up < 2 weeks. Our primary outcome was incidence/persistence of postoperative diplopia at ≥ 2 weeks. A risk model for the prediction of postoperative diplopia was derived using a development dataset (70% of population) and validated using a validation dataset (remaining 30%). The C-statistic and Hosmer-Lemeshow tests were used to assess the risk model accuracy. A total of n = 254 adults were analyzed. The factors that predicted postoperative diplopia were: age at injury, preoperative enophthalmos, fracture size/displacement, surgical timing, globe/soft tissue repair, and medial wall involvement. Our predictive model had excellent discrimination (C-statistic = 80.4%), calibration (P = 0.2), and validation (C-statistic = 80%). Our model rules out postoperative diplopia with a 100% sensitivity and negative predictive value (NPV) for a probability < 8.9%. Our predictive model rules out postoperative diplopia with an 87.9% sensitivity and a 95.8% NPV for a probability < 13.4%. We designed the first validated risk calculator that can be used as a powerful screening tool to rule out postoperative diplopia following OFR in adults.

Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images

This study aimed to assess the performance of an artificial intelligence (AI) model for predicting clinical pregnancy using enhanced inner cell mass (ICM) and trophectoderm (TE) images. In this retrospective study, we included static images of 2555 day-5-blastocysts from seven in vitro fertilization centers in South Korea. The main outcome of the study was the predictive capability of the model to detect clinical pregnancies (gestational sac). Compared with the original embryo images, the use of enhanced ICM and TE images improved the average area under the receiver operating characteristic curve for the AI model from 0.716 to 0.741. Additionally, a gradient-weighted class activation mapping analysis demonstrated that the enhanced image-trained AI model was able to extract features from crucial areas of the embryo in 99% (506/512) of the cases. Particularly, it could extract the ICM and TE. In contrast, the AI model trained on the original images focused on the main areas in only 86% (438/512) of the cases. Our results highlight the potential efficacy of using ICM- and TE-enhanced embryo images when training AI models to predict clinical pregnancy.

Heart rate complexity helps mortality prediction in the intensive care unit: A pilot study using artificial intelligence

BACKGROUND

In intensive care units (ICUs), accurate mortality prediction is crucial for effective patient management and resource allocation. The Simplified Acute Physiology Score II (SAPS-2), though commonly used, relies heavily on comprehensive clinical data and blood samples. This study sought to develop an artificial intelligence (AI) model utilizing key hemodynamic parameters to predict ICU mortality within the first 24 h and assess its performance relative to SAPS-2.

METHODS

We conducted an analysis of select hemodynamic parameters and the structure of heart rate curves to identify potential predictors of ICU mortality. A machine-learning model was subsequently trained and validated on distinct patient cohorts. The AI algorithm's performance was then compared to the SAPS-2, focusing on classification accuracy, calibration, and generalizability.

MEASUREMENTS AND MAIN RESULTS

The study included 1298 ICU admissions from March 27th, 2015, to March 27th, 2017. An additional cohort from 2022 to 2023 comprised 590 patients, resulting in a total dataset of 1888 patients. The observed mortality rate stood at 24.0%. Key determinants of mortality were the Glasgow Coma Scale score, heart rate complexity, patient age, duration of diastolic blood pressure below 50 mmHg, heart rate variability, and specific mean and systolic blood pressure thresholds. The AI model, informed by these determinants, exhibited a performance profile in predicting mortality that was comparable, if not superior, to the SAPS-2.

CONCLUSIONS

The AI model, which integrates heart rate and blood pressure curve analyses with basic clinical parameters, provides a methodological approach to predict in-hospital mortality in ICU patients. This model offers an alternative to existing tools that depend on extensive clinical data and laboratory inputs. Its potential integration into ICU monitoring systems may facilitate more streamlined mortality prediction processes.

Artificial Intelligence Applications in the Treatment of Colorectal Cancer: A Narrative Review

Colorectal cancer is the third most prevalent cancer worldwide, and its treatment has been a demanding clinical problem. Beyond traditional surgical therapy and chemotherapy, newly revealed molecular mechanisms diversify therapeutic approaches for colorectal cancer. However, the selection of personalized treatment among multiple treatment options has become another challenge in the era of precision medicine. Artificial intelligence has recently been increasingly investigated in the treatment of colorectal cancer. This narrative review mainly discusses the applications of artificial intelligence in the treatment of colorectal cancer patients. A comprehensive literature search was conducted in MEDLINE, EMBASE, and Web of Science to identify relevant papers, resulting in 49 articles being included. The results showed that, based on different categories of data, artificial intelligence can predict treatment outcomes and essential guidance information of traditional and novel therapies, thus enabling individualized treatment strategy selection for colorectal cancer patients. Some frequently implemented machine learning algorithms and deep learning frameworks have also been employed for long-term prognosis prediction in patients with colorectal cancer. Overall, artificial intelligence shows encouraging results in treatment strategy selection and prognosis evaluation for colorectal cancer patients.

Construction and evaluation of neonatal respiratory failure risk prediction model for neonatal respiratory distress syndrome

BACKGROUND

Neonatal respiratory distress syndrome (NRDS) is a common respiratory disease in preterm infants, often accompanied by respiratory failure. The aim of this study was to establish and validate a nomogram model for predicting the probability of respiratory failure in NRDS patients.

METHODS

Patients diagnosed with NRDS were extracted from the MIMIC-iv database. The patients were randomly assigned to a training and a validation cohort. Univariate and stepwise Cox regression analyses were used to determine the prognostic factors of NRDS. A nomogram containing these factors was established to predict the incidence of respiratory failure in NRDS patients. The area under the receiver operating characteristic curve (AUC), receiver operating characteristic curve (ROC), calibration curves and decision curve analysis were used to determine the effectiveness of this model.

RESULTS

The study included 2,705 patients with NRDS. Univariate and multivariate stepwise Cox regression analysis showed that the independent risk factors for respiratory failure in NRDS patients were gestational age, pH, partial pressure of oxygen (PO2), partial pressure of carbon dioxide (PCO2), hemoglobin, blood culture, infection, neonatal intracranial hemorrhage, Pulmonary surfactant (PS), parenteral nutrition and respiratory support. Then, the nomogram was constructed and verified.

CONCLUSIONS

This study identified the independent risk factors of respiratory failure in NRDS patients and used them to construct and evaluate respiratory failure risk prediction model for NRDS. The present findings provide clinicians with the judgment of patients with respiratory failure in NRDS and help clinicians to identify and intervene in the early stage.

Comparing linear discriminant analysis and supervised learning algorithms for binary classification-A method comparison study

In psychology, linear discriminant analysis (LDA) is the method of choice for two-group classification tasks based on questionnaire data. In this study, we present a comparison of LDA with several supervised learning algorithms. In particular, we examine to what extent the predictive performance of LDA relies on the multivariate normality assumption. As nonparametric alternatives, the linear support vector machine (SVM), classification and regression tree (CART), random forest (RF), probabilistic neural network (PNN), and the ensemble k conditional nearest neighbor (EkCNN) algorithms are applied. Predictive performance is determined using measures of overall performance, discrimination, and calibration, and is compared in two reference data sets as well as in a simulation study. The reference data are Likert-type data, and comprise 5 and 10 predictor variables, respectively. Simulations are based on the reference data and are done for a balanced and an unbalanced scenario in each case. In order to compare the algorithms' performance, data are simulated from multivariate distributions with differing degrees of nonnormality. Results differ depending on the specific performance measure. The main finding is that LDA is always outperformed by RF in the bimodal data with respect to overall performance. Discriminative ability of the RF algorithm is often higher compared to LDA, but its model calibration is usually worse. Still LDA mostly ranges second in cases it is outperformed by another algorithm, or the differences are only marginal. In consequence, we still recommend LDA for this type of application.

Transportability of bacterial infection prediction models for critically ill patients

OBJECTIVE

Bacterial infections (BIs) are common, costly, and potentially life-threatening in critically ill patients. Patients with suspected BIs may require empiric multidrug antibiotic regimens and therefore potentially be exposed to prolonged and unnecessary antibiotics. We previously developed a BI risk model to augment practices and help shorten the duration of unnecessary antibiotics to improve patient outcomes. Here, we have performed a transportability assessment of this BI risk model in 2 tertiary intensive care unit (ICU) settings and a community ICU setting. We additionally explored how simple multisite learning techniques impacted model transportability.

METHODS

Patients suspected of having a community-acquired BI were identified in 3 datasets: Medical Information Mart for Intensive Care III (MIMIC), Northwestern Medicine Tertiary (NM-T) ICUs, and NM "community-based" ICUs. ICU encounters from MIMIC and NM-T datasets were split into 70/30 train and test sets. Models developed on training data were evaluated against the NM-T and MIMIC test sets, as well as NM community validation data.

RESULTS

During internal validations, models achieved AUROCs of 0.78 (MIMIC) and 0.81 (NM-T) and were well calibrated. In the external community ICU validation, the NM-T model had robust transportability (AUROC 0.81) while the MIMIC model transported less favorably (AUROC 0.74), likely due to case-mix differences. Multisite learning provided no significant discrimination benefit in internal validation studies but offered more stability during transport across all evaluation datasets.

DISCUSSION

These results suggest that our BI risk models maintain predictive utility when transported to external cohorts.

CONCLUSION

Our findings highlight the importance of performing external model validation on myriad clinically relevant populations prior to implementation.

Calibrating machine learning approaches for probability estimation: A comprehensive comparison

Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies.

Adjusting for Congenital Heart Surgery Risk Using Administrative Data

BACKGROUND

Congenital heart surgery (CHS) encompasses a heterogeneous population of patients and surgeries. Risk standardization models that adjust for patient and procedural characteristics can allow for collective study of these disparate patients and procedures.

OBJECTIVES

We sought to develop a risk-adjustment model for CHS using the newly developed Risk Stratification for Congenital Heart Surgery for ICD-10 Administrative Data (RACHS-2) methodology.

METHODS

Within the Kids' Inpatient Database 2019, we identified all CHSs that could be assigned a RACHS-2 score. Hierarchical logistic regression (clustered on hospital) was used to identify patient and procedural characteristics associated with in-hospital mortality. Model validation was performed using data from 24 State Inpatient Databases during 2017.

RESULTS

Of 5,902,538 total weighted hospital discharges in the Kids' Inpatient Database 2019, 22,310 pediatric cardiac surgeries were identified and assigned a RACHS-2 score. In-hospital mortality occurred in 543 (2.4%) of cases. Using only RACHS-2, the mortality mode had a C-statistic of 0.81 that improved to 0.83 with the addition of age. A final multivariable model inclusive of RACHS-2, age, payer, and presence of a complex chronic condition outside of congenital heart disease further improved model discrimination to 0.87 (P < 0.001). Discrimination in the validation cohort was also very good with a C-statistic of 0.83.

CONCLUSIONS

We created and validated a risk-adjustment model for CHS that accounts for patient and procedural characteristics associated with in-hospital mortality available in administrative data, including the newly developed RACHS-2. Our risk model will be critical for use in health services research and quality improvement initiatives.

Deep Learning of Electrocardiograms in Sinus Rhythm From US Veterans to Predict Atrial Fibrillation

Importance

Early detection of atrial fibrillation (AF) may help prevent adverse cardiovascular events such as stroke. Deep learning applied to electrocardiograms (ECGs) has been successfully used for early identification of several cardiovascular diseases.

Objective

To determine whether deep learning models applied to outpatient ECGs in sinus rhythm can predict AF in a large and diverse patient population.

Design, Setting, and Participants

This prognostic study was performed on ECGs acquired from January 1, 1987, to December 31, 2022, at 6 US Veterans Affairs (VA) hospital networks and 1 large non-VA academic medical center. Participants included all outpatients with 12-lead ECGs in sinus rhythm.

Main Outcomes and Measures

A convolutional neural network using 12-lead ECGs from 2 US VA hospital networks was trained to predict the presence of AF within 31 days of sinus rhythm ECGs. The model was tested on ECGs held out from training at the 2 VA networks as well as 4 additional VA networks and 1 large non-VA academic medical center.

Results

A total of 907 858 ECGs from patients across 6 VA sites were included in the analysis. These patients had a mean (SD) age of 62.4 (13.5) years, 6.4% were female, and 93.6% were male, with a mean (SD) CHA2DS2-VASc (congestive heart failure, hypertension, age, diabetes mellitus, prior stroke or transient ischemic attack or thromboembolism, vascular disease, age, sex category) score of 1.9 (1.6). A total of 0.2% were American Indian or Alaska Native, 2.7% were Asian, 10.7% were Black, 4.6% were Latinx, 0.7% were Native Hawaiian or Other Pacific Islander, 62.4% were White, 0.4% were of other race or ethnicity (which is not broken down into subcategories in the VA data set), and 18.4% were of unknown race or ethnicity. At the non-VA academic medical center (72 483 ECGs), the mean (SD) age was 59.5 (15.4) years and 52.5% were female, with a mean (SD) CHA2DS2-VASc score of 1.6 (1.4). A total of 0.1% were American Indian or Alaska Native, 7.9% were Asian, 9.4% were Black, 2.9% were Latinx, 0.03% were Native Hawaiian or Other Pacific Islander, 74.8% were White, 0.1% were of other race or ethnicity, and 4.7% were of unknown race or ethnicity. A deep learning model predicted the presence of AF within 31 days of a sinus rhythm ECG on held-out test ECGs at VA sites with an area under the receiver operating characteristic curve (AUROC) of 0.86 (95% CI, 0.85-0.86), accuracy of 0.78 (95% CI, 0.77-0.78), and F1 score of 0.30 (95% CI, 0.30-0.31). At the non-VA site, AUROC was 0.93 (95% CI, 0.93-0.94); accuracy, 0.87 (95% CI, 0.86-0.88); and F1 score, 0.46 (95% CI, 0.44-0.48). The model was well calibrated, with a Brier score of 0.02 across all sites. Among individuals deemed high risk by deep learning, the number needed to screen to detect a positive case of AF was 2.47 individuals for a testing sensitivity of 25% and 11.48 for 75%. Model performance was similar in patients who were Black, female, or younger than 65 years or who had CHA2DS2-VASc scores of 2 or greater.

Conclusions and Relevance

Deep learning of outpatient sinus rhythm ECGs predicted AF within 31 days in populations with diverse demographics and comorbidities. Similar models could be used in future AF screening efforts to reduce adverse complications associated with this disease.

The predictive accuracy of coronary heart disease risk prediction models in rural Northwestern China

Cardiovascular risk models developed may have limitations when applied to rural Chinese. This study validated and compared the Framingham Risk Score (FRS) and Prediction for Atherosclerotic Cardiovascular Disease Risk in China (PAR) models in predicting 10-year risk of coronary heart disease (CHD) in a rural cohort in Ningxia, China from 2008 to 2019. The FRS and PAR models were validated by estimating predicted events, C index, calibration χ2 and plots. 1381 adults without CHD at baseline were followed up for 9.75 years on average. 168 CHD cases were observed. The FRS and PAR underestimated CHD events by 22 % and 46 % for the total population, while overestimated for males by 152 % and 78 %, respectively. The C index was slightly higher for PAR than FRS. Both models showed weak calibration with chi-square values above 20 (p < 0.001). Bland-Altman plots indicated FRS predicted higher CHD risk than PAR, lacking consistency. Overall, FRS and PAR demonstrated limited performance in predicting 10-year CHD risk in this rural population. PAR had slightly better discrimination than FRS, but require further improvement in calibration and individual risk estimation to suit the rural population in Northwest China.

Impact of COVID-19 infection experience on mental health status of intensive care unit patients' family members: a real-world study

PURPOSE

Family members of patients hospitalized in intensive care unit (ICU) are susceptible to adverse psychological outcomes. However, there is a paucity of studies specifically examining the mental health symptoms in ICU patients' family members with a prior history of coronavirus disease 2019 (COVID-19) infection.

AIM

This study aimed to investigate mental health status and its influencing factors of ICU patients' family members with previous COVID-19 infection experience in China.

DESIGN

Nationwide, cross-sectional cohort of consecutive participants of family members of ICU patients from 10 provinces randomly selected in mainland China conducted between October 2022 and May 2023.

METHODS

The basic information scale, Self-rating depression scale, Self-rating Anxiety Scale, Impact of Event Scale-Revised, Pittsburgh sleep quality index, Perceived Stress Scale, Connor-Davidson resilience scale, Simplified Coping Style Questionnaire were employed to explore mental health status among participants.

RESULTS

A total of 463 participants, comprising 156 individuals in Covid-19 family member cohort (infection group) and 307 individuals in control family member cohort (control group), met inclusion criteria. The infection group exhibited significantly higher incidence of composite mental health symptoms compared to control group (P = 0.017). Multivariable logistic regression analysis revealed that being female, engaging in physical/mental labor, residing in rural areas, and having children were identified as risk factors for the development of depression, anxiety, and post-traumatic stress disorder symptoms, while medical history of surgery was protective factor. A predictive model demonstrated a favorable discriminative ability and excellent calibration.

CONCLUSION

COVID-19 infection experience regarded as new traumatic stressors worsen mental health status of ICU patients' family members.

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Importance

Access to routine dental care prevents advanced dental disease and improves oral and overall health. Identifying individuals at risk of foregoing preventive dental care can direct prevention efforts toward high-risk populations.

Objective

To predict foregone preventive dental care among adults overall and in sociodemographic subgroups and to assess the algorithmic fairness.

Design, Setting, and Participants

This prognostic study was a secondary analyses of longitudinal data from the US Medical Expenditure Panel Survey (MEPS) from 2016 to 2019, each with 2 years of follow-up. Participants included adults aged 18 years and older. Data analysis was performed from December 2022 to June 2023.

Exposure

A total of 50 predictors, including demographic and socioeconomic characteristics, health conditions, behaviors, and health services use, were assessed.

Main Outcomes and Measures

The outcome of interest was foregoing preventive dental care, defined as either cleaning, general examination, or an appointment with the dental hygienist, in the past year.

Results

Among 32 234 participants, the mean (SD) age was 48.5 (18.2) years and 17 386 participants (53.9%) were female; 1935 participants (6.0%) were Asian, 5138 participants (15.9%) were Black, 7681 participants (23.8%) were Hispanic, 16 503 participants (51.2%) were White, and 977 participants (3.0%) identified as other (eg, American Indian and Alaska Native) or multiple racial or ethnic groups. There were 21 083 (65.4%) individuals who missed preventive dental care in the past year. The algorithms demonstrated high performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.84 (95% CI, 0.84-0.85) in the overall population. While the full sample model performed similarly when applied to White individuals and older adults (AUC, 0.88; 95% CI, 0.87-0.90), there was a loss of performance for other subgroups. Removing the subgroup-sensitive predictors (ie, race and ethnicity, age, and income) did not impact model performance. Models stratified by race and ethnicity performed similarly or worse than the full model for all groups, with the lowest performance for individuals who identified as other or multiple racial groups (AUC, 0.76; 95% CI, 0.70-0.81). Previous pattern of dental visits, health care utilization, dental benefits, and sociodemographic characteristics were the highest contributing predictors to the models' performance.

Conclusions and Relevance

Findings of this prognostic study using cohort data suggest that tree-based ensemble machine learning models could accurately predict adults at risk of foregoing preventive dental care and demonstrated bias against underrepresented sociodemographic groups. These results highlight the importance of evaluating model fairness during development and testing to avoid exacerbating existing biases.

Development and Validation of a Nomogram to Predict the Risk of Recurrent Lower Extremity Radiating Pain Within 1 Week Following Full-Endoscopic Lumbar Discectomy

BACKGROUND

Accurately predicting the risk of lower extremity (LE) radiating pain after surgery is an important endeavor for spinal surgeons. Our study aimed to identify risk factors for LE radiating pain after decompression with full-endoscopic lumbar discectomy (FELD) and develop a nomogram.

METHODS

We retrospectively reviewed the medical data of patients with lumbar disc herniation who underwent FELD. Two hundred thirty-five patients diagnosed at our hospital from January 2015 to December 2020 were used for model development. The independent risk factors for LE radiating pain after surgery were determined by least absolute shrinkage and selection operator logistic regression and multivariate logistic regression analysis. A nomogram was developed to predict the risk of LE radiating pain based on independent risk factors. Receiver operating characteristic curve, calibration curve, and decision curve analyses were used to evaluate the predictive performance. The nomogram was further verified by an independent cohort.

RESULTS

Three hundred seventy-five patients were enrolled in this study, with 102 patients in the training cohort reporting LE radiating pain after FELD, while 133 patients did not. In the validation cohort, 57 patients reported LE radiating pain after FELD, while 83 patients did not. The model was established by multivariate logistic regression analysis. The risk factors included a higher Michigan State University classification of herniated discs, increased disease course, increased time of surgery, reduced lateral recess width, and an interlaminar surgical approach, compared to transforaminal approach. The C-indices and the area under the receiver operating characteristic curve of the predictive model demonstrated good discrimination. Good predictive performance and accuracy were also observed in the validation cohort.

CONCLUSIONS

A novel nomogram for predicting recurrent LE radiating pain within 1 week after FELD was established and validated. More aggressive pain management strategies should be considered for patients at high risk of LE radiating pain after surgery, as predicted by this model.

Low dose tacrolimus exposure and early steroid withdrawal with strict body weight control can improve post kidney transplant glucose tolerance in Japanese patients

The development of diabetes mellitus (DM) after living donor kidney transplantation (KT) is a risk factor for worsening transplant kidney function, cardiac disease, and cerebrovascular disease, which may affect prognosis after KT. At our institution, all patients' glucose tolerance is evaluated perioperatively by oral glucose tolerance tests (OGTTs) at pre-KT, and 3, 6, and 12 month (mo.) after KT. We analyzed the insulinogenic index (ISI) and homeostasis model assessment beta cell (HOMA-β) based on the immunoreactive insulin (IRI) levels to determine how glucose tolerance changed after KT in 214 patients who had not been diagnosed with DM before KT. In addition, we analyzed the body mass index (BMI) which may also influence glucose tolerance after KT. The concentration of tacrolimus (TAC) in blood was also measured as the area under the curve (AUC) to examine its effects at each sampling point. The preoperative-OGTTs showed that DM was newly diagnosed in 22 of 214 patients (10.3%) who had not been given a diagnosis of DM by the pre-KT fasting blood sugar (FBS) tests. The glucose tolerance was improved in 15 of 22 DM patients at 12 mo. after KT. ISI and IRI deteriorated only at 3 mo. after KT but improved over time. There was a trend of an inverse correlation between HOMA-β and TAC-AUC. We also found inverse correlations between IRI and an increase in BMI from 3 to 12 mo. after KT. Early corticosteroid withdrawal or the steroid minimization protocol with tacrolimus to maintain a low level of diabetogenic tacrolimus and BMI decrease after KT used by our hospital individualizes lifestyle interventions for each patient might contribute to an improvement in post-KT glucose tolerance.

Temporal validation of the MMCD score to predict kidney replacement therapy and in-hospital mortality in COVID-19 patients

BACKGROUND

Acute kidney injury has been described as a common complication in patients hospitalized with COVID-19, which may lead to the need for kidney replacement therapy (KRT) in its most severe forms. Our group developed and validated the MMCD score in Brazilian COVID-19 patients to predict KRT, which showed excellent performance using data from 2020. This study aimed to validate the MMCD score in a large cohort of patients hospitalized with COVID-19 in a different pandemic phase and assess its performance to predict in-hospital mortality.

METHODS

This study is part of the "Brazilian COVID-19 Registry", a retrospective observational cohort of consecutive patients hospitalized for laboratory-confirmed COVID-19 in 25 Brazilian hospitals between March 2021 and August 2022. The primary outcome was KRT during hospitalization and the secondary was in-hospital mortality. We also searched literature for other prediction models for KRT, to assess the results in our database. Performance was assessed using area under the receiving operator characteristic curve (AUROC) and the Brier score.

RESULTS

A total of 9422 patients were included, 53.8% were men, with a median age of 59 (IQR 48-70) years old. The incidence of KRT was 8.8% and in-hospital mortality was 18.1%. The MMCD score had excellent discrimination and overall performance to predict KRT (AUROC: 0.916 [95% CI 0.909-0.924]; Brier score = 0.057). Despite the excellent discrimination and overall performance (AUROC: 0.922 [95% CI 0.914-0.929]; Brier score = 0.100), the calibration was not satisfactory concerning in-hospital mortality. A random forest model was applied in the database, with inferior performance to predict KRT requirement (AUROC: 0.71 [95% CI 0.69-0.73]).

CONCLUSION

The MMCD score is not appropriate for in-hospital mortality but demonstrates an excellent predictive ability to predict KRT in COVID-19 patients. The instrument is low cost, objective, fast and accurate, and can contribute to supporting clinical decisions in the efficient allocation of assistance resources in patients with COVID-19.

Clinical Grading Scales and Neuroprognostication in Acute Brain Injury

Prediction of neurological clinical outcome after acute brain injury is critical because it helps guide discussions with patients and families and informs treatment plans and allocation of resources. Numerous clinical grading scales have been published that aim to support prognostication after acute brain injury. However, the development and validation of clinical scales lack a standardized approach. This in turn makes it difficult for clinicians to rely on prognostic grading scales and to integrate them into clinical practice. In this review, we discuss quality measures of score development and validation and summarize available scales to prognosticate outcomes after acute brain injury. These include scales developed for patients with coma, cardiac arrest, ischemic stroke, nontraumatic intracerebral hemorrhage, subarachnoid hemorrhage, and traumatic brain injury; for each scale, we discuss available validation studies.

A convolutional neural network-based, quantitative complete blood count scattergram-mapping framework promptly screens acute promyelocytic leukemia with high sensitivity

BACKGROUND

Acute promyelocytic leukemia (APL) is a subtype of acute myeloid leukemia (AML) characterized by its rapidly progressive and fatal clinical course if untreated, although it is curable if treated in a timely manner. Promptly screening patients who have results that are suspicious for APL is vital to overcome early death.

METHODS

The authors developed an innovative framework consisting of ResNet-18, a convolutional neural network architecture, with the objective of quantitatively mapping a complete blood count (CBC) scattergram to quickly and robustly indicate a probable susceptibility to APL. Three hundred and twenty scattergrams of the white blood cell differential channel from 51 patients with APL, 510 scattergrams from 105 patients who had non-APL AML, and 320 scattergrams from 320 healthy controls were randomly stratified at a ratio of 4:1 and split into training and testing data sets to accomplish five-fold cross-validation.

RESULTS

Both the area under the curve and the average precision of >0.99 were achieved in each fold. Three hundred four of the 320 APL scattergrams (95%) were correctly flagged by the model, which outcompeted the CBC review rules recommended by the International Society of Laboratory Hematology (all p < .001). External validation based on an independent testing data set that included 56 scattergrams from 31 patients with APL, 56 scattergrams from 55 patients with non-APL AML, and 64 scattergrams from 64 healthy controls also confirmed the sensitivity and specificity of the framework.

CONCLUSIONS

To the authors' knowledge, their convolutional neural network-based framework is the first to use scattergram output from routine CBC analysis to map suspicious APL early with outstanding sensitivity, specificity, and precision. The authors also describe a new CBC workflow incorporating this framework upstream of the morphologic review, which would provide the earliest flag for APL.

PLAIN LANGUAGE SUMMARY

The authors propose an innovative way to visualize complete blood counts (CBCs) by mapping the difference in white blood cell counts using automated CBC analysis to identify potential acute promyelocytic leukemia (APL) using a convolutional neural network (CNN), which can eliminate the potential pitfalls of manual observation. Analyses of an unprecedented, realistic data set validated that the quantitative relationship between the CBC scattergram and an APL abnormality is highly consistent. This is the first study to date focusing on screening for APL using scattergrams of the difference in white blood cell counts from routine CBC tests and has significant clinical relevance. The authors recommend using this method even before analyzing cell images, which could provide the earliest way to screen for APL in a sensitive and accurate way.

A methodology to assess the population size and estimate the needed resources for new licensed medications by combining clinical and administrative databases: The example of glycated haemoglobin in type 2 diabetes

PURPOSE

To develop and validate a model to estimate glycated haemoglobin (HbA1c) values in patients with type 2 diabetes mellitus (T2DM) using a clinical data source, with the aim to apply this equation to administrative databases.

METHODS

Using a primary care and administrative Italian databases, namely the Health Search database (HSD) and the ReS (Ricerca e Salute) database, we selected all patients aged 18 years or older on 31 December 2018 being diagnosed with T2DM and without prior prescription of sodium-glucose cotransporter-2 (SGLT-2) inhibitors. We included patients prescribed with and adherent to metformin. HSD was used to develop and test (using 2019 data as well) the algorithm imputing HbA1c values ≥7% according to a series of covariates. The algorithm was gathered by combining beta-coefficients being estimated by logistic regression models using complete case (excluding missing values) and imputed (after multiple imputation) dataset. The final algorithm was applied to ReS database using the same covariates.

RESULTS

The tested algorithms were able to explain 17%-18% variation in assessing HbA1c values. Good discrimination (70%) and calibration were obtained as well. The best algorithm (three) cut-offs, namely those providing correct classifications ranging 66%-70% was therefore calculated and applied to ReS database. By doing so, from 52 999 (27.9, 95% CI: 27.7%-28.1%) to 74 250 (40.1%, 95% CI: 38.9%-39.3%) patients were estimated with HbA1c ≥7%.

CONCLUSION

Through this methodology, healthcare authorities should be able to quantify the population eligible to a new licensed medication, such as SGLT-2 inhibitors, and to simulate scenarios to assess reimbursement criteria according to precise estimates.

Machine learning and EEG can classify passive viewing of discrete categories of visual stimuli but not the observation of pain

Previous studies have demonstrated the potential of machine learning (ML) in classifying physical pain from non-pain states using electroencephalographic (EEG) data. However, the application of ML to EEG data to categorise the observation of pain versus non-pain images of human facial expressions or scenes depicting pain being inflicted has not been explored. The present study aimed to address this by training Random Forest (RF) models on cortical event-related potentials (ERPs) recorded while participants passively viewed faces displaying either pain or neutral expressions, as well as action scenes depicting pain or matched non-pain (neutral) scenarios. Ninety-one participants were recruited across three samples, which included a model development group (n = 40) and a cross-subject validation group (n = 51). Additionally, 25 participants from the model development group completed a second experimental session, providing a within-subject temporal validation sample. The analysis of ERPs revealed an enhanced N170 component in response to faces compared to action scenes. Moreover, an increased late positive potential (LPP) was observed during the viewing of pain scenes compared to neutral scenes. Additionally, an enhanced P3 response was found when participants viewed faces displaying pain expressions compared to neutral expressions. Subsequently, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The RF model achieved classification accuracies of 75%, 64%, and 69% for cross-validation, cross-subject, and within-subject classifications, respectively, along with reasonably calibrated predictions for the classification of face versus scene images. However, the RF model was unable to classify pain versus neutral stimuli above chance levels when presented with subsequent tasks involving images from either category. These results expand upon previous findings by externally validating the use of ML in classifying ERPs related to different categories of visual images, namely faces and scenes. The results also indicate the limitations of ML in distinguishing pain and non-pain connotations using ERP responses to the passive viewing of visually similar images.

A generalizable and interpretable model for mortality risk stratification of sepsis patients in intensive care unit

PURPOSE

This study aimed to construct a mortality model for the risk stratification of intensive care unit (ICU) patients with sepsis by applying a machine learning algorithm.

METHODS

Adult patients who were diagnosed with sepsis during admission to ICU were extracted from MIMIC-III, MIMIC-IV, eICU, and Zigong databases. MIMIC-III was used for model development and internal validation. The other three databases were used for external validation. Our proposed model was developed based on the Extreme Gradient Boosting (XGBoost) algorithm. The generalizability, discrimination, and validation of our model were evaluated. The Shapley Additive Explanation values were used to interpret our model and analyze the contribution of individual features.

RESULTS

A total of 16,741, 15,532, 22,617, and 1,198 sepsis patients were extracted from the MIMIC-III, MIMIC-IV, eICU, and Zigong databases, respectively. The proposed model had an area under the receiver operating characteristic curve (AUROC) of 0.84 in the internal validation, which outperformed all the traditional scoring systems. In the external validations, the AUROC was 0.87 in the MIMIC-IV database, better than all the traditional scoring systems; the AUROC was 0.83 in the eICU database, higher than the Simplified Acute Physiology Score III and Sequential Organ Failure Assessment (SOFA),equal to 0.83 of the Acute Physiology and Chronic Health Evaluation IV (APACHE-IV), and the AUROC was 0.68 in the Zigong database, higher than those from the systemic inflammatory response syndrome and SOFA. Furthermore, the proposed model showed the best discriminatory and calibrated capabilities and had the best net benefit in each validation.

CONCLUSIONS

The proposed algorithm based on XGBoost and SHAP-value feature selection had high performance in predicting the mortality of sepsis patients within 24 h of ICU admission.

Machine learning for predicting the survival in osteosarcoma patients: Analysis based on American and Hebei Province cohort

Osteosarcoma, a rare malignant tumor, has a poor prognosis. This study aimed to find the best prognostic model for osteosarcoma. There were 2912 patients included from the SEER database and 225 patients from Hebei Province. Patients from the SEER database (2008-2015) were included in the development dataset. Patients from the SEER database (2004-2007) and Hebei Province cohort were included in the external test datasets. The Cox model and three tree-based machine learning algorithms (survival tree [ST], random survival forest [RSF] and gradient boosting machine [GBM]) were used to develop the prognostic models by 10-fold cross-validation with 200 iterations. Additionally, performance of models in the multivariable group was compared with the TNM group. The 3-year and 5-year cancer specific survival (CSS) were 72.71% and 65.92% in the development dataset, respectively. The predictive ability in the multivariable group was superior to that in the TNM group. The calibration curves and consistency in the multivariable group were superior to those in the TNM group. The Cox and RSF models performed better than the ST and GBM models. A nomogram was constructed to predict the 3-year and 5-year CSS of osteosarcoma patients. The RSF model can be used as a nonparametric alternative to the Cox model. The constructed nomogram based on the Cox model can provide reference for clinicians to formulate specific therapeutic decisions both in America and China.

Predicting whether patients will achieve minimal clinically important differences following hip or knee arthroplasty

Aims

A substantial fraction of patients undergoing knee arthroplasty (KA) or hip arthroplasty (HA) do not achieve an improvement as high as the minimal clinically important difference (MCID), i.e. do not achieve a meaningful improvement. Using three patient-reported outcome measures (PROMs), our aim was: 1) to assess machine learning (ML), the simple pre-surgery PROM score, and logistic-regression (LR)-derived performance in their prediction of whether patients undergoing HA or KA achieve an improvement as high or higher than a calculated MCID; and 2) to test whether ML is able to outperform LR or pre-surgery PROM scores in predictive performance.

Methods

MCIDs were derived using the change difference method in a sample of 1,843 HA and 1,546 KA patients. An artificial neural network, a gradient boosting machine, least absolute shrinkage and selection operator (LASSO) regression, ridge regression, elastic net, random forest, LR, and pre-surgery PROM scores were applied to predict MCID for the following PROMs: EuroQol five-dimension, five-level questionnaire (EQ-5D-5L), EQ visual analogue scale (EQ-VAS), Hip disability and Osteoarthritis Outcome Score-Physical Function Short-form (HOOS-PS), and Knee injury and Osteoarthritis Outcome Score-Physical Function Short-form (KOOS-PS).

Results

Predictive performance of the best models per outcome ranged from 0.71 for HOOS-PS to 0.84 for EQ-VAS (HA sample). ML statistically significantly outperformed LR and pre-surgery PROM scores in two out of six cases.

Conclusion

MCIDs can be predicted with reasonable performance. ML was able to outperform traditional methods, although only in a minority of cases.

Development of an Electronic Health Record-Based Algorithm for Predicting Lung Cancer Screening Eligibility in the Population-Based Research to Optimize the Screening Process Lung Research Consortium

PURPOSE

Lung cancer screening (LCS) guidelines in the United States recommend LCS for those age 50-80 years with at least 20 pack-years smoking history who currently smoke or quit within the last 15 years. We tested the performance of simple smoking-related criteria derived from electronic health record (EHR) data and developed and tested the performance of a multivariable model in predicting LCS eligibility.

METHODS

Analyses were completed within the Population-based Research to Optimize the Screening Process Lung Consortium (PROSPR-Lung). In our primary validity analyses, the reference standard LCS eligibility was based on self-reported smoking data collected via survey. Within one PROSPR-Lung health system, we used a training data set and penalized multivariable logistic regression using the Least Absolute Shrinkage and Selection Operator to select EHR-based variables into the prediction model including demographics, smoking history, diagnoses, and prescription medications. A separate test data set assessed model performance. We also conducted external validation analysis in a separate health system and reported AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy metrics associated with the Youden Index.

RESULTS

There were 14,214 individuals with survey data to assess LCS eligibility in primary analyses. The overall performance for assigning LCS eligibility status as measured by the AUC values at the two health systems was 0.940 and 0.938. At the Youden Index cutoff value, performance metrics were as follows: accuracy, 0.855 and 0.895; sensitivity, 0.886 and 0.920; specificity, 0.896 and 0.850; PPV, 0.357 and 0.444; and NPV, 0.988 and 0.992.

CONCLUSION

Our results suggest that health systems can use an EHR-derived multivariable prediction model to aid in the identification of those who may be eligible for LCS.

Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study

Introduction

Venous thromboembolism (VTE) risk assessment at admission is of great importance for early screening and timely prophylaxis and management during hospitalization. The purpose of this study is to develop and validate novel risk assessment models at admission based on machine learning (ML) methods.

Methods

In this retrospective study, a total of 3078 individuals were included with their Caprini variables within 24 hours at admission. Then several ML models were built, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). The prediction performance of ML models and the Caprini risk score (CRS) was then validated and compared through a series of evaluation metrics.

Results

The values of AUROC and AUPRC were 0.798 and 0.303 for LR, 0.804 and 0.360 for RF, and 0.796 and 0.352 for XGB, respectively, which outperformed CRS significantly (0.714 and 0.180, P < 0.001). When prediction scores were stratified into three risk levels for application, RF could obtain more reasonable results than CRS, including smaller false positive alerts and larger lower-risk proportions. The boosting results of stratification were further verified by the net-reclassification-improvement (NRI) analysis.

Discussion

This study indicated that machine learning models could improve VTE risk prediction at admission compared with CRS. Among the ML models, RF was found to have superior performance and great potential in clinical practice.

Validated Pretreatment Prediction Models for Response to Neoadjuvant Therapy in Patients with Rectal Cancer: A Systematic Review and Critical Appraisal

Pretreatment response prediction is crucial to select those patients with rectal cancer who will benefit from organ preservation strategies following (intensified) neoadjuvant therapy and to avoid unnecessary toxicity in those who will not. The combination of individual predictors in multivariable prediction models might improve predictive accuracy. The aim of this systematic review was to summarize and critically appraise validated pretreatment prediction models (other than radiomics-based models or image-based deep learning models) for response to neoadjuvant therapy in patients with rectal cancer and provide evidence-based recommendations for future research. MEDLINE via Ovid, Embase.com, and Scopus were searched for eligible studies published up to November 2022. A total of 5006 studies were screened and 16 were included for data extraction and risk of bias assessment using Prediction model Risk Of Bias Assessment Tool (PROBAST). All selected models were unique and grouped into five predictor categories: clinical, combined, genetics, metabolites, and pathology. Studies generally included patients with intermediate or advanced tumor stages who were treated with neoadjuvant chemoradiotherapy. Evaluated outcomes were pathological complete response and pathological tumor response. All studies were considered to have a high risk of bias and none of the models were externally validated in an independent study. Discriminative performances, estimated with the area under the curve (AUC), ranged per predictor category from 0.60 to 0.70 (clinical), 0.78 to 0.81 (combined), 0.66 to 0.91 (genetics), 0.54 to 0.80 (metabolites), and 0.71 to 0.91 (pathology). Model calibration outcomes were reported in five studies. Two collagen feature-based models showed the best predictive performance (AUCs 0.83-0.91 and good calibration). In conclusion, some pretreatment models for response prediction in rectal cancer show encouraging predictive potential but, given the high risk of bias in these studies, their value should be evaluated in future, well-designed studies.

The BE-ALIVE score: assessing 30-day mortality risk in patients presenting with acute coronary syndromes

AIM

To create and validate a simple scoring system for predicting 30-day mortality in patients presenting with acute coronary syndromes (ACS) at their moment of admission.

METHODS AND RESULTS

2407 consecutive patients presenting to Harefield Hospital with measured arterial blood gases, from January 2011 to December 2020, were studied to build the training set. 30-day mortality in this group was 17.2%. A scoring algorithm that was built using binary logistic regression of variables available on admission was then converted to an additive risk score. The resultant scoring system is the BE-ALIVE score, which incorporates the following factors:Base Excess (1 point for <-2 mmol/L), Age (<65 years: 0 points, 65-74: 1 point, 75-84: 2 points, ≥85: 3 points), Lactate (<2 mmol/L: 0 points, 2-4.9: 1 point, 5-9.9: 3 points, ≥10: 6 points), Intubated (2 points), Left Ventricular function (mildly impaired or better: -1 point, moderately impaired: 1 point, severely impaired: 3 points) and External/out of hospital cardiac arrest 2 points).The scoring system was validated using a testing set of 515 patients presenting to Harefield Hospital in 2021. The validation metrics were excellent with a c-statistic of 0.9, Brier's score 0.06 vs a naïve classifier of 0.15, Spiegelhalter's z-statistic probability of 0.267 and a calibration slope of 1.08.

CONCLUSION

The BE-ALIVE score is a simple and accurate scoring system to predict 30-day mortality in patients presenting with ACS. Appreciating this mortality risk can allow prompt involvement of appropriate care such as the shock team.

Multilevel modelling of neonatal mortality in Ghana: Does household and community levels matter?

Background

Neonatal mortality accounts for an increasing share of under-five deaths, and they are declining at a slower rate than postnatal deaths. Apparently, neonatal mortality is increasingly becoming a major public health problem in Ghana and the world over. The current study sought to analyze neonatal mortality as a function of predictor variables and to estimate and understand unobserved household and community-level residual effects on neonatal mortality to provide data driven evidence to shape informed policies and interventions aimed at reducing the neonatal mortality burden.

Methods

The current study extracted three-level complex data on 5884 children born in the five years preceding the 2014 Ghana Demographic and Health Survey. A two-level and three-level multilevel logistic models were applied to estimate unobserved household and community-level variations in neonatal mortality in the presence of set of predictor variables. Sampling weights were incorporated in both the descriptive and inferential analysis since the data used emanated from a complex survey. Model fit statistics such as AIC scores for a weighted two-level and three-level random intercept logistic models were compared. The model with the lowest AIC score was considered the most preferred model.

Results

The household-level random intercept model suggested that the odds of neonatal mortality was higher among multiple births [OR = 3.15 (95% CI: 1.17, 8.50)], babies born to mothers who received prenatal care from non-skilled worker [OR = 5.88 (95% CI: 2.90, 11.91)], babies delivered through caesarian section [OR = 2.47 (95% CI: 1.06, 5.79)], a household with 1-4 members [OR = 10.23 (95% CI: 4.17, 25.50)], a short preceding birth interval (<24 months) [OR = 3.05 (95% CI: 1.18, 7.88)], and preceding birth interval between 24 and 47 months [OR = 2.88 (95% CI: 1.41, 5.91)]. Substantial unobserved household-level residual variations in neonatal mortality were observed.

Conclusion

The findings of the current study provide an actionable information to be used by government and other stakeholders in the health sector to renew commitment to reduce neonatal mortality to an acceptable level. There is the need to intensify maternal health education by health providers to encourage pregnant women to visit antenatal clinics at least four times so they could benefit substantially from ANC services.