Comparison of machine learning methods with logistic regression analysis in creating predictive models for risk of critical in-hospital events in COVID-19 patients on hospital admission

An investigation into the impact of temporality on COVID-19 infection and mortality predictions: new perspective based on Shapley Values

INTRODUCTION

Machine learning models have been employed to predict COVID-19 infections and mortality, but many models were built on training and testing sets from different periods. The purpose of this study is to investigate the impact of temporality, i.e., the temporal gap between training and testing sets, on model performances for predicting COVID-19 infections and mortality. Furthermore, this study seeks to understand the causes of the impact of temporality.

METHODS

This study used a COVID-19 surveillance dataset collected from Brazil in year 2020, 2021 and 2022, and built prediction models for COVID-19 infections and mortality using random forest and logistic regression, with 20 model features. Models were trained and tested based on data from different years and the same year as well, to examine the impact of temporality. To further explain the impact of temporality and its driving factors, Shapley values are employed to quantify individual contributions to model predictions.

RESULTS

For the infection model, we found that the temporal gap had a negative impact on prediction accuracy. On average, the loss in accuracy was 0.0256 for logistic regression and 0.0436 for random forest when there was a temporal gap between the training and testing sets. For the mortality model, the loss in accuracy was 0.0144 for logistic regression and 0.0098 for random forest, which means the impact of temporality was not as strong as in the infection model. Shapley values uncovered the reason behind such differences between the infection and mortality models.

CONCLUSIONS

Our study confirmed the negative impact of temporality on model performance for predicting COVID-19 infections, but it did not find such negative impact of temporality for predicting COVID-19 mortality. Shapley value revealed that there was a fixed set of four features that made predominant contributions for the mortality model across data in three years (2020-2022), while for the infection model there was no such fixed set of features across different years.

Predicting total healthcare demand using machine learning: separate and combined analysis of predisposing, enabling, and need factors

OBJECTIVE

Predicting healthcare demand is essential for effective resource allocation and planning. This study applies Andersen's Behavioral Model of Health Services Use, focusing on predisposing, enabling, and need factors, using data from the 2022 Turkey Health Survey by TUIK. Machine learning methods provide a powerful approach to analyze these factors and their combined impact on healthcare utilization, offering valuable insights for health policy.

METHODS

Seven different machine learning models-Decision Tree, Random Forest, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Logistic Regression, XGBoost, and Gradient Boosting-were utilized. Feature selection was conducted to identify the most significant factors influencing healthcare demand. The models were evaluated for accuracy and generalization ability using performance metrics such as recall, precision, F1 score, and ROC AUC.

RESULTS

The study identified key features affecting healthcare demand. For predisposing factors, gender, educational level, and age group were significant. Enabling factors included treatment costs, community interest, and payment difficulties. Need factors were influenced by smoking status, chronic diseases, and overall health status. The models demonstrated high recall (approximately 0.90) and strong F1 scores (ranging from 0.87 to 0.88), indicating a balanced performance between precision and recall. Among the models, Gradient Boosting, XGBoost, and Logistic Regression consistently outperformed others, achieving the highest predictive accuracy. Random Forest and SVM also performed well, showing robust classification capability.

CONCLUSIONS

The findings highlight the effectiveness of machine learning methods in predicting healthcare demand, providing valuable insights for health policy and resource allocation. Gradient Boosting, XGBoost, and Logistic Regression emerged as the most reliable models, demonstrating superior generalization and classification performance. Understanding the separate and combined effects of predisposing, enabling, and need factors on healthcare demand can contribute to more efficient and data-driven healthcare planning, facilitating strategic decision-making in resource allocation and service delivery.

Does machine learning improve prediction accuracy of the Endoscopic Third Ventriculostomy Success Score? A contemporary Hydrocephalus Clinical Research Network cohort study

PURPOSE

This Hydrocephalus Clinical Research Network (HCRN) study had two aims: (1) to compare the predictive performance of the original ETV Success Score (ETVSS) using logistic regression modeling with other newer machine learning models and (2) to assess whether inclusion of imaging variables improves prediction performance using machine learning models.

METHODS

We identified children undergoing first-time ETV for hydrocephalus that were enrolled prospectively at HCRN sites between 200 and 2020. The primary outcome was ETV success 6 months after index surgery. The cohort was randomly divided into training (70%) and testing (30%) datasets. The classic ETVSS variables were used for logistic regression and machine learning models. Predictive performance of each model was evaluated on the testing dataset using area under the receiver operating characteristic curve (AUROC).

RESULTS

There were 752 patients that underwent first time ETV, of which 185 patients (24.6%) experienced ETV failure within 6 months. For aim 1, using the classic ETVSS variables, machine learning models did not outperform logistic regression with AUROC 0.60 (95% CI: 0.52-0.69) for Naïve Bayes (highest machine learning model performance) and 0.68 (95% CI: 0.60-0.76) for logistic regression. After inclusion of imaging features (aim 2), machine learning model prediction improved but remained no better than the above logistic regression with the highest AUROC of 0.67 (95% CI: 0.59-0.75) attained using Naïve Bayes architecture compared to 0.68 (95% CI: 0.59-0.76) for logistic regression.

CONCLUSIONS

This contemporary multicenter observational cohort study demonstrated that machine learning modeling strategies did not improve performance of the ETVSS model over logistic regression.

Machine learning value in the diagnosis of vertebral fractures: A systematic review and meta-analysis

PURPOSE

To evaluate the diagnostic accuracy of machine learning (ML) in detecting vertebral fractures, considering varying fracture classifications, patient populations, and imaging approaches.

METHOD

A systematic review and meta-analysis were conducted by searching PubMed, Embase, Cochrane Library, and Web of Science up to December 31, 2023, for studies using ML for vertebral fracture diagnosis. Bias risk was assessed using QUADAS-2. A bivariate mixed-effects model was used for the meta-analysis. Meta-analyses were performed according to five task types (vertebral fractures, osteoporotic vertebral fractures, differentiation of benign and malignant vertebral fractures, differentiation of acute and chronic vertebral fractures, and prediction of vertebral fractures). Subgroup analyses were conducted by different ML models (including ML and DL) and modeling methods (including CT, X-ray, MRI, and clinical features).

RESULTS

Eighty-one studies were included. ML demonstrated a diagnostic sensitivity of 0.91 and specificity of 0.95 for vertebral fractures. Subgroup analysis showed that DL (SROC 0.98) and CT (SROC 0.98) performed best overall. For osteoporotic fractures, ML showed a sensitivity of 0.93 and specificity of 0.96, with DL (SROC 0.99) and X-ray (SROC 0.99) performing better. For differentiating benign from malignant fractures, ML achieved a sensitivity of 0.92 and specificity of 0.93, with DL (SROC 0.96) and MRI (SROC 0.97) performing best. For differentiating acute from chronic vertebral fractures, ML showed a sensitivity of 0.92 and specificity of 0.93, with ML (SROC 0.96) and CT (SROC 0.97) performing best. For predicting vertebral fractures, ML had a sensitivity of 0.76 and specificity of 0.87, with ML (SROC 0.80) and clinical features (SROC 0.86) performing better.

CONCLUSIONS

ML, especially DL models applied to CT, MRI, and X-ray, shows high diagnostic accuracy for vertebral fractures. ML also effectively predicts osteoporotic vertebral fractures, aiding in tailored prevention strategies. Further research and validation are required to confirm ML's clinical efficacy.

Life-Course Multidisciplinary Psychosocial Predictors of Dementia Among Older Adults: Results From the Health and Retirement Study

Background and Objectives

Identifying predictors of dementia may help improve risk assessments, increase awareness for risk reduction, and identify potential targets for interventions. We use a life-course psychosocial multidisciplinary modeling framework to examine leading predictors of dementia incidence.

Research Design and Methods

We use data from the Health and Retirement Study to measure 57 psychosocial factors across 7 different domains: (i) demographics, (ii) childhood experiences, (iii) socioeconomic conditions, (iv) health behaviors, (v) social connections, (vi) psychological characteristics, and (vii) adverse adulthood experiences. Our outcome is dementia incidence (over 8 years) operationalized using Langa-Weir classification for adults aged 65+ years who meet criteria for normal cognition at the baseline when all psychosocial factors are measured (N = 1 784 in training set and N = 1 611 in testing set). We compare the standard statistical method (Logistic regression) with machine learning (ML) method (Random Forest) in identifying predictors across the disciplines of interest.

Results

Standard and ML methods identified predictors that spanned multiple disciplines. The standard statistical methods identified lower education and childhood financial duress as among the leading predictors of dementia incidence. The ML method differed in their identification of predictors.

Discussion and Implications

The findings emphasize the importance of upstream risk and protective factors and the long-reaching impact of childhood experiences on cognitive health. The ML approach highlights the importance of life-course multidisciplinary frameworks for improving evidence-based interventions for dementia. Further investigations are needed to identify how complex interactions of life-course factors can be addressed through interventions.

The Value of Strength as a Predictor of Musculoskeletal Injury in Canadian Armed Forces Basic Infantry Candidates

INTRODUCTION

Musculoskeletal injuries (MSKI) impact military organizations by threatening their operational readiness, warranting investigation into relevant factors to inform risk reduction strategies. While several self-reported and physical performance measures have been associated with MSKI among military personnel, few have been validated and none have been reported in Canadian basic infantry candidates. The purpose of this study was to investigate associations between self-reported and physical performance measures and MSKI, and determine their validity as predictors of MSKI, in Canadian basic infantry candidates.

METHODS

This was a planned secondary analysis of a study tracking MSKI at a basic infantry training facility in Ontario, Canada approved by Defence Research & Development Canada. Before the basic infantry training, consenting candidates completed a baseline testing session including self-reported questionnaires, measures of anthropometry, and physical performance previously associated with MSKI (ankle dorsiflexion test, Y-Balance Test, Isometric Mid-Thigh Pull, and the Fitness for Operational Requirements of CAF Employment (FORCE) evaluation). All MSKI reported by candidates were diagnosed by licensed healthcare providers. From a total sample of 129 candidates, 76% (n = 98) were used to determine any associations between baseline testing variables and MSKI and to develop a predictive model (Development Sample), while 24% (n = 33) were used to offer preliminary validation of the same predictive model (Validation Sample). The binary logistic regression and independent sample t-testing determined independent associations with MSKI in the Development Sample. All continuous variables and dichotomous variables previously associated with MSKI risk (Smoker Yes/No, previous history of MSKI, and physical inactivity) were entered into a backward stepwise logistic regression analysis to assess the predictive association with MSKI incidence in the Development Sample. The regression model was then applied to the Validation Sample.

RESULTS

A total of 35 MSKI were diagnosed by Health Services Centre staff. The majority of the MSKI were acute (63%), sustained to the hip, knee, and ankle (74%). The most common diagnoses were strains and sprains (71%). Uninjured participants performed significantly better on the Relative Isometric Mid-Thigh Pull, FORCE 20 mR, FORCE ILS, and FORCE Estimated VO2peak compared to injured participants. Logistic regression analysis showed that the only variable with significant independent association with diagnosed MSKI incidence was self-reported previous history of MSKI. However, the backward stepwise logistic regression analysis retained self-reported previous history of MSKI, FORCE SBD, FORCE Estimated VO2peak, and Isometric Mid-Thigh Pull Peak Force as predictors of MKSI. The logistic regression model including these variables could predict MSKI with an accuracy of 79% in the Development Sample and 67% in the Validation Sample.

CONCLUSION

This study provides preliminary support for the value of measures of absolute muscular strength and cardiorespiratory fitness as predictors of MSKI in Canadian basic infantry candidates. Given the associations between physical performance measures and MSKI, and their necessity during occupational tasks, it is recommended that Canadian basic infantry training facilities integrate resistance training with external loads to best prepare their candidates to meet their occupational demands and potentially minimize MSKI. Further investigations to confirm the predictive capacity of these variables in a larger sample across additional facilities are warranted.

Revealing Novel Genes Related to Parkinson's Disease Pathogenesis and Establishing an associated Model

Parkinson's disease (PD) represents a multifaceted neurological disorder whose genetic underpinnings warrant comprehensive investigation. This study focuses on identifying genes integral to PD pathogenesis and evaluating their diagnostic potential. Initially, we screened for differentially expressed genes (DEGs) between PD and control brain tissues within a dataset comprising larger number of specimens. Subsequently, these DEGs were subjected to weighted gene co-expression network analysis (WGCNA) to discern relevant gene modules. Notably, the yellow module exhibited a significant correlation with PD pathogenesis. Hence, we conducted a detailed examination of the yellow module genes using a cytoscope-based approach to construct a protein-protein interaction (PPI) network, which facilitated the identification of central hub genes implicated in PD pathogenesis. Employing two machine learning techniques, including XGBoost and LASSO algorithms, along with logistic regression analysis, we refined our search to three pertinent hub genes: FOXO3, HIST2H2BE, and HDAC1, all of which demonstrated a substantial association with PD pathogenesis. To corroborate our findings, we analyzed two PD blood datasets and clinical plasma samples, confirming the elevated expression levels of these genes in PD patients. The association of the genes with PD, as reflected by the area under the curve (AUC) values for FOXO3, HIST2H2BE, and HDAC1, were moderate for each gene. Collectively, this research substantiates the heightened expression of FOXO3, HIST2H2BE, and HDAC1 in both PD brain and blood samples, underscoring their pivotal contribution to the pathogenesis of PD.

Higher in-hospital mortality in SARS-CoV-2 omicron variant infection compared to influenza infection-Insights from the CORONA Germany study

BACKGROUND

With the emergence of new subvariants, the disease severity of Severe Acute Respiratory Syndrome Coronavirus-2 has attenuated. This study aimed to compare the disease severity in patients hospitalized with omicron variant infection to those with influenza infection.

METHODS

We compared data from the multicenter observational, prospective, epidemiological "CORONA Germany" (Clinical Outcome and Risk in hospitalized COVID-19 patients) study on patients infected with Severe Acute Respiratory Syndrome Coronavirus-2 to retrospective data on influenza infection cases from November 2016 to August 2022. Severe Acute Respiratory Syndrome Coronavirus-2 cases were classified as wild-type/delta variant before January 2022, or omicron variant from January 2022 onward. The primary outcome was in-hospital mortality, adjusted for age, gender, and comorbidities.

RESULTS

The study included 35,806 patients from 53 hospitals in Germany, including 4,916 patients (13.7%) with influenza infection, 16,654 patients (46.5%) with wild-type/delta variant infection, and 14,236 patients (39.8%) with omicron variant infection. In-hospital mortality was highest in patients with wild-type/delta variant infection (16.8%), followed by patients with omicron variant infection (8.4%) and patients with influenza infection (4.7%). In the adjusted analysis, higher age was the strongest predictor for in-hospital mortality (age 80 years vs. age 50 years: OR 4.25, 95% CI 3.10-5.83). Both, patients with wild-type/delta variant infection (OR 3.54, 95% CI 3.02-4.15) and patients with omicron variant infection (OR 1.56, 95% CI 1.32-1.84) had a higher risk for in-hospital mortality than patients with influenza infection.

CONCLUSION

After adjusting for age, gender and comorbidities, patients with wild-type/delta variant infection had the highest risk for in-hospital mortality compared to patients with influenza infection. Even for patients with omicron variant infection, the adjusted risk for in-hospital mortality was higher than for patients with influenza infection. The adjusted risk for in-hospital mortality showed a strong age dependency across all virus types and variants.

TRIAL REGISTRATION NUMBER

NCT04659187.

Artificial Intelligence Models in Health Information Exchange: A Systematic Review of Clinical Implications

Electronic health record (EHR) systems collate patient data, and the integration and standardization of documents through Health Information Exchange (HIE) play a pivotal role in refining patient management. Although the clinical implications of AI in EHR systems have been extensively analyzed, its application in HIE as a crucial source of patient data is less explored. Addressing this gap, our systematic review delves into utilizing AI models in HIE, gauging their predictive prowess and potential limitations. Employing databases such as Scopus, CINAHL, Google Scholar, PubMed/Medline, and Web of Science and adhering to the PRISMA guidelines, we unearthed 1021 publications. Of these, 11 were shortlisted for the final analysis. A noticeable preference for machine learning models in prognosticating clinical results, notably in oncology and cardiac failures, was evident. The metrics displayed AUC values ranging between 61% and 99.91%. Sensitivity metrics spanned from 12% to 96.50%, specificity from 76.30% to 98.80%, positive predictive values varied from 83.70% to 94.10%, and negative predictive values between 94.10% and 99.10%. Despite variations in specific metrics, AI models drawing on HIE data unfailingly showcased commendable predictive proficiency in clinical verdicts, emphasizing the transformative potential of melding AI with HIE. However, variations in sensitivity highlight underlying challenges. As healthcare's path becomes more enmeshed with AI, a well-rounded, enlightened approach is pivotal to guarantee the delivery of trustworthy and effective AI-augmented healthcare solutions.

Supervised Machine Learning Models to Identify Early-Stage Symptoms of SARS-CoV-2

The coronavirus disease (COVID-19) pandemic was caused by the SARS-CoV-2 virus and began in December 2019. The virus was first reported in the Wuhan region of China. It is a new strain of coronavirus that until then had not been isolated in humans. In severe cases, pneumonia, acute respiratory distress syndrome, multiple organ failure or even death may occur. Now, the existence of vaccines, antiviral drugs and the appropriate treatment are allies in the confrontation of the disease. In the present research work, we utilized supervised Machine Learning (ML) models to determine early-stage symptoms of SARS-CoV-2 occurrence. For this purpose, we experimented with several ML models, and the results showed that the ensemble model, namely Stacking, outperformed the others, achieving an Accuracy, Precision, Recall and F-Measure equal to 90.9% and an Area Under Curve (AUC) of 96.4%.