The predictive effect of different machine learning algorithms for pressure injuries in hospitalized patients: A network meta-analyses

The best machine learning algorithm for building surgical site infection predictive models: A systematic review and network meta-analysis

BACKGROUND

Many machine learning (ML) algorithms have been used to develop surgical site infection (SSI) prediction models, but little is known about their predicting performance. We conducted a network meta-analysis to compare the performance of different ML algorithms and to explore which one may perform best.

METHODS

MEDLINE, EMBASE, CINAHL, Web of Science, and Cochrane Library were systematically searched from inception to November 25, 2023. We included diagnostic accuracy trials constructing SSI predictive model by ML. Two reviewers selected relevant studies and extracted data. The certainty of the evidence was rated using the QUADAS-2 tool. Performance statistics of the diagnostic analysis and the ranking of the different ML algorithms have been expressed in Relative Diagnostic Odds Ratio (RDOR) and superiority index (SI), respectively, using statistical software STATA and R.

RESULTS

Of 493 articles identified, 10 algorithms from 84 SSI prediction models in 40 articles were included in this review. The results of our study revealed that models based on solely surgical type outperformed models without discrimination of surgical type (RDOR 2.71, 95 % CI: 1.25-5.90, P = 0.01), and mixed-use of structured and textual data-based models outperformed models solely based on structured data (RDOR 8.70, 95 % CI: 3.65-20.75, P < 0.01). Combining the ML algorithms in different databased subgroups separately yields the sorted results: Boosted Classifiers had the best overall prediction for the mixed databased model (SI6.17, 95 % CI: 0.09, 13.00), and Support Vector Machine for the structured (SI 4.70, 95 % CI: 0.11, 13.00).

CONCLUSIONS

ML algorithms developed with structured and textual data provided optimal performance. Boosted Classifiers may be the best algorithm in SSI prediction.

Prediction and Stage Classification of Pressure Ulcers in Intensive Care Patients by Machine Learning

Background/Objective: Pressure ulcers are a serious clinical problem associated with high morbidity, mortality and healthcare costs, especially in intensive care unit (ICU) patients. Existing risk assessment tools, such as the Braden Score, are often inadequate in ICU patients and have poor discriminatory power between classes. This increases the need for more sensitive, predictive and integrative systems. The aim of this study was to classify pressure ulcer stages (Stages I-IV) with high accuracy using machine learning algorithms using demographic, clinical and laboratory data of ICU patients and to evaluate the model performance at a level that can be integrated into clinical decision support systems. Methods: A total of 200 patients hospitalized in the ICU were included in the study. Using demographic, clinical and laboratory data of the patients, six different machine learning algorithms (SVM, KNN, ANN, Decision Tree, Naive Bayes and Discriminant Analysis) were used for classification. The models were evaluated using confusion matrices, ROC-AUC analyses and metrics such as class-based sensitivity and error rate. Results: SVM, KNN and ANN models showed the highest success in classifying pressure ulcer stages, achieving 99% overall accuracy and excellent performance with AUC = 1.00. Variables such as Braden score, albumin and CRP levels contributed significantly to model performance. ROC curves showed that the models provided strong discrimination between classes. Key predictors of pressure ulcer severity included prolonged ICU stay (p < 0.001), low albumin (Stage I: 3.4 ± 0.5 g/dL vs. Stage IV: 2.4 ± 0.8 g/dL; p < 0.001) and high CRP (Stage I: 28 mg/L vs. Stage IV: 142 mg/L; p < 0.001). Conclusions: This study shows that machine learning algorithms offer high accuracy and generalization potential in pressure ulcer classification. In particular, the effectiveness of algorithms such as SVM, ANN and KNN in detecting early-stage ulcers is promising in terms of integration into clinical decision support systems. In future studies, the clinical validity of the model should be increased with multicenter datasets and visual-data-based hybrid models.

Development of a pressure ulcer stage determination system for community healthcare providers using a vision transformer deep learning model

This study reports the first steps toward establishing a computer vision system to help caregivers of bedridden patients detect pressure ulcers (PUs) early. While many previous studies have focused on using convolutional neural networks (CNNs) to elevate stages, hardware constraints have presented challenges related to model training and overreliance on medical opinions. This study aimed to develop a tool to classify PU stages using a Vision Transformer model to process actual PU photos. To do so, we used a retrospective observational design involving the analysis of 395 images of different PU stages that were accurately labeled by nursing specialists and doctors from 3 hospitals. In the pressure ulcer cluster vision transformer (PUC-ViT) model classifies the PU stage with a mean ROC curve value of 0.936, indicating a model accuracy of 97.76% and F1 score of 95.46%. We found that a PUC-ViT model showed higher accuracy than conventional models incorporating CNNs, and both effectively reduced computational complexity and achieved low floating point operations per second. Furthermore, we used internet of things technologies to propose a model that allows anyone to analyze input images even at low computing power. Based on the high accuracy of our proposed model, we confirm that it enables community caregivers to detect PUs early, facilitating medical referral.

Accuracy and clinical effectiveness of risk prediction tools for pressure injury occurrence: An umbrella review

BACKGROUND

Pressure injuries (PIs) pose a substantial healthcare burden and incur significant costs worldwide. Several risk prediction tools to allow timely implementation of preventive measures and a subsequent reduction in healthcare system burden are available and in use. The ability of risk prediction tools to correctly identify those at high risk of PI (prognostic accuracy) and to have a clinically significant impact on patient management and outcomes (effectiveness) is not clear. We aimed to evaluate the prognostic accuracy and clinical effectiveness of risk prediction tools for PI and to identify gaps in the literature.

METHODS AND FINDINGS

The umbrella review was conducted according to Cochrane guidance. Systematic reviews (SRs) evaluating the accuracy or clinical effectiveness of adult PI risk prediction tools in any clinical settings were eligible. Studies on paediatric tools, sensor-only tools, or staging/diagnosis of existing PIs were excluded. MEDLINE, Embase, CINAHL, and EPISTEMONIKOS were searched (inception to June 2024) to identify relevant SRs, as well as Google Scholar (2013 to 2024) and reference lists. Methodological quality was assessed using adapted AMSTAR-2 criteria. Results were described narratively. We identified 26 SRs meeting all eligibility criteria with 19 SRs assessing prognostic accuracy and 11 assessing clinical effectiveness of risk prediction tools for PI (4 SRs assessed both aspects). The 19 SRs of prognostic accuracy evaluated 70 tools (39 scales and 31 machine learning (ML) models), with the Braden, Norton, Waterlow, Cubbin-Jackson scales (and modifications thereof) the most evaluated tools. Meta-analyses from a focused set of included SRs showed that the scales had sensitivities and specificities ranging from 53% to 97% and 46% to 84%, respectively. Only 2/19 (11%) SRs performed appropriate statistical synthesis and quality assessment. Two SRs assessing machine learning-based algorithms reported high prognostic accuracy estimates, but some of which were sourced from the same data within which the models were developed, leading to potentially overoptimistic results. Two randomised trials assessing the effect of PI risk assessment tools (within the full test-intervention-outcome pathway) on the incidence of PIs were identified from the 11 SRs of clinical effectiveness; both were included in a Cochrane SR and assessed as high risk of bias. Both trials found no evidence of an effect on PI incidence. Limitations included the use of the AMSTAR-2 criteria, which may have overly focused on reporting quality rather than methodological quality, compounded by the poor reporting quality of included SRs and that SRs were not excluded based on low AMSTAR-2 ratings (in order to provide a comprehensive overview). Additionally, diagnostic test accuracy principles, rather than prognostic modelling approaches were heavily relied upon, which do not account for the temporal nature of prediction.

CONCLUSIONS

Available systematic reviews suggest a lack of high-quality evidence for the accuracy of risk prediction tools for PI and limited reliable evidence for their use leading to a reduction in incidence of PI. Further research is needed to establish the clinical effectiveness of appropriately developed and validated risk prediction tools for PI.

Risk prediction tools for pressure injury occurrence: an umbrella review of systematic reviews reporting model development and validation methods

BACKGROUND

Pressure injuries (PIs) place a substantial burden on healthcare systems worldwide. Risk stratification of those who are at risk of developing PIs allows preventive interventions to be focused on patients who are at the highest risk. The considerable number of risk assessment scales and prediction models available underscores the need for a thorough evaluation of their development, validation, and clinical utility. Our objectives were to identify and describe available risk prediction tools for PI occurrence, their content and the development and validation methods used.

METHODS

The umbrella review was conducted according to Cochrane guidance. MEDLINE, Embase, CINAHL, EPISTEMONIKOS, Google Scholar, and reference lists were searched to identify relevant systematic reviews. The risk of bias was assessed using adapted AMSTAR-2 criteria. Results were described narratively. All included reviews contributed to building a comprehensive list of risk prediction tools.

RESULTS

We identified 32 eligible systematic reviews only seven of which described the development and validation of risk prediction tools for PI. Nineteen reviews assessed the prognostic accuracy of the tools and 11 assessed clinical effectiveness. Of the seven reviews reporting model development and validation, six included only machine learning models. Two reviews included external validations of models, although only one review reported any details on external validation methods or results. This was also the only review to report measures of both discrimination and calibration. Five reviews presented measures of discrimination, such as the area under the curve (AUC), sensitivities, specificities, F1 scores, and G-means. For the four reviews that assessed the risk of bias assessment using the PROBAST tool, all models but one were found to be at high or unclear risk of bias.

CONCLUSIONS

Available tools do not meet current standards for the development or reporting of risk prediction models. The majority of tools have not been externally validated. Standardised and rigorous approaches to risk prediction model development and validation are needed.

TRIAL REGISTRATION

The protocol was registered on the Open Science Framework ( https://osf.io/tepyk ).

Implementable Prediction of Pressure Injuries in Hospitalized Adults: Model Development and Validation

Background

Numerous pressure injury prediction models have been developed using electronic health record data, yet hospital-acquired pressure injuries (HAPIs) are increasing, which demonstrates the critical challenge of implementing these models in routine care.

Objective

To help bridge the gap between development and implementation, we sought to create a model that was feasible, broadly applicable, dynamic, actionable, and rigorously validated and then compare its performance to usual care (ie, the Braden scale).

Methods

We extracted electronic health record data from 197,991 adult hospital admissions with 51 candidate features. For risk prediction and feature selection, we used logistic regression with a least absolute shrinkage and selection operator (LASSO) approach. To compare the model with usual care, we used the area under the receiver operating curve (AUC), Brier score, slope, intercept, and integrated calibration index. The model was validated using a temporally staggered cohort.

Results

A total of 5458 HAPIs were identified between January 2018 and July 2022. We determined 22 features were necessary to achieve a parsimonious and highly accurate model. The top 5 features included tracheostomy, edema, central line, first albumin measure, and age. Our model achieved higher discrimination than the Braden scale (AUC 0.897, 95% CI 0.893-0.901 vs AUC 0.798, 95% CI 0.791-0.803).

Conclusions

We developed and validated an accurate prediction model for HAPIs that surpassed the standard-of-care risk assessment and fulfilled necessary elements for implementation. Future work includes a pragmatic randomized trial to assess whether our model improves patient outcomes.

An AdaBoost-based algorithm to detect hospital-acquired pressure injury in the presence of conflicting annotations

Hospital-acquired pressure injury is one of the most harmful events in clinical settings. Patients who do not receive early prevention and treatment can experience a significant financial burden and physical trauma. Several hospital-acquired pressure injury prediction algorithms have been developed to tackle this problem, but these models assume a consensus, gold-standard label (i.e., presence of pressure injury or not) is present for all training data. Existing definitions for identifying hospital-acquired pressure injuries are inconsistent due to the lack of high-quality documentation surrounding pressure injuries. To address this issue, we propose in this paper an ensemble-based algorithm that leverages truth inference methods to resolve label inconsistencies between various case definitions and the level of disagreements in annotations. Application of our method to MIMIC-III, a publicly available intensive care unit dataset, gives empirical results that illustrate the promise of learning a prediction model using truth inference-based labels and observed conflict among annotators.

Systematic Review for Risks of Pressure Injury and Prediction Models Using Machine Learning Algorithms

Pressure injuries are increasing worldwide, and there has been no significant improvement in preventing them. This study is aimed at reviewing and evaluating the studies related to the prediction model to identify the risks of pressure injuries in adult hospitalized patients using machine learning algorithms. In addition, it provides evidence that the prediction models identified the risks of pressure injuries earlier. The systematic review has been utilized to review the articles that discussed constructing a prediction model of pressure injuries using machine learning in hospitalized adult patients. The search was conducted in the databases Cumulative Index to Nursing and Allied Health Literature (CINAHIL), PubMed, Science Direct, the Institute of Electrical and Electronics Engineers (IEEE), Cochrane, and Google Scholar. The inclusion criteria included studies constructing a prediction model for adult hospitalized patients. Twenty-seven articles were included in the study. The defects in the current method of identifying risks of pressure injury led health scientists and nursing leaders to look for a new methodology that helps identify all risk factors and predict pressure injury earlier, before the skin changes or harms the patients. The paper critically analyzes the current prediction models and guides future directions and motivations.