Few and feasible preoperative variables can identify high-risk surgical patients: derivation and validation of the Ex-Care risk model

The side effects of the pandemic on all-cause postoperative mortality in a COVID reference Hospital in Brazil: a before and after cohort study with 15156 patients

BACKGROUND

Before the pandemic, healthcare systems in Low-Middle Income Countries (LMIC) experienced a limited capacity to treat postoperative complications. It is uncertain whether the interference of the Coronavirus (COVID-19) pandemic on surgical systems has increased postoperative mortality.

METHODS

This before and after cohort study aimed to assess the pandemic's impact on in-hospital postoperative mortality in a university COVID-19 reference hospital in southern Brazil. Data from patients who underwent surgery before (January 2018 to December 2019) the pandemic were compared to data from patients who underwent surgery during the pandemic (February to December 2020). The primary outcome was in-hospital mortality. We developed Poisson regression models to examine the mortality risk of being operated on during the COVID-19 pandemic.

RESULTS

We assessed 15156 surgical patients, 12207 of whom underwent surgery before the pandemic and 2949 during the first year of the pandemic. Mortality rates were 2.5% (309/12207) in the pre-pandemic versus 7.2% (212/2949) in the pandemic. Of these, 25.8% (32/124) of patients with COVID-19 and 6.4% (80/2816) of patients without COVID-19 died. The proportion of urgent surgeries and ASA-PS III was higher in the pandemic group. After adjusting for mortality-related variables, the Relative Risk (RR) associated with undergoing surgery during the pandemic was 1.51 (95% CI 1.27 to 1.79). We excluded COVID-19-positive to perform a sensitivity analysis that confirmed the increased risk of undergoing surgery during the pandemic RR = 1.50 (95% CI 1.27 to 1.78).

CONCLUSION

The substantial number of additional deaths, even amongst those without COVID-19 infection, suggests the pandemic disrupted the surgical service in an LMIC context. Fragile surgical systems may suffer more significant adverse impacts from external stressors such as a pandemic, and urging measures are needed to increase their performance and resilience.

Development and Internal-External Validation of a Post-Operative Mortality Risk Calculator for Pediatric Surgical Patients in Low- and Middle- Income Countries Using Machine Learning

BACKGROUND

The purpose of this study was to develop and validate a mortality risk algorithm for pediatric surgery patients treated at KidsOR sites in 14 low- and middle-income countries.

METHODS

A SuperLearner machine learning algorithm was trained to predict post-operative mortality by hospital discharge using the retrospectively and prospectively collected KidsOR database including patients treated at 20 KidsOR sites from June 2018 to June 2023. Algorithm performance was evaluated by internal-external cross-validated AUC and calibration.

FINDINGS

Of 23,905 eligible patients, 21,703 with discharge status recorded were included in the analysis, representing a post-operative mortality rate of 3.1% (671 mortality events). The candidate algorithm with the best cross-validated performance was an extreme gradient boosting model. The cross-validated AUC was 0.945 (95% CI 0.936 to 0.954) and cross-validated calibration slope and intercept were 1.01 (95% CI 0.96 to 1.06) and 0.05 (95% CI -0.10 to 0.21). For Super Learner models trained on all but one site and evaluated in the holdout site for sites with at least 25 mortality events, overall external validation AUC was 0.864 (95% CI 0.846 to 0.882) with calibration slope and intercept of 1.03 (95% CI 0.97 to 1.09) and 1.18 (95% CI 0.98 to 1.39).

INTERPRETATION

The KidsOR post-operative mortality risk algorithm had outstanding cross-validated discrimination and strong cross-validated calibration. Across all external validation sites, discrimination of Super Learner models trained on the remaining sites was excellent, though re-calibration may be necessary prior to use at new sites. This model has the potential to inform clinical practice and guide resource allocation at KidsOR sites world-wide.

TYPE OF STUDY AND LEVEL OF EVIDENCE

Observational Study, Level III.

The burden of pre-operative anaemia and postoperative outcomes in 15 166 surgical patients from a public hospital in Brazil: A retrospective cohort study

BACKGROUND

Preoperative anaemia is associated with poor postoperative outcomes; however, few studies have reported its prevalence in developing countries and its association with significant postoperative outcomes.

OBJECTIVE

We aimed to identify the prevalence of anaemia and its association with postoperative outcomes in a major public hospital in Brazil.

DESIGN

Retrospective cohort study.

SETTING

Single-centre, 860-bed, quaternary university-affiliated teaching hospital in Southern Brazil.

PATIENTS

We included adult patients who had undergone surgery between 2015 and 2019. Main outcome measures: The main outcome was the in-hospital 30-day postoperative mortality. According to the World Health Organisation, we defined anaemia and its sub-classification (mild, moderate, and severe). We developed Poisson regression models to examine the association between preoperative anaemia and outcomes.

RESULTS

We included 15 166 patients, of whom 6387 (42.1%) were anaemic. After adjustment for confounding factors, patients with anaemia had an increased risk of in-hospital 30-day postoperative mortality (relative risk (RR) 1.69, 95% confidence interval (CI) 1.44 to 1.99, P < 0.001). Mild [relative risk (RR) 1.38, 95% CI 1.12 to 1.71, P = 0.003], moderate (RR 1.73, 95% CI 1.43 to 2.10, P < 0.001), and severe anaemia (RR 2.43, 95% CI 1.92 to 3.07, P < 0.001) were associated with the primary outcome. Anaemia increased the transfusion risk (RR 4.44, 95% CI 3.90 to 5.06, P < 0.001) and postoperative intensive care unit (ICU) admission (RR 1.09, 95% CI 1.04 to 1.16, P = 0.001).

CONCLUSIONS

Four out of 10 patients had anaemia. These patients had an increased risk of adverse postoperative outcomes. Comprehension of the magnitude and impact of anaemia is essential to establish interventions in low-resource scenarios to optimise the patient's journey.

STUDY REGISTRATION

Institutional Review Board Registration number 40522820000005327 (Brazilian CEP/CONEP System, available in https://plataformabrasil.saude.gov.br/).

Development and validation of the Ex-Care BR model: a multicentre initiative for identifying Brazilian surgical patients at risk of 30-day in-hospital mortality

BACKGROUND

Surgical risk stratification is crucial for enhancing perioperative assistance and allocating resources efficiently. However, existing models may not capture the complexity of surgical care in Brazil. Using data from various healthcare settings nationwide, we developed a new risk model for 30-day in-hospital mortality (the Ex-Care BR model).

METHODS

A retrospective cohort study was conducted in 10 hospitals from different geographic regions in Brazil. Data were analysed using multilevel logistic regression models. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC), Brier score, and calibration plots. Derivation and validation cohorts were randomly assigned.

RESULTS

A total of 107,372 patients were included, and 30-day in-hospital mortality was 2.1% (n=2261). The final risk model comprised four predictors related to the patient and surgery (age, ASA physical status classification, surgical urgency, and surgical size), and the random effect related to hospitals. The model showed excellent discrimination (AUROC=0.93, 95% confidence interval [CI], 0.93-0.94), calibration, and overall performance (Brier score=0.017) in the derivation cohort (n=75,094). Similar results were observed in the validation cohort (n=32,278) (AUROC=0.93, 95% CI, 0.92-0.93).

CONCLUSIONS

The Ex-Care BR is the first model to consider regional and organisational peculiarities of the Brazilian surgical scene, in addition to patient and surgical factors. It is particularly useful for identifying high-risk surgical patients in situations demanding efficient allocation of limited resources. However, a thorough exploration of mortality variations among hospitals is essential for a comprehensive understanding of risk.

CLINICAL TRIAL REGISTRATION

NCT05796024.

Postoperative intensive care allocation and mortality in high-risk surgical patients: evidence from a low- and middle-income country cohort

BACKGROUND

The escalation of surgeries for high-risk patients in Low- and Middle-Income Countries (LMICs) lacks evidence on the positive impact of Intensive Care Unit (ICU) admission and lacks universal criteria for allocation. This study explores the link between postoperative ICU allocation and mortality in high-risk patients within a LMIC. Additionally, it assesses the Ex-Care risk model's utility in guiding postoperative allocation decisions.

METHODS

A secondary analysis was conducted in a cohort of high-risk surgical patients from a 800-bed university-affiliated teaching hospital in Southern Brazil (July 2017 to January 2020). Inclusion criteria encompassed 1431 inpatients with Ex-Care Model-assessed all-cause postoperative 30-day mortality risk exceeding 5%. The study compared 30-day mortality outcomes between those allocated to the ICU and the Postanesthetic Care Unit (PACU). Outcomes were also assessed based on Ex-Care risk model classes.

RESULTS

Among 1431 high-risk patients, 250 (17.47%) were directed to the ICU, resulting in 28% in-hospital 30-day mortality, compared to 8.9% in the PACU. However, ICU allocation showed no independent effect on mortality (RR = 0.91; 95% CI 0.68‒1.20). Patients in the highest Ex-Care risk class (Class IV) exhibited a substantial association with mortality (RR = 2.11; 95% CI 1.54-2.90) and were more frequently admitted to the ICU (23.3% vs. 13.1%).

CONCLUSION

Patients in the highest Ex-Care risk class and those with complications faced elevated mortality risk, irrespective of allocation. Addressing the unmet need for adaptable postoperative care for high-risk patients outside the ICU is crucial in LMICs. Further research is essential to refine criteria and elucidate the utility of risk assessment tools like the Ex-Care model in assisting allocation decisions.

Troponin elevation as a marker of short deterioration and one-year death in a high-risk surgical patient cohort in a low and middle income country setting: a postoperative approach to increase surveillance

PURPOSE

Myocardial injury after noncardiac surgery is common and mostly asymptomatic. The ideal target population that will benefit from routine troponin measurements in low and middle income countries (LMICs) is unclear. This study aims to evaluate the clinical outcomes of a cohort of high-risk surgical patients according to high-sensitivity troponin T (hsTnT) in an LMIC setting.

METHODS

We conducted a prospective cohort study of 442 high-risk patients undergoing noncardiac surgery at a Brazilian hospital between February 2019 and March 2020. High-sensitivity troponin T levels were measured preoperatively, 24 hr, and 48 hr after surgery and stratified into three groups: normal (< 20 ng·L-1); minor elevation (20-65 ng·L-1); and major elevation (> 65 ng·L-1). We performed survival analysis to determine the association between myocardial injury and one-year mortality. We described medical interventions and evaluated unplanned intensive care unit (ICU) admission and complications using multivariable models.

RESULTS

Postoperative myocardial injury occurred in 45% of patients. Overall, 30-day mortality was 8%. Thirty-day and one-year mortality were higher in patients with hsTnT ≥ 20 ng·L-1. One-year mortality was 18% in the unaltered troponin group vs 31% and 41% for minor and major elevation groups, respectively. Multivariable analysis of one-year survival showed a hazard ratio (HR) of 1.94 (95% confidence interval [CI], 1.22 to 3.09) for the minor elevation group and a HR of 2.73 (95% CI, 1.67 to 4.45) for the troponin > 65 ng·L-1 group. Patients with altered troponin had more unplanned ICU admissions (13% vs 5%) and more complications (78% vs 48%).

CONCLUSION

This study supports evidence that hsTnT is an important prognostic marker and a strong predictor of all-cause mortality after surgery. Troponin measurement in high-risk surgical patients could potentially be used as tool to scale-up care in LMIC settings.

STUDY REGISTRATION

ClinicalTrials.gov (NCT04187664); first submitted 5 December 2019.

Developing and validating a machine learning ensemble model to predict postoperative delirium in a cohort of high-risk surgical patients: A secondary cohort analysis

BACKGROUND

Postoperative delirium (POD) has a negative impact on prognosis, length of stay and the burden of care. Although its prediction and identification may improve postoperative care, this need is largely unmet in the Brazilian public health system.

OBJECTIVE

To develop and validate a machine-learning prediction model and estimate the incidence of delirium. We hypothesised that an ensemble machine-learning prediction model that incorporates predisposing and precipitating features could accurately predict POD.

DESIGN

A secondary analysis nested in a cohort of high-risk surgical patients.

SETTING

An 800-bed, quaternary university-affiliated teaching hospital in Southern Brazil. We included patients operated on from September 2015 to February 2020.

PATIENTS

We recruited 1453 inpatients with an all-cause postoperative 30-day mortality risk greater than 5% assessed preoperatively by the ExCare Model.

MAIN OUTCOME MEASURE

The incidence of POD classified by the Confusion Assessment Method, up to 7 days postoperatively. Predictive model performance with different feature scenarios were compared with the area under the receiver operating characteristic curve.

RESULTS

The cumulative incidence of delirium was 117, giving an absolute risk of 8.05/100 patients. We developed multiple machine-learning nested cross-validated ensemble models. We selected features through partial dependence plot analysis and theoretical framework. We treated the class imbalance with undersampling. Different feature scenarios included: 52 preoperative, 60 postoperative and only three features (age, preoperative length of stay and the number of postoperative complications). The mean areas (95% confidence interval) under the curve ranged from 0.61 (0.59 to 0.63) to 0.74 (0.73 to 0.75).

CONCLUSION

A predictive model composed of three indicative readily available features performed better than those with numerous perioperative features, pointing to its feasibility as a prognostic tool for POD. Further research is required to test the generalisability of this model.

TRIAL REGISTRATION

Institutional Review Board Registration number 04448018.8.0000.5327 (Brazilian CEP/CONEP System, available in https://plataformabrasil.saude.gov.br/ ).

Perioperative Risk Assessment in Humanitarian Settings: A Scoping Review

BACKGROUND

No validated perioperative risk assessment models currently exist for use in humanitarian settings. To inform the development of a perioperative mortality risk assessment model applicable to humanitarian settings, we conducted a scoping review of the literature to identify reports that described perioperative risk assessment in surgical care in humanitarian settings and LMICs.

METHODS

We conducted a scoping review of the literature to identify records that described perioperative risk assessment in low-resource or humanitarian settings. Searches were conducted in databases including: PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature, Web of Science, World Health Organization Catalog, and Google Scholar.

RESULTS

Our search identified 1582 records. After title/abstract and full text screening, 50 reports remained eligible for analysis in quantitative and qualitative synthesis. These reports presented data from over 37 countries from public, NGO, and military facilities. Data reporting was highly inconsistent: fewer than half of reports presented the indication for surgery; less than 25% of reports presented data on injury severity or prehospital data. Most elements of perioperative risk models designed for high-resource settings (e.g., vital signs, laboratory data, and medical comorbidities) were unavailable.

CONCLUSION

At present, no perioperative mortality risk assessment model exists for use in humanitarian settings. Limitations in consistency and quality of data reporting are a primary barrier, however, can be addressed through data-driven identification of several key variables encompassed by a minimum dataset. The development of such a score is a critical step toward improving the quality of care provided to populations affected by conflict and protracted humanitarian crises.

Assessment of prognostic value of preoperative neutrophil-to-lymphocyte ratio for postoperative mortality and morbidity

Background

The preoperative elevated neutrophil-to-lymphocyte ratio (NLR) was reported to be associated with poorer outcomes after cancer and cardiovascular surgeries. It is unclear, however, if the predictive value is particular or if it may be applied to other types of surgery. We aimed to assess the prognostic value of preoperative NLR levels for morbidity and mortality after various surgery and determine an optimal threshold for NLR.

Methods

We conducted a cohort analysis on patients receiving surgery at Sichuan University West China Hospital between 2018 and 2020. Multivariable piecewise regression analysis were used to determine the optimal cutoff value of NLR. Subgroup analysis were performed to verify the correlation. Sensitivity analysis was used to explore the effect of different thresholds.

Results

We obtained data from 136,347 patients. The optimal cutoff of NLR was determined as 3.6 [95% CI (3.0, 4.1)] by piecewise regression method. After multivariable adjustment, preoperative high NLR remained significantly associated with increased in-hospital mortality (aOR, 2.19; 95% CI, 1.90–2.52; p &lt; 0.001) and ICU admission after surgery (aOR, 1.69; 95% CI, 1.59–1.79; p &lt; 0.001). Subgroup analyses confirmed the predictive value of high NLR in multiple surgical subgroups, including general, orthopedic, neurosurgical, and thoracic surgery subgroups, otorhinolaryngology, head and neck surgery, and burn plastic surgery. A NLR threshold of 3.6 gave excellent predictive value, whether employed alone or added in an extended model.

Conclusions

In conclusion, the association of elevated NLR with higher mortality and ICU admission can be extended to a wider range of procedures. NLR threshold of 3.6 could provide good prognostic value for the prognostic model.

External validation of a surgical mortality risk prediction model for inpatient noncardiac surgery in an Australian private health insurance dataset

BACKGROUND

We previously conducted a systematic review to identify surgical mortality risk prediction tools suitable for adapting in the Australian context and identified the Surgical Outcome Risk Tool (SORT) as an ideal model. The primary aim was to investigate the external validity of SORT for predicting in-hospital mortality in a large Australian private health insurance dataset.

METHODS

A cohort study using a prospectively collected Australian private health insurance dataset containing over 2 million deidentified records. External validation was conducted by applying the predictive equation for SORT to the complete case analysis dataset. Model re-estimation (recalibration) was performed by logistic regression.

RESULTS

The complete case analysis dataset contained 161 277 records. In-hospital mortality was 0.2% (308/161277). The mean estimated risk given by SORT was 0.2% and the median (IQR) was 0.01% (0.003%-0.08%). Discrimination was high (c-statistic 0.96) and calibration was accurate over the range 0%-10%, beyond which mortality was over-predicted but confidence intervals included or closely approached the perfect prediction line. Re-estimation of the equation did not improve over-prediction. Model diagnostics suggested the presence of outliers or highly influential values.

CONCLUSION

The low perioperative mortality rate suggests the dataset was not representative of the overall Australian surgical population, primarily due to selection bias and classification bias. Our results suggest SORT may significantly under-predict 30-day mortality in this dataset. Given potential differences in perioperative mortality, private health insurance status and hospital setting should be considered as covariables when a locally validated national surgical mortality risk prediction model is developed.

External validity of four risk scores predicting 30-day mortality after surgery

Background

Surgical risk prediction tools can facilitate shared decision-making and efficient allocation of perioperative resources. Such tools should be externally validated in target populations before implementation.

Methods

Predicted risk of 30-day mortality was retrospectively derived for surgical patients at Royal Perth Hospital from 2014 to 2021 using the Surgical Outcome Risk Tool (SORT) and the related NZRISK (n=44 031, 53 395 operations). In a sub-population (n=31 153), the Physiology and Operative Severity Score for the enumeration of Mortality (POSSUM) and the Portsmouth variant of this (P-POSSUM) were matched from the Copeland Risk Adjusted Barometer (C2-Ai, Cambridge, UK). The primary outcome was risk score discrimination of 30-day mortality as evaluated by area-under-receiver operator characteristic curve (AUROC) statistics. Calibration plots and outcomes according to risk decile and time were also explored.

Results

All four risk scores showed high discrimination (AUROC) for 30-day mortality (SORT=0.922, NZRISK=0.909, P-POSSUM=0.893; POSSUM=0.881) but consistently over-predicted risk. SORT exhibited the best discrimination and calibration. Thresholds to denote the highest and second-highest deciles of SORT risk (>3.92% and 1.52-3.92%) captured the majority of deaths (76% and 13%, respectively) and hospital-acquired complications. Year-on-year SORT calibration performance drifted towards over-prediction, reflecting a decrease in 30-day mortality over time despite an increase in the surgical population risk.

Conclusions

SORT was the best performing risk score in predicting 30-day mortality after surgery. Categorising patients based on SORT into low, medium (80-90th percentile), and high risk (90-100th percentile) might guide future allocation of perioperative resources. No tools were sufficiently calibrated to support shared decision-making based on absolute predictions of risk.

Enhanced peri-operative care to improve outcomes for high-risk surgical patients in Brazil: a single-centre before-and-after cohort study

Mortality and morbidity for high-risk surgical patients are often high, especially in low-resource settings. Enhanced peri-operative care has the potential to reduce preventable deaths but must be designed to meet local needs. This before-and-after cohort study aimed to assess the effectiveness of a postoperative 48-hour enhanced care pathway for high-risk surgical patients ('high-risk surgical bundle') who did not meet the criteria for elective admission to intensive care. The pathway comprised of six elements: risk identification and communication; adoption of a high-risk post-anaesthesia care unit discharge checklist; prompt nursing admission to ward; intensification of vital signs monitoring; troponin measurement; and prompt access to medical support if required. The primary outcome was in-hospital mortality. Data describing 1189 patients from two groups, before and after implementation of the pathway, were compared. The usual care group comprised a retrospective cohort of high-risk surgical patients between September 2015 and December 2016. The intervention group prospectively included high-risk surgical patients from February 2019 to March 2020. Unadjusted mortality rate was 10.5% (78/746) for the usual care and 6.3% (28/443) for the intervention group. After adjustment, the intervention effect remained significant (RR 0.46 (95%CI 0.30-0.72). The high-risk surgical bundle group received more rapid response team calls (24% vs. 12.6%; RR 0.63 [95%CI 0.49-0.80]) and surgical re-interventions (18.9 vs. 7.5%; RR 0.41 [95%CI 0.30-0.59]). These data suggest that a clinical pathway based on enhanced surveillance for high-risk surgical patients in a resource-constrained setting could reduce in-hospital mortality.

The impact of the incorporation of a feasible postoperative mortality model at the Post-Anaesthestic Care Unit (PACU) on postoperative clinical deterioration: A pragmatic trial with 5,353 patients

Background

Practical use of risk predictive tools and the assessment of their impact on outcome reduction is still a challenge. This pragmatic study of quality improvement (QI) describes the preoperative adoption of a customised postoperative death probability model (SAMPE model) and the evaluation of the impact of a Postoperative Anaesthetic Care Unit (PACU) pathway on the clinical deterioration of high-risk surgical patients.

Methods

A prospective cohort of 2,533 surgical patients compared with 2,820 historical controls after the adoption of a quality improvement (QI) intervention. We carried out quick postoperative high-risk pathways at PACU when the probability of postoperative death exceeded 5%. As outcome measures, we used the number of rapid response team (RRT) calls within 7 and 30 postoperative days, in-hospital mortality, and non-planned Intensive Care Unit (ICU) admission.

Results

Not only did the QI succeed in the implementation of a customised risk stratification model, but it also diminished the postoperative deterioration evaluated by RRT calls on very high-risk patients within 30 postoperative days (from 23% before to 14% after the intervention, p = 0.05). We achieved no survival benefits or reduction of non-planned ICU. The small group of high-risk patients (13% of the total) accounted for the highest proportion of RRT calls and postoperative death.

Conclusion

Employing a risk predictive tool to guide immediate postoperative care may influence postoperative deterioration. It encouraged the design of pragmatic trials focused on feasible, low-technology, and long-term interventions that can be adapted to diverse health systems, especially those that demand more accurate decision making and ask for full engagement in the control of postoperative morbi-mortality.

Derivation and validation of a national multicenter mortality risk stratification model - the ExCare model: a study protocol

Introduction

Surgical care is essential for proper management of various diseases. However, it can result in unfavorable outcomes. In order to identify patients at higher risk of complications, several risk stratification models have been developed. Ideally, these tools should be simple, reproducible, accurate, and externally validated. Unfortunately, none of the best-known risk stratification instruments have been validated in Brazil. In this sense, the Ex-Care model was developed by retrospective data analysis of surgical patients in a major Brazilian university hospital. It consists of four independent predictors easily collected in the preoperative evaluation, showing high accuracy in predicting death within 30 days after surgery.

Objectives

To update and validate a Brazilian national-based model of postoperative death probability within 30 days based on the Ex-Care model. Also, to develop an application for smartphones that allows preoperative risk stratification by Ex-Care model.

Methods

Ten participating centers will collect retrospective data from digital databases. Variables age, American Society of Anesthesiologists (ASA) physical status, surgical severity (major or non-major) and nature (elective or urgent) will be evaluated as predictors for in-hospital mortality within 30 postoperative days, considered the primary outcome.

Expected results

We believe that the Ex-Care model will present discriminative capacity similar to other classically used scores validated for surgical mortality prediction. Furthermore, the mobile application to be developed will provide a practical and easy-to-use tool to the professionals enrolled in perioperative care.