The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery

Prognostic accuracy of emergency surgery score: a systematic review

PURPOSE

This systematic review aimed to summarize the literature regarding the prognostic accuracy of the emergency surgery score (ESS).

METHOD

PubMed, Embase, Web of Science, and Scopus were comprehensively searched by May 30, 2023. Two independent researchers performed the initial screening by reviewing the titles and abstracts of the non-duplicate records and selecting the full text of articles meeting our inclusion criteria. Finally, original studies that reported the prognostic accuracy of ESS in any emergency surgeries were included. Data from the included studies were extracted into a checklist designed based on the PRISMA guidelines. The area under the curve (AUC) was used to compare the prognostic accuracy of ESS in different settings.

RESULTS

Twenty-six studies met the inclusion criteria. ESS performed excellently in 30-day post-op mortality (AUC 0.84-0.89) and incidence of cardiac arrest (AUC 0.86-0.88) in emergency general surgeries. The AUC of ESS in overall 30-day morbidities varied from 0.72 to 0.82 in five cohort studies. In predicting the need for ICU admission, the study with the largest sample size reported the best sensitivity of ESS at 80% and the specificity at 85%. Moreover, an outstanding accuracy was observed for the prediction of 30-day sepsis/septic shock in emergency general surgeries (AUC 0.75-0.92).

CONCLUSION

Despite the acceptable prognostic accuracy of ESS in 30-day mortality, morbidities, and in-hospital ICU admission in different emergency surgeries, the high number of required variables and the high probability of missing data highlight the need for modifications to this scoring system.

Preoperative shock index in major abdominal emergency surgery

Introduction

Major abdominal emergency surgery (MAES) patients have a high risk of mortality and complications. The time-sensitive nature of MAES necessitates an easily calculable risk-scoring tool. Shock index (SI) is obtained by dividing heart rate (HR) by systolic blood pressure (SBP) and provides insight into a patient's haemodynamic status. We aimed to evaluate SI's usefulness in predicting postoperative mortality, acute kidney injury (AKI), requirements for intensive care unit (ICU) and high-dependency monitoring, and the ICU length of stay (LOS).

Method

We retrospectively reviewed 212,089 MAES patients from January 2013 to December 2020. The cohort was propensity matched, and 3960 patients were included. The first HR and SBP recorded in the anaesthesia chart were used to calculate SI. Regression models were used to investigate the association between SI and outcomes. The relationship between SI and survival was explored with Kaplan-Meier curves.

Results

There were significant associations between SI and mortality at 1 month (odds ratio [OR] 2.40 [1.67-3.39], P<0.001), 3 months (OR 2.13 [1.56-2.88], P<0.001), and at 2 years (OR 1.77 [1.38-2.25], P<0.001). Multivariate analysis revealed significant relationships between SI and mortality at 1 month (OR 3.51 [1.20-10.3], P=0.021) and at 3 months (OR 3.05 [1.07-8.54], P=0.034). Univariate and multivariate analysis also revealed significant relationships between SI and AKI (P<0.001), postoperative ICU admission (P<0.005) and ICU LOS (P<0.001). SI does not significantly affect 2-year mortality.

Conclusion

SI is useful in predicting postopera-tive mortality at 1 month, 3 months, AKI, postoperative ICU admission and ICU LOS.

Development and validation of a multivariable preoperative prediction model for postoperative length of stay in a broad inpatient surgical population

BACKGROUND

Postoperative length of stay is a meaningful patient-centered outcome and an important determinant of healthcare costs. The Surgical Risk Preoperative Assessment System preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict postoperative length of stay has not been assessed. We aimed to determine whether the Surgical Risk Preoperative Assessment System variables could accurately predict postoperative length of stay up to 30 days in a broad inpatient surgical population.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database from 2012 to 2018. A model using the Surgical Risk Preoperative Assessment System variables and a 28-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables, were fit to the analytical cohort (2012-2018) using multiple linear regression and compared using model performance metrics. Internal chronological validation of the Surgical Risk Preoperative Assessment System model was conducted using training (2012-2017) and test (2018) datasets.

RESULTS

We analyzed 3,295,028 procedures. The adjusted R2 for the Surgical Risk Preoperative Assessment System model fit to this cohort was 93.3% of that for the full model (0.347 vs 0.372). In the internal chronological validation of the Surgical Risk Preoperative Assessment System model, the adjusted R2 for the test dataset was 97.1% of that for the training dataset (0.3389 vs 0.3489).

CONCLUSION

The parsimonious Surgical Risk Preoperative Assessment System model can preoperatively predict postoperative length of stay up to 30 days for inpatient surgical procedures almost as accurately as a model using all 28 American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables and has shown acceptable internal chronological validation.

Preoperative Prediction of Unplanned Reoperation in a Broad Surgical Population

INTRODUCTION

Unplanned reoperation is an undesirable outcome with considerable risks and an increasingly assessed quality of care metric. There are no preoperative prediction models for reoperation after an index surgery in a broad surgical population in the literature. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict unplanned reoperation has not been assessed. This study's objective was to determine whether the SURPAS model could accurately predict unplanned reoperation.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database, 2012-2018. An unplanned reoperation was defined as any unintended operation within 30 d of an initial scheduled operation. The 8-variable SURPAS model and a 29-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program nonlaboratory preoperative variables, were developed using multiple logistic regression and compared using discrimination and calibration metrics: C-indices (C), Hosmer-Lemeshow observed-to-expected plots, and Brier scores (BSs). The internal chronological validation of the SURPAS model was conducted using "training" (2012-2017) and "test" (2018) datasets.

RESULTS

Of 5,777,108 patients, 162,387 (2.81%) underwent an unplanned reoperation. The SURPAS model's C-index of 0.748 was 99.20% of that for the full model (C = 0.754). Hosmer-Lemeshow plots showed good calibration for both models and BSs were similar (BS = 0.0264, full; BS = 0.0265, SURPAS). Internal chronological validation results were similar for the training (C = 0.749, BS = 0.0268) and test (C = 0.748, BS = 0.0250) datasets.

CONCLUSIONS

The SURPAS model accurately predicted unplanned reoperation and was internally validated. Unplanned reoperation can be integrated into the SURPAS tool to provide preoperative risk assessment of this outcome, which could aid patient risk education.

Frailty Index (mFI-5) Predicts Resource Utilization after Nonruptured Endovascular Aneurysm Repair

INTRODUCTION

The 5- factor frailty index (mFI-5) has reliably predicted outcomes after vascular surgeries. The purpose of this study was to determine the performance of this index in aortic endovascular surgery ( endovascular aneurysm repair [EVAR]) MATERIALS AND METHODS: The American College of Surgeons' National Surgical Quality Improvement Program Database (NSQIP) was retrospectively analyzed for patients undergoing nonruptured EVAR between 2015 and 2019. Outcomes were assessed using bivariate analysis (Mann Whitney U test, chi-squared test, and t-test) and multivariate logistic regression analysis.

RESULTS

10,450 patients were identified with a mean age of 73.59 (SD 8.93) y. 8222 (78.7%) were performed for large diameter with the remaining indications including dissection, symptomatic, and embolization/thrombosis. 30-d mortality was 1.3%. Univariate analysis showed that mFI-5≥0.6 was associated with higher rates of prolonged hospital stay (18.8% versus 5.7%, P < 0.001, reference mFI-5 = 0), readmission (12.3% versus 5.9%, P < 0.001), mortality (3.6 % versus 1.2%, P = 0.01), intensive care unit (ICU) length of stay more than 3 d (7.2% versus 2.7%, P < 0.001). Female gender higher age, indication for surgery, and mFI-5 were all associated with increased mortality. Multivariate logistic regression showed that mFI-5 remained as a significant predictor with mFI-5≥0.6 predicting a close to 3 times higher odds for 30-d mortality (odds ratio OR 2.83, P = 0.003), ICU length of stay >3 d (OR 2.48, P < 0.001), >7 d hospital stay (OR 3.94, P < 0.001), readmission (OR 2.16, P < 0.001), and pneumonia (OR 4.2, P < 0.001) CONCLUSIONS: The modified frailty index (mFI-5) is a good predictor for postoperative complications and hospital resource utilization after nonruptured EVAR.

Does Adding a Measure of Social Vulnerability to a Surgical Risk Calculator Improve Its Performance?

BACKGROUND

Emerging literature suggests that measures of social vulnerability should be incorporated into surgical risk calculators. The Social Vulnerability Index (SVI) is a measure designed by the CDC that encompasses 15 socioeconomic and demographic variables at the census tract level. We examined whether adding the SVI into a parsimonious surgical risk calculator would improve model performance.

STUDY DESIGN

The eight-variable Surgical Risk Preoperative Assessment System (SURPAS), developed using the entire American College of Surgeons (ACS) NSQIP database, was applied to local ACS-NSQIP data from 2012 to 2018 to predict 12 postoperative outcomes. Patient addresses were geocoded and used to estimate the SVI, which was then added to the model as a ninth predictor variable. Brier scores and c-indices were compared for the models with and without the SVI.

RESULTS

The analysis included 31,222 patients from five hospitals. Brier scores were identical for eight outcomes and improved by only one to two points in the fourth decimal place for four outcomes with addition of the SVI. Similarly, c-indices were not significantly different (p values ranged from 0.15 to 0.96). Of note, the SVI was associated with most of the eight SURPAS predictor variables, suggesting that SURPAS may already indirectly capture this important risk factor.

CONCLUSION

The eight-variable SURPAS prediction model was not significantly improved by adding the SVI, showing that this parsimonious tool functions well without including a measure of social vulnerability.

Attitudes about use of preoperative risk assessment tools: a survey of surgeons and surgical residents in an academic health system

Background

Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk.

Methods

Surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey via REDCap (Research Electronic Data Capture) between 8/2019-6/2020. Reminder emails were sent once weekly for three weeks. All completed surveys by surgical residents and attendings were included; incomplete surveys were excluded. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher’s exact test and unpaired t-tests comparing responses by surgical attendings vs. residents.

Results

A total of 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons were emailed the survey. Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy.

Conclusions

A small percentage of surgeons use surgical risk calculators as part of their routine practice. Time, inaccessibility, and trust in accuracy were the most significant barriers to use.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13037-022-00320-1.

Associations between preoperative risks of postoperative complications: Results of an analysis of 4.8 Million ACS-NSQIP patients

BACKGROUND

Surgical Risk Preoperative Assessment System (SURPAS) estimates patient's preoperative risk of 12 postoperative complications, yet little is known about associations between these probabilities- We sought to examine relationships between predicted probabilities.

METHODS

Risk of 12 postoperative complications was calculated using SURPAS and the 2012-2018 ACS-NSQIP database. Pearson correlation coefficients (r) were computed to examine relationships between predicted outcomes. "High-risk" was predicted risk in the 10th decile.

RESULTS

4,777,267 patients were included. 71.1% were not high risk, 10.7% were high risk for 1, and 18.2% were high risk for ≥2 complications. High mortality risk was associated with high risk for pulmonary (r = 0.94), cardiac (r = 0.98), renal (r = 0.93), and stroke (0.96) complications. Patients high-risk for ≥2 complications had the most comorbidities and actual adverse outcomes.

CONCLUSIONS

High preoperative risk for certain postoperative complications had strong correlations. 18.2% of patients were high-risk for ≥2 complications and could be targeted for risk reduction interventions.

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons Versus a Standardized Surgical Risk Prediction Tool

Considerable variability exists between surgeons' assessments of a patient's individual pre-operative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes versus a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP) database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs. 5.1%, p=<0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs. 69.8%, p<0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs. 44.3%, p<0.001). C-indices for SURPAS vs. surgeons were 0.84 vs. 0.76 (p=0.3) for morbidity and 0.98 vs. 0.85 (p=0.001) for mortality. Brier scores for SURPAS vs. surgeons were 0.1579 vs. 0.1986 for morbidity (p=0.03) and 0.0409 vs. 0.0543 for mortality (p=0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC=0.654) and mortality (ICC=0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.