What is the Accuracy of the ACS-NSQIP Surgical Risk Calculator in Emergency Abdominal Surgery? A Meta-Analysis

Preliminary results of the implementation of a Complex Surgical Patient Area as a tool to improve the quality of care

INTRODUCTION

Given the increasing complexity of surgical patients, their evaluation within a Complex Surgical Patient Area (APQC) is essential. This study aims to present the functioning of the APQC and analyse its outcomes.

METHODS

Between 2022 and 2024, 73 patients were evaluated, with a mean age of 72.8 ± 10 years. Of these, 97.3% were ASA ≥ III, and 41.1% had a Clinical Frailty Score ≥4. The evaluation centered on a multidisciplinary committee responsible for determining the patient's operability and guiding the intrahospital circuit. During postoperative evolution, patient follow-up was carried out by two complementary teams in continuous communication. Failure to Rescue (FTR) was defined as the death of a patient following one or more serious complications.

RESULTS

The main reason for including patients in the CSPA was multimorbidity in 53.4% of cases and a specific pathology in 28.8%. In 31.5% of cases, the intervention was ruled out, with one-year survival below 40%. Among the 35 operated patients, the Comprehensive Complication Index (CCI) was 18.034 ± 21.94, the average hospital stay was 14.34 ± 20.15 days, and the readmission rate was 25.7%. The FTR rate was 12.5%.

CONCLUSIONS

Current data suggest a positive impact of the APQC on the evolution of complex patients. A larger patient sample is needed for a detailed analysis of the factors where APQC activities may have the greatest influence.

Hope for the best, but prepare for the worst - Diagnostic accuracy of the American College of Surgeons National Surgical Quality Improvement Program - Risk model for patients undergoing abdominoplasty after massive weight loss - Results from a Retrospective Cohort Study

Background

This study aimed to validate the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) risk calculator for predicting outcomes in patients undergoing abdominoplasty after massive weight loss.

Methods

Patients' characteristics, pre-existing comorbidities and adverse outcomes in our department from 2013 to 2023 were collected retrospectively. Adverse events were defined according to ACS-NSQIP standards and predicted risks were calculated manually using the ACS-NSQIP risk calculator. Binary logistic regression and the Brier score were used to assess the diagnostic accuracy of the model.

Results

Among the 337 individuals who underwent abdominoplasty, 251 had achieved significant weight loss before surgery. After excluding 46 cases due to incomplete data, 205 cases remained for analysis. There were 20% cases of serious complications, 26.3% of some complications, 10.2% of readmissions, 18.8% returned to the operating theatre, 15.6% of surgical site infections and 0.5% each of pneumonia and venous thromboembolism. Although the calculator predicted a 1.5% discharge rate to nursing or rehabilitation facilities and a 0.1% rate of sepsis, neither outcome was observed. Elevated American Society of Anesthesiologists (ASA) status was significantly associated with a higher complication rate, except for surgical site infections (SSI) (p = 0.06). Additionally, an elevated Body Mass Index (BMI) before post-bariatric surgery and a higher resection weight were both associated with increased rates of return to the operating theatre (p = 0.01) and serious complications (p = 0.01). Predicted complication rates (0.1%-8.6%) underestimated actual complication rates (0.5%-26.3%). The Brier scores did not differ significantly from the null model for any outcomes except for general complications (p = 0.001) and logistic regression models demonstrated low sensitivity (0.0-9.8%) and weak odds ratios (1.28-1.46), indicating limited reliability.

Conclusion

The ACS-NSQIP risk calculator does not reliably predict adverse outcomes in this patient cohort.

THE AMERICAN COLLEGE OF SURGEONS-NATIONAL SURGICAL QUALITY IMPROVEMENT PROGRAM CALCULATOR AND SURGICAL APGAR AS PREDICTORS OF POST-CHOLECYSTECTOMY COMPLICATIONS

BACKGROUND

Laparoscopic cholecystectomy is considered safe; however, it is not free from complications, such as bile duct injuries, bleeding, and infection of the surgical site.

AIMS

The aim of this study was to determine the effectiveness of two prediction tools, the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) calculator and the surgical Apgar, in predicting post-cholecystectomy complications.

METHODS

A cross-sectional, analytical, and comparative study was conducted on patients over 18 years old diagnosed with acute cholecystitis who underwent open or laparoscopic cholecystectomy at the Regional Teaching Hospital of Trujillo between 2015 and 2019. A chi-square test was used for bivariate analysis, and the receiver operating characteristic (ROC) curve analysis was employed to determine the discriminative capacity of the ACS-NSQIP and surgical Apgar calculators in predicting severe complications.

RESULTS

A total of 227 patients were included in the study. The analysis revealed that the mean age of patients who experienced severe complications was 75.32±4.58 years. Additionally, 52.6% of these patients were male. Regarding the prediction analysis based on the ROC curve, the ACS-NSQIP calculator showed an area under the curve of 0.895 (95%CI 0.819-0.971; p=0.01), whereas the surgical Apgar calculator showed an area under the curve of 0.611 (95%CI 0.488-0.735; p=0.11).

CONCLUSIONS

The obtained results indicate that the ACS-NSQIP calculator is effective in predicting severe complications in patients undergoing cholecystectomy due to acute cholecystitis. These findings may have important implications for clinical practice and medical decision-making, focusing on the appropriate use of prediction tools to improve outcomes in this type of surgical procedure.

Evaluation of the American College of Surgeons risk calculator in hepatectomy for metastatic colorectal cancer in a Southeast Asian population

PURPOSE

This study evaluated the accuracy of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) calculator in predicting outcomes after hepatectomy for colorectal cancer (CRC) liver metastasis in a Southeast Asian population.

METHODS

Predicted and actual outcomes were compared for 166 patients undergoing hepatectomy for CRC liver metastasis identified between 2017 and 2022, using receiver operating characteristic curves with area under the curve (AUC) and Brier score.

RESULTS

The ACS-NSQIP calculator accurately predicted most postoperative complications (AUC > 0.70), except for surgical site infection (AUC = 0.678, Brier score = 0.045). It also exhibited satisfactory performance for readmission (AUC = 0.818, Brier score = 0.011), reoperation (AUC = 0.945, Brier score = 0.002), and length of stay (LOS, AUC = 0.909). The predicted LOS was close to the actual LOS (5.9 vs. 5.0 days, P = 0.985).

CONCLUSION

The ACS-NSQIP calculator demonstrated generally accurate predictions for 30-day postoperative outcomes after hepatectomy for CRC liver metastasis in our patient population.

Development and Validation of HAS (Hajibandeh Index, ASA Status, Sarcopenia) - A Novel Model for Predicting Mortality After Emergency Laparotomy

OBJECTIVES

To develop and validate a predictive model to predict the risk of postoperative mortality after emergency laparotomy taking into account the following variables: age, age ≥ 80, ASA status, clinical frailty score, sarcopenia, Hajibandeh Index (HI), bowel resection, and intraperitoneal contamination.

SUMMARY BACKGROUND DATA

The discriminative powers of the currently available predictive tools range between adequate and strong; none has demonstrated excellent discrimination yet.

METHODS

The TRIPOD and STROCSS statement standards were followed to protocol and conduct a retrospective cohort study of adult patients who underwent emergency laparotomy due to non-traumatic acute abdominal pathology between 2017 and 2022. Multivariable binary logistic regression analysis was used to develop and validate the model via two protocols (Protocol A and B). The model performance was evaluated in terms of discrimination (ROC curve analysis), calibration (calibration diagram and Hosmer-Lemeshow test), and classification (classification table).

RESULTS

One thousand forty-three patients were included (statistical power = 94%). Multivariable analysis kept HI (Protocol-A: P =0.0004; Protocol-B: P =0.0017), ASA status (Protocol-A: P =0.0068; Protocol-B: P =0.0007), and sarcopenia (Protocol-A: P <0.0001; Protocol-B: P <0.0001) as final predictors of 30-day postoperative mortality in both protocols; hence the model was called HAS (HI, ASA status, sarcopenia). The HAS demonstrated excellent discrimination (AUC: 0.96, P <0.0001), excellent calibration ( P <0.0001), and excellent classification (95%) via both protocols.

CONCLUSIONS

The HAS is the first model demonstrating excellent discrimination, calibration, and classification in predicting the risk of 30-day mortality following emergency laparotomy. The HAS model seems promising and is worth attention for external validation using the calculator provided. HAS mortality risk calculator https://app.airrange.io/#/element/xr3b_E6yLor9R2c8KXViSAeOSK .

Prospective multicenter external validation of postoperative mortality prediction tools in patients undergoing emergency laparotomy

BACKGROUND

Accurate preoperative risk assessment in emergency laparotomy (EL) is valuable for informed decision making and rational use of resources. Available risk prediction tools have not been validated adequately across diverse health care settings. Herein, we report a comparative external validation of four widely cited prognostic models.

METHODS

A multicenter cohort was prospectively composed of consecutive patients undergoing EL in 11 Greek hospitals from January 2020 to May 2021 using the National Emergency Laparotomy Audit (NELA) inclusion criteria. Thirty-day mortality risk predictions were calculated using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), NELA, Portsmouth Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity (P-POSSUM), and Predictive Optimal Trees in Emergency Surgery Risk tools. Surgeons' assessment of postoperative mortality using predefined cutoffs was recorded, and a surgeon-adjusted ACS-NSQIP prediction was calculated when the original model's prediction was relatively low. Predictive performances were compared using scaled Brier scores, discrimination and calibration measures and plots, and decision curve analysis. Heterogeneity across hospitals was assessed by random-effects meta-analysis.

RESULTS

A total of 631 patients were included, and 30-day mortality was 16.3%. The ACS-NSQIP and its surgeon-adjusted version had the highest scaled Brier scores. All models presented high discriminative ability, with concordance statistics ranging from 0.79 for P-POSSUM to 0.85 for NELA. However, except the surgeon-adjusted ACS-NSQIP (Hosmer-Lemeshow test, p = 0.742), all other models were poorly calibrated ( p < 0.001). Decision curve analysis revealed superior clinical utility of the ACS-NSQIP. Following recalibrations, predictive accuracy improved for all models, but ACS-NSQIP retained the lead. Between-hospital heterogeneity was minimum for the ACS-NSQIP model and maximum for P-POSSUM.

CONCLUSION

The ACS-NSQIP tool was most accurate for mortality predictions after EL in a broad external validation cohort, demonstrating utility for facilitating preoperative risk management in the Greek health care system. Subjective surgeon assessments of patient prognosis may optimize ACS-NSQIP predictions.

LEVEL OF EVIDENCE

Diagnostic Test/Criteria; Level II.

Current use of the National Surgical Quality Improvement Program surgical risk calculator in academic surgery: a mixed-methods study

Background

This study aims to quantitatively assess use of the NSQIP surgical risk calculator (NSRC) in contemporary surgical practice and to identify barriers to use and potential interventions that might increase use.

Materials and methods

We performed a cross-sectional study of surgeons at seven institutions. The primary outcomes were self-reported application of the calculator in general clinical practice and specific clinical scenarios as well as reported barriers to use.

Results

In our sample of 99 surgeons (49.7% response rate), 73.7% reported use of the NSRC in the past month. Approximately half (51.9%) of respondents reported infrequent NSRC use (<20% of preoperative discussions), while 14.3% used it in ≥40% of preoperative assessments. Reported use was higher in nonelective cases (30.2% vs 11.1%) and in patients who were ≥65 years old (37.1% vs 13.0%), functionally dependent (41.2% vs 6.6%), or with surrogate consent (39.9% vs 20.4%). NSRC use was not associated with training status or years in practice. Respondents identified a lack of influence on the decision to pursue surgery as well as concerns regarding the calculator's accuracy as barriers to use. Surgeons suggested improving integration to workflow and better education as strategies to increase NSRC use.

Conclusions

Many surgeons reported use of the NSRC, but few used it frequently. Surgeons reported more frequent use in nonelective cases and frail patients, suggesting the calculator is of greater utility for high-risk patients. Surgeons raised concerns about perceived accuracy and suggested additional education as well as integration of the calculator into the electronic health record.

Can the American College of Surgeons NSQIP Surgical Risk Calculator Accurately Predict Adverse Postoperative Outcomes in Emergency Abdominal Surgery? An Italian Multicenter Analysis

BACKGROUND

The American College of Surgeons NSQIP surgical risk calculator provides an estimation of 30-day postoperative adverse outcomes. It is useful in the identification of high-risk patients needing clinical optimization and supports the informed consent process. The purpose of this study is to validate its predictive value in the Italian emergency setting.

STUDY DESIGN

Six Italian institutions were included. Inclusion diagnoses were acute cholecystitis, appendicitis, gastrointestinal perforation or obstruction. Areas under the receiving operating characteristic curves, Brier score, Hosmer-Lemeshow index, and observed-to-expected event ratio were measured to assess both discrimination and calibration. Effect of the Surgeon Adjustment Score on calibration was then tested. A patient's personal risk ratio was obtained, and a cutoff was chosen to predict mortality with a high negative predicted value.

RESULTS

A total of 2,749 emergency procedures were considered for the analysis. The areas under the receiving operating characteristic curve were 0.932 for death (0.921 to 0.941, p < 0.0001; Brier 0.041) and 0.918 for discharge to nursing or rehabilitation facility (0.907 to 0.929, p < 0.0001; 0.070). Discrimination was also strong (area under the receiving operating characteristic curve >0.8) for renal failure, cardiac complication, pneumonia, venous thromboembolism, serious complication, and any complication. Brier score was informative (<0.25) for all the presented variables. The observed-to-expected event ratios were 1.0 for death and 0.8 for discharge to facility. For almost all other variables, there was a general risk underestimation, but the use of the Surgeon Adjustment Score permitted a better calibration of the model. A risk ratio >3.00 predicted the onset of death with sensitivity = 86%, specificity = 77%, and negative predicted value = 99%.

CONCLUSIONS

The American College of Surgeons NSQIP surgical risk calculator has proved to be a reliable predictor of adverse postoperative outcomes also in Italian emergency settings, with particular regard to mortality. We therefore recommend the use of the surgical risk calculator in the multidisciplinary care of patients undergoing emergency abdominal surgery.

Emergency General Surgery: Predicting Morbidity and Mortality in the Geriatric Population

Introduction  Numerous scoring systems have been created to predict the risk of morbidity and mortality in patients undergoing emergency general surgery (EGS). In this article, we compared the different scoring systems utilized at Humanitas Research Hospital and analyzed which one performed the best when assessing geriatric patients (>65 years of age). The scoring systems that were utilized were the APACHE II (Acute Physiology and Chronic Health Evaluation II), ASA (American Society of Anesthesiologists), ACS-NSQIP (American College of Surgeons-National Surgical Quality Improvement Program), Clinical Frailty Score, and the Clavien-Dindo classification as control. Materials and Methods  We compiled a database consisting of all patients over the age of 65 who underwent EGS in a consecutive 24-month period between January 1, 2017 and December 31, 2018. We used the biostatistical program "Stata Version 15" to analyze our results. Results  We found 213 patients who matched our inclusion criteria. Regarding death, we found that the ACS-NSQIP death calculator performed the best with an area under the curve of 0.9017 (odds ratio: 1.09; 95% confidence interval: 1.06-1.12). The APACHE II score had the lowest discriminator when predicting death. Considering short-term complications, the Clavien-Dindo classification scored highly, while both the APACHE II score and Clinical Frailty Score produced the lowest results. Conclusion  The results obtained from our research showed that scoring systems and classifications produced different results depending on whether they were used to predict deaths or short-term complications among geriatric patients undergoing EGS.