The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator Does Not Accurately Predict Risk of 30-Day Complications Among Patients Undergoing Microvascular Head and Neck Reconstruction

The efficacy of the National Surgical Quality Improvement Program surgical risk calculator in head and neck surgery: A meta-analysis

BACKGROUND

The National Surgical Quality Improvement Program surgical risk calculator (SRC) estimates the risk for postoperative complications. This meta-analysis assesses the efficacy of the SRC in the field of head and neck surgery.

METHODS

A systematic review identified studies comparing the SRC's predictions to observed outcomes following head and neck surgeries. Predictive accuracy was assessed using receiver operating characteristic curves (AUCs) and Brier scoring.

RESULTS

Nine studies totaling 1774 patients were included. The SRC underpredicted the risk of all outcomes (including any complication [observed (ob) = 35.9%, predicted (pr) = 21.8%] and serious complication [ob = 28.7%, pr = 17.0%]) except mortality (ob = 0.37%, pr = 1.55%). The observed length of stay was more than twice the predicted length (p < 0.02). Discrimination was acceptable for postoperative pneumonia (AUC = 0.778) and urinary tract infection (AUC = 0.782) only. Predictive accuracy was low for all outcomes (Brier scores ≥0.01) and comparable for patients with and without free-flap reconstructions.

CONCLUSION

The SRC is an ineffective instrument for predicting outcomes in head and neck surgery.

Development and preliminary assessment of a machine learning model to predict myocardial infarction and cardiac arrest after major operations

INTRODUCTION

Accurate prediction of complications often informs shared decision-making. Derived over 10 years ago to enhance prediction of intra/post-operative myocardial infarction and cardiac arrest (MI/CA), the Gupta score has been criticized for unreliable calibration and inclusion of a wide spectrum of unrelated operations. In the present study, we developed a novel machine learning (ML) model to estimate perioperative risk of MI/CA and compared it to the Gupta score.

METHODS

Patients undergoing major operations were identified from the 2016-2020 ACS-NSQIP. The Gupta score was calculated for each patient, and a novel ML model was developed to predict MI/CA using ACS NSQIP-provided data fields as covariates. Discrimination (C-statistic) and calibration (Brier score) of the ML model were compared to the existing Gupta score within the entire cohort and across operative subgroups.

RESULTS

Of 2,473,487 patients included for analysis, 25,177 (1.0%) experienced MI/CA (55.2% MI, 39.1% CA, 5.6% MI and CA). The ML model, which was fit using a randomly selected training cohort, exhibited higher discrimination within the testing dataset compared to the Gupta score (C-statistic 0.84 vs 0.80, p < 0.001). Furthermore, the ML model had significantly better calibration in the entire cohort (Brier score 0.0097 vs 0.0100). Model performance was markedly improved among patients undergoing thoracic, aortic, peripheral vascular and foregut surgery.

CONCLUSIONS

The present ML model outperformed the Gupta score in the prognostication of MI/CA across a heterogenous range of operations. Given the growing integration of ML into healthcare, such models may be readily incorporated into clinical practice and guide benchmarking efforts.

Length of Stay Prediction Models for Oral Cancer Surgery: Machine Learning, Statistical and ACS-NSQIP

OBJECTIVE

Accurate prediction of hospital length of stay (LOS) following surgical management of oral cavity cancer (OCC) may be associated with improved patient counseling, hospital resource utilization and cost. The objective of this study was to compare the performance of statistical models, a machine learning (ML) model, and The American College of Surgeons National Surgical Quality Improvement Program's (ACS-NSQIP) calculator in predicting LOS following surgery for OCC.

MATERIALS AND METHODS

A retrospective multicenter database study was performed at two major academic head and neck cancer centers. Patients with OCC who underwent major free flap reconstructive surgery between January 2008 and June 2019 surgery were selected. Data were pooled and split into training and validation datasets. Statistical and ML models were developed, and performance was evaluated by comparing predicted and actual LOS using correlation coefficient values and percent accuracy.

RESULTS

Totally 837 patients were selected with mean patient age being 62.5 ± 11.7 [SD] years and 67% being male. The ML model demonstrated the best accuracy (validation correlation 0.48, 4-day accuracy 70%), compared with the statistical models: multivariate analysis (0.45, 67%) and least absolute shrinkage and selection operator (0.42, 70%). All were superior to the ACS-NSQIP calculator's performance (0.23, 59%).

CONCLUSION

We developed statistical and ML models that predicted LOS following major free flap reconstructive surgery for OCC. Our models demonstrated superior predictive performance to the ACS-NSQIP calculator. The ML model identified several novel predictors of LOS. These models must be validated in other institutions before being used in clinical practice.

LEVEL OF EVIDENCE

Level 3 Laryngoscope, 2024.

Modified intraoperative temperature management prevents prolonged length of stay after head and neck surgery with free flap reconstruction

The present study aimed to investigate the association between intraoperative body temperature and prolonged length of stay (PLOS) after free flap reconstruction. A total of 753 patients who underwent head and neck surgery with free flap reconstruction were collected and randomly assigned into primary and validation cohorts. In the primary cohort, univariable and multivariable analyses were conducted to evaluate associations between intraoperative time-weighted (TW) temperature (TW average [TWA] temperature, TW hypothermia and TW hyperthermia) and PLOS. Nomograms were developed with and without intraoperative TW temperature, and validated in the validation cohort. Severe intraoperative TW hypothermia (OR = 1.004; 95% CI: 1.000, 1.007; p = 0.032) was identified as an independent risk factor for PLOS. Intraoperative TWA temperature and TW hypothermia showed linear related predictive effect for PLOS. The nomogram incorporating intraoperative TW temperature showed higher C-index (0.652, 95% CI: 0.591, 0.713) and improved net reclassification improvement for non-event (0.277, 95% CI: 0.118, 0.435; p < 0.001). Lower TWA temperature with mild TW hypothermia had a preventive effect on PLOS with a linear association, which may provide a modified range for intraoperative temperature management. The proposed nomogram incorporating intraoperative TW temperature could be used to develop personalized preventive strategies for PLOS after free flap reconstruction. IRB NUMBER: SYSEC-KY-KS-2022-037. CLINICAL TRIAL REGISTRATION NUMBER: Not applicable.

Enhanced Recovery After Surgery for Head and Neck Oncologic Surgery Requiring Microvascular Reconstruction

It has been demonstrated since the 1990's that surgical outcomes can be improved through protocolized perioperative interventions. Since then, multiple surgical societies have engaged in adopting Enhanced Recovery After Surgery (ERAS) Societal recommendations to improve patient satisfaction, decrease the cost of interventions, and improve outcomes. In 2017, ERAS released consensus recommendations detailing the perioperative optimization of patients undergoing head and neck free flap reconstruction. This population was identified as a high resource demand, oftentimes burdened with challenging comorbidity, and poorly described cohort for which a perioperative management protocol could help to optimize outcomes. The following pages aim to further detail perioperative strategies to streamline patient recovery after head and neck reconstructive surgery.

Performance and usability of pre-operative prediction models for 30-day peri-operative mortality risk: a systematic review

Estimating pre-operative mortality risk may inform clinical decision-making for peri-operative care. However, pre-operative mortality risk prediction models are rarely implemented in routine clinical practice. High predictive accuracy and clinical usability are essential for acceptance and clinical implementation. In this systematic review, we identified and appraised prediction models for 30-day postoperative mortality in non-cardiac surgical cohorts. PubMed and Embase were searched up to December 2022 for studies investigating pre-operative prediction models for 30-day mortality. We assessed predictive performance in terms of discrimination and calibration. Risk of bias was evaluated using a tool to assess the risk of bias and applicability of prediction model studies. To further inform potential adoption, we also assessed clinical usability for selected models. In all, 15 studies evaluating 10 prediction models were included. Discrimination ranged from a c-statistic of 0.82 (MySurgeryRisk) to 0.96 (extreme gradient boosting machine learning model). Calibration was reported in only six studies. Model performance was highest for the surgical outcome risk tool (SORT) and its external validations. Clinical usability was highest for the surgical risk pre-operative assessment system. The SORT and risk quantification index also scored high on clinical usability. We found unclear or high risk of bias in the development of all models. The SORT showed the best combination of predictive performance and clinical usability and has been externally validated in several heterogeneous cohorts. To improve clinical uptake, full integration of reliable models with sufficient face validity within the electronic health record is imperative.

Evaluating the Inaccuracy of the National Surgical Quality Improvement Project Surgical Risk Calculator in Plastic Surgery: A Meta-analysis of Short-Term Predicted Complications

BACKGROUND

Preoperative surgical risk assessment is a major component of clinical decision making. The ability to provide accurate, individualized risk estimates has become critical because of growing emphasis on quality metrics benchmarks. The American College of Surgeons National Surgical Quality Improvement Project (NSQIP) Surgical Risk Calculator (SRC) was designed to quantify patient-specific risk across various surgeries. Its applicability to plastic surgery is unclear, however, with multiple studies reporting inaccuracies among certain patient populations. This study uses meta-analysis to evaluate the NSQIP SRC's ability to predict complications among patients having plastic surgery.

METHODS

OVID MEDLINE and PubMed were searched for all studies evaluating the predictive accuracy of the NSQIP SRC in plastic surgery, including oncologic reconstruction, ventral hernia repair, and body contouring. Only studies directly comparing SCR predicted to observed complication rates were included. The primary measure of SRC prediction accuracy, area under the curve (AUC), was assessed for each complication via DerSimonian and Laird random-effects analytic model. The I2 statistic, indicating heterogeneity, was judged low (I2 < 50%) or borderline/unacceptably high (I2 > 50%). All analyses were conducted in StataSE 16.1 (StataCorp LP, College Station, Tex).

RESULTS

Ten of the 296 studies screened met criteria for inclusion (2416 patients). Studies were classified as follows: (head and neck: n = 5, breast: n = 1, extremity: n = 1), open ventral hernia repair (n = 2), and panniculectomy (n = 1). Predictive accuracy was poor for medical and surgical complications (medical: pulmonary AUC = 0.67 [0.48-0.87], cardiac AUC = 0.66 [0.20-0.99], venous thromboembolism AUC = 0.55 [0.47-0.63]), (surgical: surgical site infection AUC = 0.55 [0.46-0.63], reoperation AUC = 0.54 [0.49-0.58], serious complication AUC = 0.58 [0.43-0.73], and any complication AUC = 0.60 [0.57-0.64]). Although mortality was accurately predicted in 2 studies (AUC = 0.87 [0.54-0.99]), heterogeneity was high with I2 = 68%. Otherwise, heterogeneity was minimal (I2 = 0%) or acceptably low (I2 < 50%) for all other outcomes.

CONCLUSIONS

The NSQIP Universal SRC, aimed at offering individualized quantifiable risk estimates for surgical complications, consistently demonstrated poor risk discrimination in this plastic surgery-focused meta-analysis. The limitations of the SRC are perhaps most pronounced where complex, multidisciplinary reconstructions are needed. Future efforts should identify targets for improving SRC reliability to better counsel patients in the perioperative setting and guide appropriate healthcare resource allocation.

Failure of preoperative co-morbidity indices to predict the successful use of the composite scapula free flap for maxillofacial reconstruction in patients with significant medical co-morbidities

The aim of this study was to evaluate the accuracy of validated preoperative patient co-morbidity assessments, including the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), with the use of the composite scapula free flap (CSFF) in maxillofacial reconstruction in patients with significant medical co-morbidities. A retrospective cohort review was performed at an academic institution, covering the period from July 2010 through January 2019. All patients who underwent reconstruction with a CSFF with significant medical co-morbidities were included. Co-morbidity assessments and risk factors were analyzed by comparing predicted versus observed early and late medical and surgical complications. Forty-five patients met the inclusion criteria. The surgical complication rate was 47%; the medical complication rate was 38%. Over 90% of patients returned to successful function at 3 months post-surgery. The ACS-NSQIP prediction of complications ranged from 58% to 75% for accuracy, 76% to 100% for sensitivity, and 50% to 69% for specificity. The prediction of a serious complication was statistically significant in patients with a Charlson Co-morbidity Index ≥7. Age ≥80 years did not significantly increase the risk of a serious complication (P = 0.23). The ACS-NSQIP failed to predict the successful use of the CSFF for patients with significant co-morbidities undergoing maxillofacial reconstruction. The selection of patients who will tolerate complex reconstruction cannot be based solely on co-morbidity charts and standardized preoperative indices.

Predicted versus actual complications in Australian women undergoing post-mastectomy breast reconstruction: a retrospective cohort study using the BRA Score tool

INTRODUCTION

The Breast Reconstruction Risk Assessment (BRA) Score tool is a risk calculator developed to predict the risk of complications in individual patients undergoing breast reconstruction. It was developed in a North American population exclusively undergoing immediate breast reconstruction. This study sought to assess the predictions of the BRA Score tool against the measured outcomes of surgery for an Australian public hospital population, including both immediate and delayed reconstructions.

METHOD

This was a retrospective cohort study of data from women at a single Australian public teaching hospital unit. Data from the Flinders Breast Reconstruction Database was retrieved and compared to BRA Scores calculated for each patient. Receiver operating curve area under the curve analysis was performed as well as Brier scores to compare predicted versus observed complications.

RESULTS

BRA Score predictions were reasonable or good (C-statistic >0.7, Brier score <0.09) for the complications of overall surgical complications, surgical site infection (SSI) and seroma at 30 days, and similarly accurate for prediction of the same complications for implant reconstructions at 12 months. There were similar findings between delayed and immediate reconstructions.

CONCLUSION

The BRA Score risk calculator is valid to detect some risks in both patients undergoing immediate and delayed breast reconstruction in an Australian public hospital setting. SSI is the best predicted complication and is well-predicted across both autologous and prosthetic reconstruction types.

Predicting 30-Day and 180-Day Mortality in Elderly Proximal Hip Fracture Patients: Evaluation of 4 Risk Prediction Scores at a Level I Trauma Center

This study evaluated the use of risk prediction models in estimating short- and mid-term mortality following proximal hip fracture in an elderly Austrian population. Data from 1101 patients who sustained a proximal hip fracture were retrospectively analyzed and applied to four models of interest: Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity (POSSUM), Charlson Comorbidity Index, Portsmouth-POSSUM and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP^®) Risk Score. The performance of these models according to the risk prediction of short- and mid-term mortality was assessed with a receiver operating characteristic curve (ROC). The median age of participants was 83 years, and 69% were women. Six point one percent of patients were deceased by 30 days and 15.2% by 180 days postoperatively. There was no significant difference between the models; the ACS-NSQIP had the largest area under the receiver operating characteristic curve for within 30-day and 180-day mortality. Age, male gender, and hemoglobin (Hb) levels at admission <12.0 g/dL were identified as significant risk factors associated with a shorter time to death at 30 and 180 days postoperative (p < 0.001). Among the four scores, the ACS-NSQIP score could be best-suited clinically and showed the highest discriminative performance, although it was not specifically designed for the hip fracture population.

The American College of Surgeons National Quality Improvement Program Incompletely Captures Implant-Based Breast Reconstruction Complications

BACKGROUND

Implant-based breast reconstruction (IBR) accounts for 70% of postmastectomy reconstructions in the United States. Improving the quality of surgical care in IBR patients through accurate measurements of outcomes is necessary. The purpose of this study is to compare the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) data from our institution to our complete institutional health records database.

METHODS

Data were collected and recorded for all patients undergoing IBR at our institution from 2015 to 2017. The data were completely identified and compared with our institutional NSQIP database for demographics and complications.

RESULTS

The electronic health records data search identified 768 IBR patients in 3 years and NSQIP reported on 229 (30%) patients. Demographics were reported similarly among the 2 databases. Rates of tissue expander/implant infections (5.9% vs 1.8%; P = 0.003) and wound dehiscence (3.5% vs 0.4%; P = 0.003) were not reported similarly between our database and NSQIP. However, the rates of hematoma (2.7% vs 1.8%) and skin flap necrosis (2.5% vs 1.8%) were comparable between the two databases. In our database, 43% of all complications presented after 30 days of surgery, beyond NSQIP's capture period.

CONCLUSIONS

Databases built on partial sampling, such as the NSQIP, may be useful for demographic analyses, but fall short of providing data for complications after IBR, such as infections and wound dehiscence. These results highlight the utility and importance of complete databases. National comparisons of clinical outcomes for IBR should be interpreted with caution when using partial databases.

Risk-adjustment models in patients undergoing head and neck surgery with reconstruction

BACKGROUND

With the current focus on value-based outcomes and reimbursement models, perioperative risk adjustment is essential. Specialty surgical outcomes are not well predicted by the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP); the Head and Neck-Reconstructive Surgery NSQIP was created as a specialty-specific platform for patients undergoing head and neck surgery with flap reconstruction. This study aims to investigate risk prediction models in these patients.

METHODS

The Head and Neck-Reconstructive Surgery NSQIP collected data on patients undergoing head and neck surgery with flap reconstruction from August 1, 2012 to October 20, 2016. Multivariable logistic regression models were created for 9 outcomes (postoperative ventilator dependence, pneumonia, superficial recipient surgical site infection, presence of tracheostomy/nasoenteric (NE)/gastrostomy/gastrojejunostomy(G/GJ) tube 30 days postoperatively, conversion from NE to G/GJ tube, unplanned return to the operating room, length of stay > 7 days). External validation was completed with a more contemporary cohort.

RESULTS

A total of 1095 patients were included in the modelling cohort and 407 in the validation cohort. Models performed well predicting tracheostomy, NE, G/GJ tube presence at 30 days postoperatively and conversion from NE to G/GJ tube (c-indices = 0.75-0.91). Models for postoperative pneumonia, superficial recipient surgical site infection, ventilator dependence > 48 h, and length of stay > 7 days were fair (concordance [c]-indices = 0.63-0.69). The predictive model for unplanned return to the operating room was poor (c-index = 0.58).

CONCLUSIONS AND RELEVANCE

Reliable and discriminant risk prediction models were able to be created for postoperative outcomes using the specialty-specific Head and Neck-Reconstructive Surgery Specific NSQIP.

Performance of the American College of Surgeons NSQIP Surgical Risk Calculator for Total Gastrectomy

BACKGROUND

To encourage implementation of the American College of Surgeons (ACS) NSQIP Risk Calculator for total gastrectomy for gastric cancer, its predictive performance for this specific procedure should be validated. We assessed its discriminatory accuracy and goodness of fit for predicting 12 adverse outcomes.

STUDY DESIGN

Data were collected on all patients with gastric cancer who underwent total gastrectomy with curative intent at Memorial Sloan Kettering Cancer Center between 2002 and 2017. Preoperative risk factors from the electronic medical record were manually inserted into the ACS-NSQIP Risk Calculator. Predictions for adverse outcomes were compared with observed outcomes by Brier scores, c-statistics, and Hosmer-Lemeshow p value.

RESULTS

In a total of 452 patients, the predicted rate of all complications (29%) was lower than the observed rate (45%). Brier scores varied between 0.017 for death and 0.272 for any complication. C-statistics were moderate (0.7-0.8) for death and renal failure, good (0.8-0.9) for cardiac complication, and excellent (≥0.9) for discharge to nursing or rehabilitation facility. Hosmer-Lemeshow p value found poor goodness of fit for pneumonia only.

CONCLUSIONS

For adverse outcomes after total gastrectomy with curative intent in gastric cancer patients, performance of the ACS-NSQIP Risk Calculator is variable. Its predictive performance is best for cardiac complications, renal failure, death, and discharge to nursing or rehabilitation facility.

Evaluating the association of the new National Surgical Quality Improvement Program modified 5-factor frailty index with outcomes in elective colorectal surgery

BACKGROUND

The 5-factor modified frailty index (mFI-5) is a new, NSQIP-based, predictive tool for mortality and postoperative complications. The mFI-5's predictive ability has been validated within the large-scale NSQIP database but applicability in institutional databases has not been investigated. We sought to assess the association between the mFI-5 and morbidity/mortality at the institutional level.

METHODS

A divisional database was queried for 2017 elective colorectal resections and an mFI-5 calculated. The main outcome measure was the association and predictive value of the mFI-5 with major morbidity/mortality and minor complications. Univariable analyses were performed via the Cochran-Armitage Test and Cramer's V. Logistic regression evaluated the relationship between the mFI-5 and morbidity/mortality while accounting for demographics and pre-operative risk factors. Receiver operating characteristic (ROC) curves were plotted to visualize the predictive strength for outcomes.

RESULTS

Four hundred and twelve patients were analyzed. 8.7% had major morbidity/mortality and 31.6% minor complications. The mFI-5 categorized patients into 0 (n = 335), 1 (n = 58), and 2+ (n = 19) groups. Univariable analysis showed a higher mFI-5 was associated significantly with major morbidity/mortality (P = 0.004), but not minor (P = 0.281). Multivariable logistic regression showed a strong association between an mFI-5 score of 2+ with major complications (Major: OR = 4.616, CI [1.442-14.776], P = 0.010). ROC curves showed the mFI-5 was poor for predicting outcomes and performed better when other risk factors were added to the model.

CONCLUSION

The mFI-5 tool has an independent association with major morbidity/mortality in an institutional dataset for elective colorectal surgery, but is not predictive. Its predictive ability is enhanced when other patient-specific risk factors are incorporated.

Assessment of the American College of Surgeons National Surgical Quality Improvement Program Calculator in Predicting Outcomes and Length of Stay After Ivor Lewis Esophagectomy: A Single-Center Experience

BACKGROUND

The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) calculator is a useful tool used by physicians to better inform patients on the surgical risk of postoperative complications. It makes use of the NSQIP database to derive the chance for several adverse outcomes to occur postoperatively given certain patient's factors. The aim of this study was to assess its applicability in a series of patients undergoing an Ivor Lewis esophagectomy.

METHODS

Data from 100 consecutive patients who underwent an Ivor Lewis esophagectomy between September 2013 and November 2017 at our institution were reviewed. Each patient was assessed using the ACS NSQIP surgical risk calculator. Actual events in this group were compared with their particular NSQIP-assessed risk. Logistic regression models were used to compare surgical risk calculator estimates binary outcomes such as incidence of postoperative complications such as cardiac events, renal events, surgical site infection, and death. Mixed linear model was used for length of stay (LOS) duration versus observed LOS. C-statistic was for predictive accuracy each binary outcome and intraclass correlation was used for LOS.

RESULTS

C-statistic values were higher than the cutoff (0.75) for surgical site infection and death. The ACS NSQIP risk calculator was poorly predictive of other reported outcomes by the calculator such as cardiac or renal complications. Corroboration between observed LOS and predicted LOS was weak (8 d versus 11 d, respectively, intraclass coefficient 0.04).

CONCLUSIONS

This study suggests that the risk calculator is useful for identifying risk of death or surgical site infection but poor at discriminating likelihood of other reported outcomes occurring, such as pneumonia, acute renal failure and cardiac complications for patients who underwent an Ivor Lewis esophagectomy. Estimations for procedure-specific complications for esophagectomy may need reevaluated.

Optimizing classical risk scores to predict complications in head and neck surgery: a new approach

Purpose

To validate tools to identify patients at risk for perioperative complications to implement prehabilitation programmes in head and neck surgery (H&N).

Methods

Retrospective cohort including 128 patients submitted to H&N, with postoperative Intermediate Care Unit admittance. The accuracy of the risk calculators ASA, P-POSSUM, ACS-NSQIP and ARISCAT to predict postoperative complications and mortality was assessed. A multivariable analysis was subsequently performed to create a new risk prediction model for serious postoperative complications in our institution.

Results

Our 30-day morbidity and mortality were 45.3% and 0.8%, respectively. The ACS-NSQIP failed to predict complications and had an acceptable discrimination ability for predicting death. The discrimination ability of ARISCAT for predicting respiratory complications was acceptable. ASA and P-POSSUM were poor predictors for mortality and morbidity. Our new prediction model included ACS-NSQIP and ARISCAT (area under the curve 0.750, 95% confidence intervals: 0.63–0.87).

Conclusion

Despite the insufficient value of these risk calculators when analysed individually, we designed a risk tool combining them which better predicts the risk of serious complications.

Electronic supplementary material

The online version of this article (10.1007/s00405-020-06133-1) contains supplementary material, which is available to authorized users.

Predicting Post-operative Complications in Crohn's Disease: an Appraisal of Clinical Scoring Systems and the NSQIP Surgical Risk Calculator

BACKGROUND

Surgery is common in patients with Crohn's disease and can contribute significantly to patient morbidity. The National Surgical Quality Improvement Program surgical risk calculator (NSQIP-SRC) that is currently utilized to predict surgical risk does not take Crohn's disease into account and, as a result, seems to underestimate risk in this patient population. This study aimed to evaluate the accuracy of the NSQIP-SRC in Crohn's disease patients and to evaluate the utility of disease severity scores in predicting surgical risk.

METHODS

Between 2011 and 2017, there were 176 surgical cases involving Crohn's disease patients. Demographic data and 30-day surgical outcomes were collected. Disease severity scores including Harvey Bradshaw Index (HBI), Crohn's Disease Activity Index (CDAI), Simple Endoscopic Score for Crohn's Disease (SES-CD), and NSQIP-SRC risk percentages were calculated.

RESULTS

Patients in remission based on HBI had a complication rate of 8.57% (n = 3), while those with mild or moderate-severe disease had rates of 33.33% (n = 11) and 38.46% (n = 20) respectively (p = 0.0045). In multivariable analysis, those with mild (OR; 8.37, 95% CI; 1.64, 42.78; p = 0.011) or moderate-severe (OR; 11.69, 95% CI; 2.42, 56.46; p = 0.002) disease had increased odds of complication compared to remission. Complication rate was not associated with NSQIP-SRC percent risk of any complication.

CONCLUSION

NSQIP-SRC does not accurately predict risk in patients with CD undergoing surgery. Higher disease activity based on HBI is associated with increased odds of complication and may prove to be more predictive of surgical complication in the Crohn's patient population.

Evaluating Unplanned Returns to the Operating Room in Head and Neck Free Flap Patients

BACKGROUND

Head and neck oncologic surgery with reconstruction represents one of the most complex operations in otolaryngology. Unplanned return to the operating room represents an objective measure of postoperative complications. The purpose of this study was to identify reasons and risk factors for unplanned return to the operating room in patients undergoing head and neck surgery with reconstruction.

METHODS

This retrospective cohort study of 467 patients undergoing head and neck surgery with free flap reconstruction used a previously-developed Head and Neck-Reconstructive Surgery-specific National Surgical Quality Improvement Program. Disease and site-specific preoperative, intraoperative, and postoperative data were gathered. Comparisons between those with and without an unexpected return to the operating room were completed with univariate and multiple logistic regression models.

RESULTS

The rate of unexpected return to the operating room was 18.8% (88 patients). Most common reasons for URTOR were flap compromise (24 patients, 5.1%), postoperative infection (21 patients, 4.5%), and hematoma (20 patients, 4.3%). Two risk factors were identified by multivariate analysis: coagulopathy (OR_adjusted = 2.83, 95% CI = 1.24-6.19, P = 0.010), and use of alcohol (OR_adjusted = 1.9, 95% CI = 1.14-3.33, P = 0.025).

CONCLUSIONS

Preexisting coagulopathy and increased alcohol consumption were associated with increased risk of unexpected return to the operating room. These findings can aid physicians in preoperative patient counseling and medical optimization and can inform more precise risk stratification of patients undergoing head and neck surgery with reconstruction. Strategies to prevent and mitigate unexpected returns to the operating room will improve patient outcomes, decrease resource utilization, and facilitate successful integration into alternative payment models.

Predictive value of the ACS NSQIP calculator for head and neck reconstruction free tissue transfer

BACKGROUND

Predictive models to forecast the likelihood of specific outcomes after surgical intervention allow informed shared decision-making by surgeons and patients. Previous studies have suggested that existing general surgical risk calculators poorly forecast head and neck surgical outcomes. However, no large study has addressed this question while subdividing subjects by surgery performed.

OBJECTIVES

To determine the accuracy of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator in estimating length of hospital stay and risk of postoperative complications after free tissue transfer surgery.

STUDY DESIGN

A retrospective chart review of patients at one institution was performed using Current Procedural Terminology codes for anterolateral thigh (ALT) flap, fibula free flap (FFF), and radial forearm free flap (RFFF) reconstruction. Output data from the ACS NSQIP surgical risk calculator were compared with the observed rates in our patients.

METHODS

Incidences of cardiac complications, pneumonia, venous thromboembolism, return to the operating room, and discharge to skilled nursing facility (SNF) were compared to predicted incidences. Length of stay was also compared to the predicted length of stay.

RESULTS

Three hundred thirty-six free flap reconstructions with 197 ALT flaps, 85 RFFFs, and 54 FFFFs were included. Brier scores were calculated using ACS NSQIP forecast and actual incidences. No Brier score was <0.01 for the entire sample or any subgroup, which indicates that the NSQIP risk calculator does not accurately forecast outcomes after free tissue reconstruction.

CONCLUSION

The ACS NSQIP failed to accurately forecast postoperative outcomes after head and neck free flap reconstruction for the entire sample or subgroup analyses.

LEVEL OF EVIDENCE

4 Laryngoscope, 130:679-684, 2020.

Quality assurance in head and neck cancer surgery: where are we, and where are we going?

PURPOSE OF REVIEW

The scope of this review is to summarize current efforts in quality assurance for head and neck cancer surgery. National and international initiatives are summarized and progress in terms of identification of process indicators and outcome indicators delineated.

RECENT FINDINGS

Massive efforts have been made in order to improve quality of head and neck cancer surgery. New guidelines for quality assurance of head and neck cancer surgery in clinical trials have recently been proposed by EORTC.

SUMMARY

Quality assurance programs can be tested within the clearly defined environment of prospective clinical trials. If positive, such programs could be rolled out within national healthcare systems, if feasible. Testing quality programs in clinical trials could be a versatile tool to help head neck cancer patients benefit from such initiatives on a global level.

Mild hypothermia is associated with improved outcomes in patients undergoing microvascular head and neck reconstruction

OBJECTIVE

Microvascular free tissue transfer has become the standard for reconstruction for large defects. With long operative times and an increased surface area exposed, transient hypothermia is common, but it is unclear how this impacts surgical outcomes. This study evaluated the impact of core body temperature on free tissue flap outcomes in patients undergoing microvascular reconstruction.

STUDY DESIGN

Retrospective data analysis.

SETTING

Mount Sinai Hospital; NYC, NY; 2007-2016.

SUBJECTS AND METHODS

Demographic information, mean/minimum/maximum body temperatures, and the presence of flap complications (venous thrombosis, arterial insufficiency, flap death, wound infection/dehiscence, fistula, chyle leak, hematoma/seroma) of 519 free tissue transfer patients were documented. Binomial logistic regression was used to examine associations between the presence of flap complications and mean temperature. Statistical analysis used SPSS, with p-values ≤0.05 deemed statistically significant.

RESULTS

393 soft-tissue and 125 osteocutaneous flaps were included. 19.8% (n = 103) patients had the presence of ≥1 flap complication, while 80.2% (n = 416) did not. Average temperature for all patients was 36.12 ± 0.84 °C, with minimum at 34.43 ± 0.97 °C and maximum at 37.24 ± 1.23 °C. After controlling for several factors including: tumor stage, radiation, diabetes, BMI, age, sex, and flap type, there was a significant association between flap complications and mean intraoperative temperature (Exp(B) = 1.559, p = 0.004).

CONCLUSION

Higher intraoperative temperatures were associated with worse outcomes. A mild relative hypothermia may improve flap outcomes in this population. This represents the largest study to date evaluating the impact of intraoperative temperature on free tissue transfer outcomes.

Scoping Review of the National Surgical Quality Improvement Program in Plastic Surgery Research

Background

The National Surgical Quality Improvement Program (NSQIP) is a robust, high-quality surgical outcomes database that measures risk-adjusted 30-day outcomes of surgical interventions. The purpose of this scoping review is to describe how the NSQIP is being used in plastic surgery research.

Methods

A comprehensive electronic literature search was completed in PubMed, Embase, MEDLINE, and CINAHL. Two reviewers independently reviewed articles to determine their relevance using predefined inclusion criteria. Articles were included if they utilized NSQIP data to conduct research in a domain of plastic surgery or analyzed surgical procedures completed by plastic surgeons. Extracted information included the domain of plastic surgery, country of origin, journal, and year of publication.

Results

A total of 106 articles met the inclusion criteria. The most common domain of plastic surgery was breast reconstruction representing 35% of the articles. Of the 106 articles, 95% were published within the last 5 years. The Plastic and Reconstructive Surgery journal published most of the (59%) NSQIP-related articles. All of the studies were retrospective. Of note, there were no articles on burns and only one study on trauma as the domain of plastic surgery.

Conclusion

This scoping review describes how NSQIP data are being used to analyze plastic surgery interventions and outcomes in order to guide quality improvement in 106 articles. It demonstrates the utility of NSQIP in the literature, however also identifies some limitations of the program as it applies to plastic surgery.

Assessment of the NSQIP Surgical Risk Calculator in Predicting Microvascular Head and Neck Reconstruction Outcomes

Objective

This study evaluated the accuracy of the Surgical Risk Calculator (SRC) of the ACS NSQIP (American College of Surgeons National Surgical Quality Improvement Program) in predicting head and neck microvascular reconstruction outcomes.

Study Design

Retrospective analysis.

Setting

Tertiary medical center.

Subjects and Methods

A total of 561 free flaps were included in the analysis. The SRC-predicted 30-day rates of postoperative complications, hospital length of stay (LOS), and rehabilitation discharge were compared with the actual rates and events. The SRC’s predictive value was examined with Brier scores and receiver operating characteristic area under the curve.

Results

A total of 425 myocutaneous, 134 osseous (84 fibula, 47 scapula, and 3 iliac crest), and 2 omental free flaps were included in this study. All perioperative complications evaluated had area under the curve values ≤0.75, ranging from 0.480 to 0.728. All but 2 postoperative complications had Brier scores >0.01. SRC-predicted LOS was 9.4 ± 2.38 days (mean ± SD), which did not strongly correlate with the actual LOS of 11.98 ± 9.30 days ( r = 0.174, P < .0001).

Conclusion

The SRC is a poor predictor for surgical outcome among patients undergoing microvascular head and neck reconstruction.

Predicting complications of major head and neck oncological surgery: an evaluation of the ACS NSQIP surgical risk calculator

Background

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient’s risk of developing specific postoperative outcomes.

Methods

Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed–ranked test.

Results

The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6).

Conclusion

The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

Predictive performance of the American College of Surgeons universal risk calculator in neurosurgical patients

OBJECTIVE

The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) universal Surgical Risk Calculator is an online decision-support tool that uses patient characteristics to estimate the risk of adverse postoperative events. Further validation of this risk calculator in the neurosurgical population is needed; therefore, the object of this study was to assess the predictive performance of the ACS NSQIP Surgical Risk Calculator in neurosurgical patients treated at a tertiary care center.

METHODS

A single-center retrospective review of 1006 neurosurgical patients treated in the period from September 2011 through December 2014 was performed. Individual patient characteristics were entered into the NSQIP calculator. Predicted complications were compared with actual occurrences identified through chart review and administrative quality coding data. Statistical models were used to assess the predictive performance of risk scores. Traditionally, an ideal risk prediction model demonstrates good calibration and strong discrimination when comparing predicted and observed events.

RESULTS

The ACS NSQIP risk calculator demonstrated good calibration between predicted and observed risks of death (p = 0.102), surgical site infection (SSI; p = 0.099), and venous thromboembolism (VTE; p = 0.164) Alternatively, the risk calculator demonstrated a statistically significant lack of calibration between predicted and observed risk of pneumonia (p = 0.044), urinary tract infection (UTI; p < 0.001), return to the operating room (p < 0.001), and discharge to a rehabilitation or nursing facility (p < 0.001). The discriminative performance of the risk calculator was assessed using the c-statistic. Death (c-statistic 0.93), UTI (0.846), and pneumonia (0.862) demonstrated strong discriminative performance. Discharge to a rehabilitation facility or nursing home (c-statistic 0.794) and VTE (0.767) showed adequate discrimination. Return to the operating room (c-statistic 0.452) and SSI (0.556) demonstrated poor discriminative performance. The risk prediction model was both well calibrated and discriminative only for 30-day mortality.

CONCLUSIONS

This study illustrates the importance of validating universal risk calculators in specialty-specific surgical populations. The ACS NSQIP Surgical Risk Calculator could be used as a decision-support tool for neurosurgical informed consent with respect to predicted mortality but was poorly predictive of other potential adverse events and clinical outcomes.

Performance Improvement in Head and Neck Cancer

Performance improvement requires establishing a platform to set benchmarks and monitor the quality of care provided through quality indicators and metrics. This has long been recognized as critical to overall quality improvement and more recently, has become federally mandated. Here, we review recent studies evaluating performance in head and neck cancer care, from those spanning all phases of head and neck cancer care to others focused on head and neck surgical performance, including both national and departmental/institutional efforts.

Quality Indicators: Measurement and Predictors in Head and Neck Cancer Free Flap Patients

Objective

To determine the predictors of length of stay (LOS), readmission within 30 days, and unplanned return to the operating room (OR) within 30 days in head and neck free flap patients.

Study Design

Case series with chart review.

Setting

Tertiary academic cancer hospital.

Subjects and Methods

All head and neck free flap patients at The Ohio State University (OSU, 2006-2012) were assessed. Multivariable logistic regression to assess the impact of patient factors, flap and wound factors, and intraoperative factors on the aforementioned quality metric outcomes.

Results

In total, 515 patients were identified, of whom 66% had oral cavity cancers, 33% had recurrent tumors, and 28% underwent primary radiotherapy. Of the patients, 31.5% had a LOS greater than 9 days, predicted by longer operative time, oral cavity and pharyngeal tumor sites, blood transfusion, diabetes mellitus, and any complication. A total of 12.6% of patients were readmitted within 30 days predicted by absent OSU preoperative assessment clinic attendance and any complication, and 14.8% of patients had an unplanned OR return predicted by advanced age.

Conclusions

When assessing quality metrics, adjustment for the complexity involved in managing patients with head and neck cancer with a high comorbidity index, clean contaminated wounds, and a high degree of primary radiotherapy is important. Patients seen in a preoperative assessment clinic had a lower risk of readmission postoperatively, and this should be recommended for all head and neck free flap patients. Quality improvement projects should focus on predictors and prevention of complications as this was the number one predictor of both increased length of stay and readmission.

Comparison of 4 Cardiac Risk Calculators in Predicting Postoperative Cardiac Complications After Noncardiac Operations

The 2014 American College of Cardiology/American Heart Association Perioperative Guidelines suggest using the Revised Cardiac Risk Index, myocardial infarction or cardiac arrest, or American College of Surgeons-National Surgical Quality Improvement Program calculators for combined patient-surgical risk assessment. There are no published data comparing their performance. This study compared these risk calculators and a reconstructed Revised Cardiac Risk Index in predicting postoperative cardiac complications, both during hospitalization and 30 days after operation, in a patient cohort who underwent select surgical procedures in various risk categories. Cardiac complications occurred in 14 of 663 patients (2.1%), of which 11 occurred during hospitalization. Only 3 of 663 patients (0.45%) had a myocardial infarction or cardiac arrest. Because these calculators used different risk factors, different outcomes, and different durations of observation, a true direct comparison is not possible. We found that all 4 risk calculators performed well in the setting they were originally studied but were less accurate when applied in a different manner. In conclusion, all calculators were useful in defining low-risk patients in whom further cardiac testing was unnecessary, and the myocardial infarction or cardiac arrest may be the most reliable in selecting higher risk patients.

The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator Has a Role in Predicting Discharge to Post-Acute Care in Total Joint Arthroplasty

BACKGROUND

Patient demand and increasing cost awareness have led to the creation of surgical risk calculators that attempt to predict the likelihood of adverse events and to facilitate risk mitigation. The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator is an online tool available for a wide variety of surgical procedures, and has not yet been fully evaluated in total joint arthroplasty.

METHODS

A single-center, retrospective review was performed on 909 patients receiving a unilateral primary total knee (496) or hip (413) arthroplasty between January 2012 and December 2014. Patient characteristics were entered into the risk calculator, and predicted outcomes were compared with observed results. Discrimination was evaluated using the receiver-operator area under the curve (AUC) for 90-day readmission, return to operating room (OR), discharge to skilled nursing facility (SNF)/rehab, deep venous thrombosis (DVT), and periprosthetic joint infection (PJI).

RESULTS

The risk calculator demonstrated adequate performance in predicting discharge to SNF/rehab (AUC 0.72). Discrimination was relatively limited for DVT (AUC 0.70, P = .2), 90-day readmission (AUC 0.63), PJI (AUC 0.67), and return to OR (AUC 0.59). Risk score differences between those who did and did not experience discharge to SNF/rehab, 90-day readmission, and PJI reached significance (P < .01). Predicted length of stay performed adequately, only overestimating by 0.2 days on average (rho = 0.25, P < .001).

CONCLUSION

The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator has fair utility in predicting discharge to SNF/rehab, but limited usefulness for 90-day readmission, return to OR, DVT, and PJI. Although length of stay predictions are similar to actual outcomes, statistical correlation remains relatively weak.

Can we predict postoperative complications in elderly Chinese patients with hip fractures using the surgical risk calculator?

Purpose

Hip fractures are associated with poor prognosis in elderly patients partly due to the high rate of postoperative complications. This study was aimed to investigate whether the surgical risk calculator is suitable for predicting postoperative complications in elderly Chinese patients with hip fractures.

Methods

The incidence of postoperative complications among 410 elderly patients with hip fractures was predicted by the surgical risk calculator and then compared with the actual value. The risk calculator model was evaluated using the following three metrics: Hosmer–Lemeshow test for the goodness-of-fit of the model, receiver operating characteristic curve (ROC) (also referred as C-statistic) for the predictive specificity and sensitivity, and the Brier’s score test for predictive accuracy.

Results

Preoperative risk factors including gender, age, preoperative functional status, American Society of Anesthesiologists grade, hypertension, dyspnea, dialysis, previous cardio-vascular history, and cerebrovascular disease were positively correlated with the incidence of postoperative complications in elderly patients with hip fractures. The predicted complication incidence rate was well matched with the actual complication rate by Hosmer–Lemeshow test. The model had high sensitivity and specificity for predicting the mortality rate of these patients with a C-statistic index of 0.931 (95% CI [0.883, 0.980]). The surgical calculator model had an accuracy of 90% for predicting the reoperation rate (Brier’s score <0.01).

Conclusions

The surgical risk calculator could be useful for predicting mortality and reoperation in elderly patients with hip fracture. Patients and surgeons may use this simple calculator to better manage the preoperative risks.

Surgical data science: The new knowledge domain

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care, with the goal of maximizing the quality and value of care. Whereas innovations in diagnostic and therapeutic technologies have driven past improvements in the quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytical techniques, and translation or integration of research findings into patient care. We foresee the emergence of surgical/interventional data science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data are pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care, including prevention, diagnosis, intervention, or postoperative recovery. The existing literature already provides preliminary results, suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from preoperative, intraoperative, and postoperative contexts, how it could support intraoperative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

Quality Improvement and Reporting Systems: What the Oral and Maxillofacial Surgeon Should Know

Health care is an inherently dangerous environment, and patient safety should be an explicit goal of oral and maxillofacial surgery. Important components of a safety program include a nonpunitive safety culture, the implementation of patient safety practices, standardized incident reporting and adverse event analysis, regular self-assessment, and internal and external benchmarking. Implementation of a safety program requires the strong commitment of leadership and the engagement and empowerment of all employees. Oral and maxillofacial surgery can become the model dental specialty by implementing patient safety programs for office-based surgery. The programs could then be used by all dental practitioners performing oral surgery in the office.