Can the American College of Surgeons NSQIP surgical risk calculator identify patients at risk of complications following microsurgical breast reconstruction?

MICRO: Microsurgical Index for Complication Risk and Outcomes

Free tissue transfer (FTT) is determined by a multitude of patient and surgeon factors. However, no tool exists to quantify patient risk for complications following FTT. This study developed the microsurgical index for complication risk and outcomes (MICRO) to address this.Patients were queried from the 2007 to 2015 MarketScan Databases with CPT codes for FTT requiring microsurgical anastomosis. ICD-9 codes were used to query comorbidity and 90-day postoperative complication data for each patient. The Charlson and Elixhauser Comorbidity Indexes were constructed for each patient. The MICRO was then constructed with a forward stepwise selection from Elixhauser comorbidities and domain expert input. Indexes were used as covariates in multivariate logistic regression models with patient age, sex, and flap tissue type to predict complications following FTT. The area under the receiver operating characteristic curve and fivefold cross-validation classification accuracy was determined.A total of 5,595 patients were included. The final MICRO consists of seven variables (Charlson: 19; Elixhauser: 30). It had the highest area under the receiver operating characteristic curve (0.60) and accuracy (60.4%) of all indexes when predicting complications.The MICRO outperforms available patient comorbidity indexes at predicting complications following FTT with far fewer variables. Future studies could augment the MICRO with more granular or institutional data consisting of surgeon, donor-site, and recipient-site data to create a sharper risk-stratification tool for the plastic surgeon.

Comparative analysis of comorbidity indexes in implant-based breast reconstruction

INTRODUCTION

Recent studies have used comorbidity indexes to stratify postsurgical complication risk across multiple surgical fields, including plastic surgery. However, such studies are lacking in implant-based breast reconstruction (IBBR). Understanding how comorbidity scores affect postsurgical outcomes in IBBR can help identify patients who may require additional medical surveillance after surgery.

METHODS

We conducted a retrospective analysis of adult female patients who underwent IBBR between January 2017 and December 2022 using the Merative™ Marketscan® Research Databases. ICD-10 diagnosis codes were used to calculate patients' scores on four different comorbidity indexes. Chi-squared tests were performed for demographic analysis, and multivariable logistic regression controlling for demographic and surgical variables, was conducted to determine associations between comorbidity indexes and adverse outcomes.

RESULTS

Among 16,287 IBBR patients, 3145 (19.3%) experienced a complication within 90 days. On regression analysis, the Modified Frailty Index 5-Item (mFI-5) demonstrated the strongest associations with complications (odds ratio=1.25 [1.18, 1.32]), compared to the Charlson Comorbidity Index, Elixhauser Comorbidity Index, and Modified Frailty Index 11-Item. Further regression analysis revealed that four out of the five mFI-5 comorbidities were independent risk factors for postsurgical complications.

CONCLUSION

This study highlights important associations between comorbidity indexes and adverse postoperative outcomes in IBBR. mFI-5 demonstrated superior performance in postoperative risk stratification compared to the other indexes analyzed. These findings suggest that using mFI-5 in clinical settings may help identify high comorbidity patients who may benefit from additional monitoring and prophylactic measures. Further research is needed to understand how to mitigate the increased postoperative complication risk in these patients.

Trends in Top Surgery Patient Characteristics, Wound Complications, and CPT Code Use by Plastic Surgeons: A Decade-Long Analysis

BACKGROUND

Gender-affirming top surgery is becoming increasingly common, with greater diversity in the patients receiving top surgery. The purpose of this study was to examine national trends in patient demographics, characteristics, wound complication rates, and concurrent procedures in patients receiving gender-affirming top surgeries.

METHODS

Patients with gender dysphoria who underwent breast procedures, including mastectomy, mastopexy, breast augmentation, or breast reduction by a plastic surgeon between 2013 and 2022, were identified from the American College of Surgeons National Surgical Quality Improvement Program database. These procedures were considered to be gender-affirming "top surgery." Univariate analyses were performed to examine trend changes in the patient population and types of additional procedures performed over the last decade.

RESULTS

There was a 38-fold increase in the number of patients who received top surgery during the most recent years compared to the first 2 years of the decade. Significantly more individuals receiving top surgery in recent years were nonbinary ( P < 0.01). There was a significant decrease in percentage of active smokers ( P < 0.01) while there was an increase in percentage of patients with diabetes ( P = 0.03). While there was a significant increase in the number of obese patients receiving top surgery ( P < 0.01), there were no differences in postoperative wound complications between the years. Significantly more patients received additional procedures ( P < 0.01) and had about a 9-fold increase in distinct number of additional CPT codes from 2013-2014 to 2021-2022.

CONCLUSIONS

Our study found that there has been (1) a significant increase in the number of top surgery patients from 2013 to 2022 overall and (2) a particular increase in patients with preoperative comorbidities, such as a higher body mass index and diabetes. Understanding current and evolving trends in patients undergoing surgical treatment for gender dysphoria can inform individualized care plans that best serve the needs of patients and optimize overall outcomes.

Evaluation of the American College of Surgeons National Surgical Quality Improvement Program Risk Calculator to predict outcomes after hysterectomies

OBJECTIVE

To evaluate the American College of Surgeons (ACS) surgical risk calculator's reliability in predicting outcomes in hysterectomies.

METHODS

This is a prospective cohort study at a large community-based hospital. Twenty-one preoperative and postoperative criteria were abstracted from the electronic medical record and entered into the online ACS calculator to determine a risk score. Logistical regression was used to determine the association between risk score and actual outcome. The prediction capability was analyzed with c-statistic, Hosmer-Lemeshow, and Brier score.

RESULTS

A total of 634 hysterectomies were performed during the study period from January to April 2019. Patients were predominantly 55 years old, white (53%) and overweight (body mass index 30). Predicted perioperative adverse events were significantly higher than actual adverse events across all domains. In all, 54/634 (8.5%) patients experienced postoperative urinary tract infection. C-statistics for return to operating room, renal failure, and readmission were 0.607 (95% C Statistic index [CI] 0.370-0.845), 0.882 (95% CI 0.802-0.962), 0.637 (95% CI 0.524-0.750), respectively. Brier scores approached one in all categorical domains.

CONCLUSION

The ACS surgical risk calculator holds the promise of predicting postoperative complications or length of stay for patients undergoing hysterectomy. Further adjustment to this tool is required before it can be advocated for use in the clinical setting.

Frailty Is Superior to Age for Predicting Readmission, Prolonged Length of Stay, and Wound Infection in Elective Otology Procedures

OBJECTIVE

To determine the predictive ability of the 5-point modified frailty index relative to age in elective otology patients.

STUDY DESIGN

Retrospective database analysis.

SETTING

Multicenter, national database of surgical patients.

PATIENTS

We selected all elective surgical patients who received tympanoplasty, tympanomastoidectomy, mastoidectomy, revision mastoidectomy, and cochlear implant procedures from 2016 to 2019 from the National Surgical Quality Improvement database.

INTERVENTIONS

Therapeutic.

MAIN OUTCOME MEASURES

Readmission rates, discharge disposition, reoperation rates, and extended length of hospital stay.

RESULTS

Utilizing receiver operating characteristics with area under the curve (AUC) analysis, nonrobust status was determined to be a superior predictor relative to age of readmission (AUC = 0.628 [p < 0.001] versus AUC = 0.567 [p = 0.047], respectively) and open wound infection relative to age (AUC = 0.636 [p = 0.024] versus AUC = 0.619 [p = 0.048], respectively). Nonrobust otology patients were more likely to have dyspnea at rest and an American Society of Anesthesiology score higher than 2 before surgery (odds ratios, 13.304 [95% confidence interval, 2.947-60.056; p < 0.001] and 7.841 [95% confidence interval, 7.064-8.704; p < 0.001], respectively).

CONCLUSION

Nonrobust status was found to be a useful predictor of readmission and prolonged length of stay in patients undergoing elective otology procedures, which generally have low complication rate. Given the aging population and corresponding increase in otology disease, it is important to use age-independent risk stratification measures. Frailty may provide a useful risk stratification tool to select surgical candidates within the aging population.

"In Free Flap Autologous Breast Reconstruction Frailty Is a More Accurate Predictor of Postoperative Complications than Age, Body Mass Index, or ASA class: A Retrospective Cohort Analysis on the ACS-NSQIP Database."

BACKGROUND

Free flap autologous breast reconstruction (f-ABR) improves quality of life in cancer survivors but has a 5-47% higher postoperative complication (PCs) rate in vulnerable patients, such as those with obesity or the elderly. Given the high (respectively: 43% and 16%) and rising prevalence of these conditions, operative risk prediction is critical to guide targeted care. Age, BMI, and ASA class have shown inaccuracies as predictive factors of PCs in f-ABR. Since frailty, a measure of vulnerability, was reported to be a reliable predictor of PCs in multiple other surgical fields, we hypothesized that it would be an accurate predictor of PCs also in f-ABR.

METHODS

Patients undergoing f-ABR (CPT: 19364) were identified using the ACS-NSQIP (American College of Surgeons-National Surgical Quality Improvement Program) database (01/2010-12/2018). Frailty was calculated using the validated modified Frailty Index (mFI). Rates of wound complications, bleeding episodes, readmissions, returns to operating room (ROR), and DVTs were compared across mFI score, BMI, age, and ASA class.

RESULTS

mFI ≥ 2 was associated with 22.22% (p <0.001) wound complications; 15.79% (p <0.001) bleeding episodes; 8.20% (p <0.001) readmissions; 17.19% (p <0.001) ROR; and 1.81% (p <0.05) DVTs. Higher BMI, age, and ASA class did not significantly correlate with increased rates in one or more PCs. Only a high mFI was consistently associated with significantly higher odds of complications in all complication types.

CONCLUSIONS

As a reliable and accurate predictor of PCs in f-ABR, frailty could be used preoperatively to counsel patients and guide surgical care.

A Case for the Use of the 5-Item Modified Frailty Index in Preoperative Risk Assessment for Tissue Expander Placement in Breast Reconstruction

BACKGROUND

Preoperative risk assessment is essential in determining which surgical candidates will have the most to gain from an operation. The 5-item modified frailty index (mFI-5) has been validated as an effective way to determine this risk. This study sought to evaluate the performance of the mFI-5 as a predictor of postoperative complications after tissue expander placement.

METHODS

Patients who underwent placement of a tissue expander were identified using the 2012 to 2018 American College of Surgeons National Surgical Quality Improvement Project database. Univariate and multivariate regression analysis models were used to assess how mFI-5, the components of the mFI-5 (functional status, diabetes, chronic obstructive pulmonary disease, chronic heart failure, and hypertension), and other factors commonly used to risk stratify (age, body mass index [BMI], American Society of Anesthesiologists (ASA) classification, and history of smoking) were associated with complications.

RESULTS

In 44,728 tissue expander placement cases, the overall complication rate was 10.5% (n = 4674). The mFI-5 score was significantly higher in the group that experienced complications (0.08 vs 0.06, P < 0.001). Compared with the mFI-5 individual components and other common variables used preoperatively to risk stratify patients, univariate analysis demonstrated that mFI-5 had the largest effect size (odds ratio [OR], 5.46; confidence interval [CI], 4.29-6.94; P < 0.001). After controlling for age, BMI, ASA classification, and history of smoking, the mFI-5 still remained the predictor of complications with the largest effect size (OR, 2.25; CI, 1.70-2.97; P < 0.001). In assessing specific complications, the mFI-5 is the independent predictor with the largest significant effect size for surgical dehiscence (OR, 12.76; CI, 5.58-28.18; P < 0.001), surgical site infection (OR, 6.68; CI, 4.53-9.78; P < 0.001), reoperation (OR, 5.23; CI, 3.90-6.99; P < 0.001), and readmission (OR, 4.59; CI, 3.25-6.45; P < 0.001) when compared with age, BMI, ASA class, and/or history of smoking alone.

CONCLUSIONS

The mFI-5 can be used as an effective preoperative predictor of postoperative complications in patients undergoing tissue expander placement. Not only does it have the largest effect size compared with other historical perioperative risk factors, it is more predictive than each of its individual components.

An Ounce of Prediction is Worth a Pound of Cure: Risk Calculators in Breast Reconstruction

Preoperative risk calculators provide individualized risk assessment and stratification for surgical patients. Recently, several general surgery–derived models have been applied to the plastic surgery patient population, and several plastic surgery–specific calculators have been developed. In this scoping review, the authors aimed to identify and critically appraise risk calculators implemented in postmastectomy breast reconstruction.

Evaluating the Inaccuracy of the National Surgical Quality Improvement Project Surgical Risk Calculator in Plastic Surgery: A Meta-analysis of Short-Term Predicted Complications

BACKGROUND

Preoperative surgical risk assessment is a major component of clinical decision making. The ability to provide accurate, individualized risk estimates has become critical because of growing emphasis on quality metrics benchmarks. The American College of Surgeons National Surgical Quality Improvement Project (NSQIP) Surgical Risk Calculator (SRC) was designed to quantify patient-specific risk across various surgeries. Its applicability to plastic surgery is unclear, however, with multiple studies reporting inaccuracies among certain patient populations. This study uses meta-analysis to evaluate the NSQIP SRC's ability to predict complications among patients having plastic surgery.

METHODS

OVID MEDLINE and PubMed were searched for all studies evaluating the predictive accuracy of the NSQIP SRC in plastic surgery, including oncologic reconstruction, ventral hernia repair, and body contouring. Only studies directly comparing SCR predicted to observed complication rates were included. The primary measure of SRC prediction accuracy, area under the curve (AUC), was assessed for each complication via DerSimonian and Laird random-effects analytic model. The I2 statistic, indicating heterogeneity, was judged low (I2 < 50%) or borderline/unacceptably high (I2 > 50%). All analyses were conducted in StataSE 16.1 (StataCorp LP, College Station, Tex).

RESULTS

Ten of the 296 studies screened met criteria for inclusion (2416 patients). Studies were classified as follows: (head and neck: n = 5, breast: n = 1, extremity: n = 1), open ventral hernia repair (n = 2), and panniculectomy (n = 1). Predictive accuracy was poor for medical and surgical complications (medical: pulmonary AUC = 0.67 [0.48-0.87], cardiac AUC = 0.66 [0.20-0.99], venous thromboembolism AUC = 0.55 [0.47-0.63]), (surgical: surgical site infection AUC = 0.55 [0.46-0.63], reoperation AUC = 0.54 [0.49-0.58], serious complication AUC = 0.58 [0.43-0.73], and any complication AUC = 0.60 [0.57-0.64]). Although mortality was accurately predicted in 2 studies (AUC = 0.87 [0.54-0.99]), heterogeneity was high with I2 = 68%. Otherwise, heterogeneity was minimal (I2 = 0%) or acceptably low (I2 < 50%) for all other outcomes.

CONCLUSIONS

The NSQIP Universal SRC, aimed at offering individualized quantifiable risk estimates for surgical complications, consistently demonstrated poor risk discrimination in this plastic surgery-focused meta-analysis. The limitations of the SRC are perhaps most pronounced where complex, multidisciplinary reconstructions are needed. Future efforts should identify targets for improving SRC reliability to better counsel patients in the perioperative setting and guide appropriate healthcare resource allocation.

Validation of a risk prediction calculator in Australian patients undergoing head and neck microsurgery reconstruction

BACKGROUND

The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) surgical risk calculator (SRC) is an open access calculator predicting patients' risk of postoperative complications. This study aims to assess the validity of the SRC in patients undergoing microsurgical free flap reconstruction at an Australian tertiary referral centre.

METHODS

This is a retrospective cohort study of 200 consecutive patients treated up to November 2020. SRC-predicted rates of postoperative complications and hospital length of stay (LOS) were compared to those observed for the ablative and reconstructive components of the procedure. The performance of the SRC was assessed using Brier scores, area under the receiver operating characteristic (ROC) curve (AUC), and the Hosmer-Lemeshow test.

RESULTS

For both ablative and reconstructive components, the SRC discriminates well for pneumonia and urinary tract infection, and it is calibrated well for readmission and sepsis, but it does not discriminate and calibrate well for any single outcome. SRC-predicted hospital LOS and actual LOS did not correlate well for the reconstructive component, but they correlated strongly for the ablative component.

CONCLUSIONS

The SRC is a poor predictor of postoperative complication rates and hospital LOS in patients undergoing head and neck microsurgical reconstruction.

A non-linear ensemble model-based surgical risk calculator for mixed data from multiple surgical fields

BACKGROUND

The misestimation of surgical risk is a serious threat to the lives of patients when implementing surgical risk calculator. Improving the accuracy of postoperative risk prediction has received much attention and many methods have been proposed to cope with this problem in the past decades. However, those linear approaches are inable to capture the non-linear interactions between risk factors, which have been proved to play an important role in the complex physiology of the human body, and thus may attenuate the performance of surgical risk calculators.

METHODS

In this paper, we presented a new surgical risk calculator based on a non-linear ensemble algorithm named Gradient Boosting Decision Tree (GBDT) model, and explored the corresponding pipeline to support it. In order to improve the practicability of our approach, we designed three different modes to deal with different data situations. Meanwhile, considering that one of the obstacles to clinical acceptance of surgical risk calculators was that the model was too complex to be used in practice, we reduced the number of input risk factors according to the importance of them in GBDT. In addition, we also built some baseline models and similar models to compare with our approach.

RESULTS

The data we used was three-year clinical data from Surgical Outcome Monitoring and Improvement Program (SOMIP) launched by the Hospital Authority of Hong Kong. In all experiments our approach shows excellent performance, among which the best result of area under curve (AUC), Hosmer-Lemeshow test ([Formula: see text]) and brier score (BS) can reach 0.902, 7.398 and 0.047 respectively. After feature reduction, the best result of AUC, [Formula: see text] and BS of our approach can still be maintained at 0.894, 7.638 and 0.060, respectively. In addition, we also performed multiple groups of comparative experiments. The results show that our approach has a stable advantage in each evaluation indicator.

CONCLUSIONS

The experimental results demonstrate that NL-SRC can not only improve the accuracy of predicting the surgical risk of patients, but also effectively capture important risk factors and their interactions. Meanwhile, it also has excellent performance on the mixed data from multiple surgical fields.

Mining Pre-Surgical Patterns Able to Discriminate Post-Surgical Outcomes in the Oncological Domain

Understanding the individualized risks of undertaking surgical procedures is essential to personalize preparatory, intervention and post-care protocols for minimizing post-surgical complications. This knowledge is key in oncology given the nature of interventions, the fragile profile of patients with comorbidities and cytotoxic drug exposure, and the possible cancer recurrence. Despite its relevance, the discovery of discriminative patterns of post-surgical risk is hampered by major challenges: i) the unique physiological and demographic profile of individuals, as well as their differentiated post-surgical care; ii) the high-dimensionality and heterogeneous nature of available biomedical data, combining non-identically distributed risk factors, clinical and molecular variables; iii) the need to generalize tumors have significant histopathological differences and individuals undertake unique surgical procedures; iv) the need to focus on non-trivial patterns of post-surgical risk, while guaranteeing their statistical significance and discriminative power; and v) the lack of interpretability and actionability of current approaches. Biclustering, the discovery of groups of individuals correlated on subsets of variables, has unique properties of interest, being positioned to satisfy the aforementioned challenges. In this context, this work proposes a structured view on why, when and how to apply biclustering to mine discriminative patterns of post-surgical risk with guarantees of usability, a subject remaining unexplored up to date. These patterns offer a comprehensive view on how the patient profile, cancer histopathology and entailed surgical procedures determine: i) post-surgical complications, ii) survival, and iii) hospitalization needs. The gathered results confirm the role of biclustering in comprehensively finding interpretable, actionable and statistically significant patterns of post-surgical risk. The found patterns are already assisting healthcare professionals at IPO-Porto to establish specialized pre-habilitation protocols and bedside care.

Inadequate prediction of postoperative complications in breast cancer surgery: An evaluation of the ACS Surgical Risk Calculator

BACKGROUND

The American College of Surgeon (ACS) Surgical Risk Calculator is an online tool that helps surgeons estimate the risk of postoperative complications for numerous surgical procedures across several surgical specialties.

METHODS

We evaluated the predictive performance of the calculator in 385 cancer patients undergoing breast surgery. Calculator-predicted complication rates were compared with observed complication rates; calculator performance was evaluated using calibration and discrimination analyses.

RESULTS

The mean calculator-predicted rates for any complication (4.1%) and serious complication (3.2%) were significantly lower than the observed rates (11.2% and 5.2%, respectively). The area under the curve was 0.617 for any complication and 0.682 for serious complications. p Values for the Hosmer-Lemeshow test were significant (<.05) for both outcomes. Brier scores were 0.102 for any complication and 0.048 for serious complication.

CONCLUSIONS

The ACS risk calculator is not an ideal tool for predicting individual risk of complications following breast surgery in a Mexican cohort. The most valuable use of the calculator may reside in its role as an aid for patient-led surgery planning. The possibility of introducing breast surgery-specific data could improve the performance of the calculator. Furthermore, a disease-specific calculator could provide more accurate predictions and include complications more frequently found in breast cancer surgery.

A Comparison of Common Plastic Surgery Operations Using the NSQIP and TOPS Databases

Both the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) and the American Society of Plastic Surgeons Tracking Operations and Outcomes for Plastic Surgeons (TOPS) databases track 30-day outcomes.

Methods

Using the 2008-2016 TOPS and NSQIP databases, we compared patient characteristics and postoperative outcomes for 5 common plastic surgery procedures. A weighted TOPS population was used to mirror the NSQIP population in clinical and demographic characteristics to compare postoperative outcomes.

Results

We identified 154,181 cases. Compared with NSQIP patients, TOPS patients were more likely to be younger (47.9 versus 50.0 years), have American Society of Anesthesiologists class I-II (92.1% versus 74.6%), be outpatient (66.0% versus 49.3%), and be smokers (18.7% versus 11.7%). TOPS had extensive missing data: body mass index (40.6%), American Society of Anesthesiologists class (34.9%), diabetes (39.3%), and smoking status (37.2%). NSQIP was missing <1% of all shared categories except race (15.6%). The entire TOPS cohort versus only TOPS patients without missing data had higher rates of dehiscence (5.1% versus 3.5%) and infection (2.1% versus 1.7%). TOPS versus NSQIP patients had higher dehiscence rates (5.1% versus 1.0%) but lower rates of return to the operating room (3.1% versus 6.6%), infection (2.1% versus 3.0%), and medical complications (0.3% versus 2.2%). Nonweighted and weighted TOPS cohorts had similar 30-day outcomes.

Conclusions

NSQIP and TOPS populations are different in characteristics and outcomes, likely due to differences in collection methodology and the types physicians using the databases. The strengths of each dataset can be used together for research and quality improvement.

No news is good news? Three-year postdischarge mortality of octogenarian and nonagenarian patients following emergency general surgery

BACKGROUND

Outcome data on the very elderly patients undergoing emergency general surgery (EGS) are sparse. We sought to examine short- and long-term mortality in the 80 plus years population following EGS.

METHODS

Using our institutional 2008-2018 EGS Database, all the 80 plus years patients undergoing EGS were identified. The data were linked to the Social Security Death Index to determine cumulative mortality rates up to 3 years after discharge. Univariate and multivariable logistic regression analyses were used to determine predictors of in-hospital and 1-year cumulative mortality.

RESULTS

A total of 385 patients were included with a mean age of 84 years; 54% were female. The two most common comorbidities were hypertension (76.1%) and cardiovascular disease (40.5%). The most common procedures performed were colectomy (20.0%), small bowel resection (18.2%), and exploratory laparotomy for other procedures (15.3%; e.g., internal hernia, perforated peptic ulcer). The overall in-hospital mortality was 18.7%. Cumulative mortality rates at 1, 2, and 3 years after discharge were 34.3%, 40.5%, and 43.4%, respectively. The EGS procedure associated with the highest 1-year mortality was colectomy (49.4%). Although hypertension, renal failure, hypoalbuminemia, hyperbilirubinemia, and elevated liver enzymes predicted in-hospital mortality, the only independent predictors of cumulative 1-year mortality were hypoalbuminemia (odds ratio, 2.17; 95% confidence interval, 1.10-4.27; p = 0.025) and elevated serum glutamic pyruvic transaminase (SGOT) level (odds ratio, 2.56; 95% confidence interval, 1.09-4.70; p = 0.029) at initial presentation. Patients with both factors had a cumulative 1-year mortality rate of 75.0%.

CONCLUSION

More than half of the very elderly patients undergoing major EGS were still alive at 3 years postdischarge. The combination of hypoalbuminemia and elevated liver enzymes predicted the highest 1-year mortality. Such information can prove useful for patient and family counseling preoperatively.

LEVEL OF EVIDENCE

Prognostic, Level III.

Predicting Post-operative Complications in Crohn's Disease: an Appraisal of Clinical Scoring Systems and the NSQIP Surgical Risk Calculator

BACKGROUND

Surgery is common in patients with Crohn's disease and can contribute significantly to patient morbidity. The National Surgical Quality Improvement Program surgical risk calculator (NSQIP-SRC) that is currently utilized to predict surgical risk does not take Crohn's disease into account and, as a result, seems to underestimate risk in this patient population. This study aimed to evaluate the accuracy of the NSQIP-SRC in Crohn's disease patients and to evaluate the utility of disease severity scores in predicting surgical risk.

METHODS

Between 2011 and 2017, there were 176 surgical cases involving Crohn's disease patients. Demographic data and 30-day surgical outcomes were collected. Disease severity scores including Harvey Bradshaw Index (HBI), Crohn's Disease Activity Index (CDAI), Simple Endoscopic Score for Crohn's Disease (SES-CD), and NSQIP-SRC risk percentages were calculated.

RESULTS

Patients in remission based on HBI had a complication rate of 8.57% (n = 3), while those with mild or moderate-severe disease had rates of 33.33% (n = 11) and 38.46% (n = 20) respectively (p = 0.0045). In multivariable analysis, those with mild (OR; 8.37, 95% CI; 1.64, 42.78; p = 0.011) or moderate-severe (OR; 11.69, 95% CI; 2.42, 56.46; p = 0.002) disease had increased odds of complication compared to remission. Complication rate was not associated with NSQIP-SRC percent risk of any complication.

CONCLUSION

NSQIP-SRC does not accurately predict risk in patients with CD undergoing surgery. Higher disease activity based on HBI is associated with increased odds of complication and may prove to be more predictive of surgical complication in the Crohn's patient population.

Predicting complications in immediate microvascular breast reconstruction: Validity of the breast reconstruction assessment (BRA) surgical risk calculator

BACKGROUND

The Breast Reconstruction Assessment (BRA)-score is a disease-specific risk calculator that estimates the likelihood of postoperative complications in an individual patient. The tool has not been previously externally validated in microvascular breast reconstruction. The purpose of this study was to evaluate the efficacy of the calculator in patients who underwent microvascular reconstruction at a single specialist institution.

METHODS

Data from 415 patients who had immediate microvascular breast reconstruction were entered into the calculator. The predicted and observed rates of surgical complications, medical complications, reoperation, and total or partial flap failure were compared. The accuracy of the calculator was assessed using statistical measures of calibration and discrimination.

RESULTS

The calculator accurately predicted the proportion of patients who would experience surgical complications and reoperations but overestimated the rates of medical complications and flap failures. The C-statistics were low for all four prediction models (0.49-0.59), suggesting weak discriminatory power, and the Brier scores were relatively high (0.09-0.44), indicating poor correlation between predicted and actual probability of complications.

CONCLUSION

These results suggest that the BRA score cannot accurately identify patients at risk for complications following immediate microvascular breast reconstruction at our institution.

Predicting Postoperative Complications after Acute Care Surgery: How Accurate is the ACS NSQIP Surgical Risk Calculator?

The ACS NSQIP Surgical Risk Calculator (SRC) is an evidence-based clinical tool commonly used for evaluating postoperative risk. The goal of this study was to validate SRC-predicted complications by comparing them with observed outcomes in the acute care surgical setting. In this study, pre- and postoperative data from 1693 acute care surgeries (hernia repair, enterolysis, intestinal incision/excision and enterectomy, gastrectomy, debridement, colectomy, appendectomy, cholecystectomy, gastrorrhaphy, and incision and drainage of soft tissue, breast abscesses, and removal of foreign bodies) performed at a Level I trauma center over a five-year time period were abstracted. Predictions for any and serious complications were based on SRC were compared with observed outcomes using various measures of diagnostic. When evaluated as one group, the SRC had good discriminative power for predicting any and serious complications after acute care surgeries (Area Under the Curve (AUC) 0.79, 0.81). In addition, the SRC met Brier score requirements for an informative model overall. However, the predictive accuracy of the SRC varied for various procedures within the acute care patient population. For serious complications, the diagnostic measures ranged from an AUC of 0.61 and negative likelihood ratio of 0.716 for incision & drainage soft tissue to AUC of 0.91 and negative likelihood ratio of 0.064 for gastrorrhaphy. Length of stay was significantly underestimated by the SRC overall (8.56 days, P < 0.01) and for individual procedures. The SRC performs well at predicting complications after acute care surgeries overall; however, there is great variability in performance between procedure types. Further refinements in risk stratification may improve SRC predictions.

Scoping Review of the National Surgical Quality Improvement Program in Plastic Surgery Research

BACKGROUND

The National Surgical Quality Improvement Program (NSQIP) is a robust, high-quality surgical outcomes database that measures risk-adjusted 30-day outcomes of surgical interventions. The purpose of this scoping review is to describe how the NSQIP is being used in plastic surgery research.

METHODS

A comprehensive electronic literature search was completed in PubMed, Embase, MEDLINE, and CINAHL. Two reviewers independently reviewed articles to determine their relevance using predefined inclusion criteria. Articles were included if they utilized NSQIP data to conduct research in a domain of plastic surgery or analyzed surgical procedures completed by plastic surgeons. Extracted information included the domain of plastic surgery, country of origin, journal, and year of publication.

RESULTS

A total of 106 articles met the inclusion criteria. The most common domain of plastic surgery was breast reconstruction representing 35% of the articles. Of the 106 articles, 95% were published within the last 5 years. The Plastic and Reconstructive Surgery journal published most of the (59%) NSQIP-related articles. All of the studies were retrospective. Of note, there were no articles on burns and only one study on trauma as the domain of plastic surgery.

CONCLUSION

This scoping review describes how NSQIP data are being used to analyze plastic surgery interventions and outcomes in order to guide quality improvement in 106 articles. It demonstrates the utility of NSQIP in the literature, however also identifies some limitations of the program as it applies to plastic surgery.

Predicting complications of major head and neck oncological surgery: an evaluation of the ACS NSQIP surgical risk calculator

Background

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient’s risk of developing specific postoperative outcomes.

Methods

Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed–ranked test.

Results

The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6).

Conclusion

The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

The National Surgical Quality Improvement Program 30-Day Challenge: Microsurgical Breast Reconstruction Outcomes Reporting Reliability

Supplemental Digital Content is available in the text.

Background:

The aim was to assess reliability of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) 30-day perioperative outcomes and complications for immediate, free-tissue transfer breast reconstruction by direct comparisons with our 30-day and overall institutional data, and assessing those that occur after 30 days.

Methods:

Data were retrieved for consecutive immediate, free-tissue transfer breast reconstruction patients from a single-institution database (2010–2015) and the ACS-NSQIP (2011–2014). Multiple logistic regressions were performed to compare adjusted outcomes between the 2 datasets.

Results:

For institutional versus ACS-NSQIP outcomes, there were no significant differences in surgical-site infection (SSI; 30-day, 3.6% versus 4.1%, P = 0.818; overall, 5.3% versus 4.1%, P = 0.198), wound disruption (WD; 30-day, 1.3% versus 1.5%, P = 0.526; overall, 2.3% versus 1.5%, P = 0.560), or unplanned readmission (URA; 30-day, 2.3% versus 3.3%, P = 0.714; overall, 4.6% versus 3.3%, P = 0.061). However, the ACS-NSQIP reported a significantly higher unplanned reoperation (URO) rate (30-day, 3.6% versus 9.5%, P < 0.001; overall, 5.3% versus 9.5%, P = 0.025). Institutional complications consisted of 5.3% SSI, 2.3% WD, 5.3% URO, and 4.6% URA, of which 25.0% SSI, 28.6% WD, 12.5% URO, and 7.1% URA occurred at 30–60 days, and 6.3% SSI, 14.3% WD, 18.8% URO, and 42.9% URA occurred after 60 days.

Conclusion:

For immediate, free-tissue breast reconstruction, the ACS-NSQIP may be reliable for monitoring and comparing SSI, WD, URO, and URA rates. However, clinicians may find it useful to understand limitations of the ACS-NSQIP for complications and risk factors, as it may underreport complications occurring beyond 30 days.

Comparison of 4 Cardiac Risk Calculators in Predicting Postoperative Cardiac Complications After Noncardiac Operations

The 2014 American College of Cardiology/American Heart Association Perioperative Guidelines suggest using the Revised Cardiac Risk Index, myocardial infarction or cardiac arrest, or American College of Surgeons-National Surgical Quality Improvement Program calculators for combined patient-surgical risk assessment. There are no published data comparing their performance. This study compared these risk calculators and a reconstructed Revised Cardiac Risk Index in predicting postoperative cardiac complications, both during hospitalization and 30 days after operation, in a patient cohort who underwent select surgical procedures in various risk categories. Cardiac complications occurred in 14 of 663 patients (2.1%), of which 11 occurred during hospitalization. Only 3 of 663 patients (0.45%) had a myocardial infarction or cardiac arrest. Because these calculators used different risk factors, different outcomes, and different durations of observation, a true direct comparison is not possible. We found that all 4 risk calculators performed well in the setting they were originally studied but were less accurate when applied in a different manner. In conclusion, all calculators were useful in defining low-risk patients in whom further cardiac testing was unnecessary, and the myocardial infarction or cardiac arrest may be the most reliable in selecting higher risk patients.

An Examination of American College of Surgeons NSQIP Surgical Risk Calculator Accuracy

BACKGROUND

The American College of Surgeons NSQIP offers a Surgical Risk Calculator (SRC) that provides detailed, patient-level, risk assessments for many adverse outcomes to surgeons, patients, and the general public. The SRC calculator was designed to help guide discussion and decisions by providing generally applicable (not hospital-specific) information about surgical risk using easily understood and broadly available preoperative variables. Although large, internal evaluations have shown that the SRC has good accuracy (model discrimination and calibration), external validations have been inconsistent and tend to favor a conclusion of inadequate performance.

STUDY DESIGN

External studies, attempting to validate the SRC, were examined with respect to 3 design features: sample size (small samples reduce reliability), case-mix homogeneity (homogeneity reduces discrimination); and number of institutions providing data (few institutions reduces generalizability). The impact of each feature was then examined in several sets of simulation studies.

RESULTS

Each of the 3 design features has the potential to act as an artifactual cause for apparent SRC predictive failure. In addition, demonstrations that SRC estimates are inferior to those from models that use additional (sometimes operation-specific) predictor variables were seen as not relevant with respect to the SRC's intended scope.

CONCLUSIONS

The SRC predictive failures, reported by studies with the described design limitations, should not be misunderstood as disqualifying the SRC as an accurate and appropriate tool for its intended purpose of providing a general purpose risk calculator, applicable across many surgical domains, using easily understood and generally available predictive information.