Failure of Colorectal Surgical Site Infection Predictive Models Applied to an Independent Dataset: Do They Add Value or Just Confusion?

Evaluation of ACS-NSQIP and CR-POSSUM risk calculators for the prediction of mortality after colorectal surgery: A retrospective cohort study

BACKGROUND

Several risk calculating tools have been introduced into clinical practice to provide patients and clinicians with objective, individualised estimates of procedure-related unfavourable outcomes. The currently available risk calculators (RCs) have been developed by well-endowed health systems in Europe and the USA. Applicability of these RCs in low-middle income country (LMIC) settings with wide disparities in patient population, surgical practice and healthcare infrastructure has not been adequately examined.

PATIENTS AND METHODS

Through this single tertiary care, LMIC-centre, retrospective cohort study, we investigated the accuracy of the two most widely validated RCs - American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) RC and ColoRectal Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity (CR-POSSUM) - for the prediction of mortality in patients undergoing elective and emergency colorectal surgery (CRS) from March 2013 to March 2020. Online RCs were used to predict mortality and other outcomes. Accuracy was assessed by Brier score and C statistic.

RESULTS

Of 105 patients, 69 (65.71%) underwent elective and 36 (34.28%) underwent emergency CRS. The 30-day overall mortality was 12 - elective 1 (1.4%) and emergency 11 (30.5%). ACS-NSQIP RC performed better for the prediction of overall ( C statistic 0.939, Brier score 0.065) and emergency ( C statistic 0.840, Brier score 0.152) mortality. However, for elective CRS mortality, Brier scores were similar for both models (0.014), whereas C statistic (0.934 vs. 0.890) value was better for ACS-NSQIP.

CONCLUSIONS

Both ACS-NSQIP and CR-POSSUM were accurate for the prediction of CRS mortality. However, compared to CR-POSSUM, ACS-NSQIP performed better. The overall performance of both models is indicative of their wider applicability in LMIC centres also.

Exploring surgical infection prediction: A comparative study of established risk indexes and a novel model

BACKGROUND

Surgical site infections are a major health problem that deteriorates the patients' health and increases health care costs. A reliable method to identify patients with modifiable risk of surgical site infection is necessary to reduce the incidence of them but data are limited. Hence the objective is to assess the predictive validity of a logistic regression model compared to risk indexes to identify patients at risk of surgical site infections.

METHODS

In this study, we evaluated the predictive validity of a new model which incorporates important predictors based on logistic regression model compared to three state-of-the-art risk indexes to identify high risk patients, recruited from 2016 to 2020 from a medium size hospital in North Norway, prone to surgical site infection.

RESULTS

The logistic regression model demonstrated significantly higher scores, defined as high-risk, in 110 patients with surgical site infections than in 110 patients without surgical site infections (p < 0.001, CI 19-44) compared to risk indexes. The logistic regression model achieved an area under the curve of 80 %, which was better than the risk indexes SSIRS (77 %), NNIS (59 %), and JSS-SSI (52 %) for predicting surgical site infections. The logistic regression model identified operating time and length of stay as the major predictors of surgical site infections.

CONCLUSIONS

The logistic regression model demonstrated better performance in predicting surgical site infections compared to three state-of-the-art risk indexes. The model could be further developed into a decision support tool, by incorporating predictors available prior to surgery, to identify patients with modifiable risk prone to surgical site infection.

Improved Prediction of Surgical-Site Infection After Colorectal Surgery Using Machine Learning

BACKGROUND

Surgical-site infection is a source of significant morbidity after colorectal surgery. Previous efforts to develop models that predict surgical-site infection have had limited accuracy. Machine learning has shown promise in predicting postoperative outcomes by identifying nonlinear patterns within large data sets.

OBJECTIVE

This study aimed to seek usage of machine learning to develop a more accurate predictive model for colorectal surgical-site infections.

DESIGN

Patients who underwent colorectal surgery were identified in the American College of Surgeons National Quality Improvement Program database from years 2012 to 2019 and were split into training, validation, and test sets. Machine-learning techniques included random forest, gradient boosting, and artificial neural network. A logistic regression model was also created. Model performance was assessed using area under the receiver operating characteristic curve.

SETTINGS

A national, multicenter data set.

PATIENTS

Patients who underwent colorectal surgery.

MAIN OUTCOME MEASURES

The primary outcome (surgical-site infection) included patients who experienced superficial, deep, or organ-space surgical-site infections.

RESULTS

The data set included 275,152 patients after the application of exclusion criteria. Of all patients, 10.7% experienced a surgical-site infection. Artificial neural network showed the best performance with area under the receiver operating characteristic curve of 0.769 (95% CI, 0.762-0.777), compared with 0.766 (95% CI, 0.759-0.774) for gradient boosting, 0.764 (95% CI, 0.756-0.772) for random forest, and 0.677 (95% CI, 0.669-0.685) for logistic regression. For the artificial neural network model, the strongest predictors of surgical-site infection were organ-space surgical-site infection present at time of surgery, operative time, oral antibiotic bowel preparation, and surgical approach.

LIMITATIONS

Local institutional validation was not performed.

CONCLUSIONS

Machine-learning techniques predict colorectal surgical-site infections with higher accuracy than logistic regression. These techniques may be used to identify patients at increased risk and to target preventive interventions for surgical-site infection. See Video Abstract at http://links.lww.com/DCR/C88 .

PREDICCIN MEJORADA DE LA INFECCIN DEL SITIO QUIRRGICO DESPUS DE LA CIRUGA COLORRECTAL MEDIANTE EL APRENDIZAJE AUTOMTICO

ANTECEDENTES:La infección del sitio quirúrgico es una fuente de morbilidad significativa después de la cirugía colorrectal. Los esfuerzos anteriores para desarrollar modelos que predijeran la infección del sitio quirúrgico han tenido una precisión limitada. El aprendizaje automático se ha mostrado prometedor en la predicción de los resultados posoperatorios mediante la identificación de patrones no lineales dentro de grandes conjuntos de datos.OBJETIVO:Intentamos utilizar el aprendizaje automático para desarrollar un modelo predictivo más preciso para las infecciones del sitio quirúrgico colorrectal.DISEÑO:Los pacientes que se sometieron a cirugía colorrectal se identificaron en la base de datos del Programa Nacional de Mejoramiento de la Calidad del Colegio Estadounidense de Cirujanos de los años 2012 a 2019 y se dividieron en conjuntos de capacitación, validación y prueba. Las técnicas de aprendizaje automático incluyeron conjunto aleatorio, aumento de gradiente y red neuronal artificial. También se creó un modelo de regresión logística. El rendimiento del modelo se evaluó utilizando el área bajo la curva característica operativa del receptor.CONFIGURACIÓN:Un conjunto de datos multicéntrico nacional.PACIENTES:Pacientes intervenidos de cirugía colorrectal.PRINCIPALES MEDIDAS DE RESULTADO:El resultado primario (infección del sitio quirúrgico) incluyó pacientes que experimentaron infecciones superficiales, profundas o del espacio de órganos del sitio quirúrgico.RESULTADOS:El conjunto de datos incluyó 275.152 pacientes después de la aplicación de los criterios de exclusión. El 10,7% de los pacientes presentó infección del sitio quirúrgico. La red neuronal artificial mostró el mejor rendimiento con el área bajo la curva característica operativa del receptor de 0,769 (IC del 95 %: 0,762 - 0,777), en comparación con 0,766 (IC del 95 %: 0,759 - 0,774) para el aumento de gradiente, 0,764 (IC del 95 %: 0,756 - 0,772) para conjunto aleatorio y 0,677 (IC 95% 0,669 - 0,685) para regresión logística. Para el modelo de red neuronal artificial, los predictores más fuertes de infección del sitio quirúrgico fueron la infección del sitio quirúrgico del espacio del órgano presente en el momento de la cirugía, el tiempo operatorio, la preparación intestinal con antibióticos orales y el abordaje quirúrgico.LIMITACIONES:No se realizó validación institucional local.CONCLUSIONES:Las técnicas de aprendizaje automático predicen infecciones del sitio quirúrgico colorrectal con mayor precisión que la regresión logística. Estas técnicas se pueden usar para identificar a los pacientes con mayor riesgo y para orientar las intervenciones preventivas para la infección del sitio quirúrgico. Consulte Video Resumen en http://links.lww.com/DCR/C88 . (Traducción-Dr Yolanda Colorado ).

Reliability and validity of multicentre surveillance of surgical site infections after colorectal surgery

Background

Surveillance is the cornerstone of surgical site infection prevention programs. The validity of the data collection and awareness of vulnerability to inter-rater variation is crucial for correct interpretation and use of surveillance data. The aim of this study was to investigate the reliability and validity of surgical site infection (SSI) surveillance after colorectal surgery in the Netherlands.

Methods

In this multicentre prospective observational study, seven Dutch hospitals performed SSI surveillance after colorectal surgeries performed in 2018 and/or 2019. When executing the surveillance, a local case assessment was performed to calculate the overall percentage agreement between raters within hospitals. Additionally, two case-vignette assessments were performed to estimate intra-rater and inter-rater reliability by calculating a weighted Cohen’s Kappa and Fleiss’ Kappa coefficient. To estimate the validity, answers of the two case-vignettes questionnaires were compared with the answers of an external medical panel.

Results

1111 colorectal surgeries were included in this study with an overall SSI incidence of 8.8% (n = 98). From the local case assessment it was estimated that the overall percent agreement between raters within a hospital was good (mean 95%, range 90–100%). The Cohen’s Kappa estimated for the intra-rater reliability of case-vignette review varied from 0.73 to 1.00, indicating substantial to perfect agreement. The inter-rater reliability within hospitals showed more variation, with Kappa estimates ranging between 0.61 and 0.94. In total, 87.9% of the answers given by the raters were in accordance with the medical panel.

Conclusions

This study showed that raters were consistent in their SSI-ascertainment (good reliability), but improvements can be made regarding the accuracy (moderate validity). Accuracy of surveillance may be improved by providing regular training, adapting definitions to reduce subjectivity, and by supporting surveillance through automation.

Development and Validation of a Model for Predicting Surgical Site Infection After Pelvic Organ Prolapse Surgery

IMPORTANCE

Surgical site infection (SSI) is a common and costly complication. Targeted interventions in high-risk patients may lead to a reduction in SSI; at present, there is no method to consistently identify patients at increased risk of SSI.

OBJECTIVE

The aim of this study was to develop and validate a model for predicting risk of SSI after pelvic organ prolapse surgery.

STUDY DESIGN

Women undergoing surgery between 2011 and 2017 were identified using Current Procedural Terminology codes from the Centers for Medicare and Medicaid Services 5% Limited Data Set. Surgical site infection ≤90 days of surgery was the primary outcome, with 41 candidate predictors identified, including demographics, comorbidities, and perioperative variables. Generalized linear regression was used to fit a full specified model, including all predictors and a reduced penalized model approximating the full model. Model performance was measured using the c-statistic, Brier score, and calibration curves. Accuracy measures were internally validated using bootstrapping to correct for bias and overfitting. Decision curves were used to determine the net benefit of using the model.

RESULTS

Of 12,334 women, 4.7% experienced SSI. The approximated model included 10 predictors. Model accuracy was acceptable (bias-corrected c-statistic [95% confidence interval], 0.603 [0.578-0.624]; Brier score, 0.045). The model was moderately calibrated when predicting up to 5-6 times the average risk of SSI between 0 and 25-30%. There was a net benefit for clinical use when risk thresholds for intervention were between 3% and 12%.

CONCLUSIONS

This model provides estimates of probability of SSI within 90 days after pelvic organ prolapse surgery and demonstrates net benefit when considering prevention strategies to reduce SSI.

Effect of closed incision negative pressure wound therapy on incidence rate of surgical site infection after stoma reversal: a pilot study

Introduction

The stoma reversal (SR) procedure is associated with a relatively high risk of perioperative complications with surgical site infection (SSI) as the most common. Recently closed incision negative pressure wound therapy (ciNPWT) was applied widely to prevent SSI.

Aim

To investigate the efficiency of ciNPWT in terms of the incidence rate of SSI after SR surgery.

Material and methods

As an exploratory observational cohort study patients were treated either with ciNPWT (n = 15) or standard sterile dressing (SSD) (n = 15). CiNPWT was applied every 3 days whereas SSD was changed every day. Clinical evaluation for SSI signs, C-reactive protein level and pain assessment using the visual analogue scale (VAS) were analyzed.

Results

The incidence rate of SSI was in 13% (2/15) in the ciNPWT group and 26% (4/15) in the SSD group (p = 0.651, OR = 0.44, 95% CI: 0.03–3.73). All patients in the SSD group who developed SSI presented both local and generalized signs of infection. Pain-VAS levels assessed on the 1^st (MdnciNPWT = 4, MdnSSD = 5, p = 0.027, W = 51.5) and 3^rd postoperative day (MdnciNPWT = 2, MdnSSD = 4, p = 0.014, W = 45.5) were significantly lower in the ciNPWT group than in the SSD group.

Conclusions

CiNPWT seems not to have a benefit to reduce SSI after the SR procedure. Further investigation is needed to establish firmly the benefit of using ciNPWT in this group of patients.

Challenges of Modeling Outcomes for Surgical Infections: A Word of Caution

Background: We developed a novel analytic tool for colorectal deep organ/space surgical site infections (C-OSI) prediction utilizing both institutional and extra-institutional American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) data. Methods: Elective colorectal resections (2006-2014) were included. The primary end point was C-OSI rate. A Bayesian-Probit regression model with multiple imputation (BPMI) via Dirichlet process handled missing data. The baseline model for comparison was a multivariable logistic regression model (generalized linear model; GLM) with indicator parameters for missing data and stepwise variable selection. Out-of-sample performance was evaluated with receiver operating characteristic (ROC) analysis of 10-fold cross-validated samples. Results: Among 2,376 resections, C-OSI rate was 4.6% (n = 108). The BPMI model identified (n = 57; 56% sensitivity) of these patients, when set at a threshold leading to 80% specificity (approximately a 20% false alarm rate). The BPMI model produced an area under the curve (AUC) = 0.78 via 10-fold cross- validation demonstrating high predictive accuracy. In contrast, the traditional GLM approach produced an AUC = 0.71 and a corresponding sensitivity of 0.47 at 80% specificity, both of which were statstically significant differences. In addition, when the model was built utilizing extra-institutional data via inclusion of all (non-Mayo Clinic) patients in ACS-NSQIP, C-OSI prediction was less accurate with AUC = 0.74 and sensitivity of 0.47 (i.e., a 19% relative performance decrease) when applied to patients at our institution. Conclusions: Although the statistical methodology associated with the BPMI model provides advantages over conventional handling of missing data, the tool should be built with data specific to the individual institution to optimize performance.

Early prediction model of organ/space surgical site infection after elective gastrointestinal or hepatopancreatobiliary cancer surgery

BACKGROUND

Organ/space SSI is a significant clinical problem. However, early detection of organ/space SSI is difficult, and previous predictive models are limited in their prognostic ability. We aimed to develop and validate a prediction model of organ/space surgical site infection (SSI) using postoperative day 3 laboratory data in patients who underwent gastrointestinal or hepatopancreatobiliary cancer resection.

METHODS

This retrospective cohort study using a single-center hospital data from April 2013 to September 2017 included all adult patients who underwent elective gastrointestinal or hepatopancreatobiliary cancer resection. The primary outcome was a presence of organ/space SSI including anastomotic leakage, pancreatic fistula, biliary fistula, or intra-abdominal abscess. We developed and validated a logistic regression model to predict organ/space SSI using laboratory data on postoperative day (POD) 3. Similar models using laboratory data on POD 1 or 5 were developed to compare the predictive ability of each model.

RESULTS

A total of 1578 patients were included. Organ/space SSI was diagnosed in 107 patients, with median diagnosis days of 6 (interquartile range, 4-9 days) after surgery. A prediction model using five commonly measured variables on POD 3 was created with the area under the curve (AUC) of 0.883 (95%CI 0.819-0.946). The AUC of a model with POD 1 laboratory data was 0.751 (95%CI 0.655-0.848), while that of POD 5 laboratory data was 0.818 (95%CI 0.730-0.906).

CONCLUSIONS

Laboratory data on POD 3 could forecast organ/space SSI precisely. Further prospective studies are warranted to investigate the clinical impact of this model.

A proposal for a comprehensive approach to infections across the surgical pathway

Despite evidence supporting the effectiveness of best practices in infection prevention and management, many healthcare workers fail to implement them and evidence-based practices tend to be underused in routine practice. Prevention and management of infections across the surgical pathway should always focus on collaboration among all healthcare workers sharing knowledge of best practices. To clarify key issues in the prevention and management of infections across the surgical pathway, a multidisciplinary task force of experts convened in Ancona, Italy, on May 31, 2019, for a national meeting. This document represents the executive summary of the final statements approved by the expert panel.

Development of a Risk Score to Predict Anastomotic Leak After Left-Sided Colectomy: Which Patients Warrant Diversion?

BACKGROUND

Anastomotic leak is a feared complication after left-sided colectomy, but its risk can potentially be reduced with the use of a diverting ostomy. However, an ostomy has its own associated negative sequelae; therefore, it is critical to appropriately identify patients to divert. This is difficult in practice since many risk factors for anastomotic leak exist and outside factors bias this decision. We aimed to develop and validate a risk score to predict an individual's risk of anastomotic leak and aid in the decision.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program Colectomy Targeted PUF was queried from 2012 to 2016 for patients undergoing elective left-sided resection for malignancy, benign neoplasm, or diverticular disease. Multivariable logistic regression identified predictors of anastomotic leak in non-diverted patients, and a risk score was developed and validated.

RESULTS

38,475 patients underwent resection with an overall anastomotic leak rate of 3%. Independent risk factors for anastomotic leak included younger age, male sex, tobacco use, and omission of combined bowel preparation. A risk score incorporating independent predictors demonstrated excellent calibration. There was strong visual correspondence between predicted and observed anastomotic leak rates. 3960 patients underwent resection with diversion, yet over half of these patients had a predicted leak rate of less than 4%.

CONCLUSION

A novel risk score can be used to stratify patients according to anastomotic leak risk after elective left-sided resection. Intraoperative calculation of scores for patients can help guide surgical decision-making in both diverting the highest risk patients and avoiding diversion in low-risk patients.

Prophylactic antibiotic bundle compliance and surgical site infections: an artificial neural network analysis

Background

Best practice "bundles" have been developed to lower the occurrence rate of surgical site infections (SSI's). We developed artificial neural network (ANN) models to predict SSI occurrence based on prophylactic antibiotic compliance.

Methods

Using the American College of Surgeons National Quality Improvement Program (ACS-NSQIP) Tampa General Hospital patient dataset for a six-month period, 780 surgical procedures were reviewed for compliance with SSI guidelines for antibiotic type and timing. SSI rates were determined for patients in the compliant and non-compliant groups. ANN training and validation models were developed to include the variables of age, sex, steroid use, bleeding disorders, transfusion, white blood cell count, hematocrit level, platelet count, wound class, ASA class, and surgical antimicrobial prophylaxis (SAP) bundle compliance.

Results

Overall compliance to recommended antibiotic type and timing was 92.0%. Antibiotic bundle compliance had a lower incidence of SSI's (3.3%) compared to the non-compliant group (8.1%, p = 0.07). ANN models predicted SSI with a 69-90% sensitivity and 50-60% specificity. The model was more sensitive when bundle compliance was not used in the model, but more specific when it was. Preoperative white blood cell (WBC) count had the most influence on the model.

Conclusions

SAP bundle compliance was associated with a lower incidence of SSI's. In an ANN model, inclusion of the SAP bundle compliance reduced sensitivity, but increased specificity of the prediction model. Preoperative WBC count had the most influence on the model.

Performance of surgical site infection risk prediction models in colorectal surgery: external validity assessment from three European national surveillance networks

OBJECTIVE

To assess the validity of multivariable models for predicting risk of surgical site infection (SSI) after colorectal surgery based on routinely collected data in national surveillance networks.

DESIGN

Retrospective analysis performed on 3 validation cohorts.

PATIENTS

Colorectal surgery patients in Switzerland, France, and England, 2007-2017.

METHODS

We determined calibration and discrimination (ie, area under the curve, AUC) of the COLA (contamination class, obesity, laparoscopy, American Society of Anesthesiologists [ASA]) multivariable risk model and the National Healthcare Safety Network (NHSN) multivariable risk model in each cohort. A new score was constructed based on multivariable analysis of the Swiss cohort following colorectal surgery, then based on colon and rectal surgery separately.

RESULTS

We included 40,813 patients who had undergone elective or emergency colorectal surgery to validate the COLA score, 45,216 patients to validate the NHSN colon and rectal surgery risk models, and 46,320 patients in the construction of a new predictive model. The COLA score's predictive ability was poor, with AUC values of 0.64 (95% confidence interval [CI], 0.63-0.65), 0.62 (95% CI, 0.58-0.67), 0.60 (95% CI, 0.58-0.61) in the Swiss, French, and English cohorts, respectively. The NHSN colon-specific model (AUC, 0.61; 95% CI, 0.61-0.62) and the rectal surgery-specific model (AUC, 0.57; 95% CI, 0.53-0.61) showed limited predictive ability. The new predictive score showed poor predictive accuracy for colorectal surgery overall (AUC, 0.65; 95% CI, 0.64-0.66), for colon surgery (AUC, 0.65; 95% CI, 0.65-0.66), and for rectal surgery (AUC, 0.63; 95% CI, 0.60-0.66).

CONCLUSION

Models based on routinely collected data in SSI surveillance networks poorly predict individual risk of SSI following colorectal surgery. Further models that include other more predictive variables could be developed and validated.

Using the National Surgical Quality Improvement Project (NSQIP) to Perform Clinical Research in Colon and Rectal Surgery

The American College of Surgeons' National Surgical Quality Improvement Project (ACS-NSQIP) is probably the most well-known surgical database in North American and worldwide. This clinical database was first proposed by Dr. Clifford Ko, a colorectal surgeon, to the ACS, and NSQIP first started collecting data ca. 2005 with the intent of comparing hospitals (benchmarking) and for hospital-level quality improvement projects. Since then, its popularity has grown from just a few participating hospitals in the United States to more than 708 participating hospitals worldwide, and collaboration allows regional or disease-specific data sharing. Importantly, from a methodological perspective, as the number of hospitals has grown so has the hospital heterogeneity and thus generalizability of the results and conclusions of the individual studies. In this article, we will first briefly present the structure of the database (aka the Participant User File) and other important methodological considerations specific to performing clinical research. We will then briefly review and summarize the approximately 60 published colectomy articles and 30 published articles on proctectomy. We will conclude with future directions relevant to colorectal clinical research.

ERAS protocol validation in a propensity-matched cohort of patients undergoing colorectal surgery

PURPOSE

Enhanced recovery after surgery (ERAS) provides many benefits. However, important knowledge gaps with respect to specific components of enhanced recovery after surgery remain because of limited validation data. The aim of the study was to validate a mature ERAS protocol at a different hospital and in a similar population of patients.

METHODS

This is a retrospective analysis of patients undergoing elective colorectal surgery from 2009 through 2016. Patients enrolled in ERAS are compared with those undergoing the standard of care. Patient demographic characteristics, length of stay, pain scores, and perioperative morbidity are described.

RESULTS

Patients (1396) were propensity matched into two equal groups (ERAS vs non-ERAS). No significant difference was observed for age, Charlson Comorbidity Index, American Society of Anesthesiologists score, body mass index, sex, operative approach, and surgery duration. Median length of stay in ERAS and non-ERAS groups was 3 and 5 days (P < .001). Mean pain scores were lower in the ERAS group, measured at discharge from the postanesthesia unit (P < .001), on postoperative day 1 (P = .002) and postoperative day 2 (P = .02) but were identical on discharge.

CONCLUSIONS

This ERAS protocol was validated in a similar patient population but at a different hospital. ERAS implementation was associated with an improved length of stay and pain scores similar to the original study. Different than most retrospective studies, propensity score matching ensured that groups were evenly matched. To our knowledge, this study is the only ERAS validation study in a propensity-matched cohort of patients undergoing elective colorectal surgery.

Effectiveness of a multidisciplinary patient care bundle for reducing surgical-site infections

BACKGROUND

Surgical-site infection (SSI) is associated with significant healthcare costs. To reduce the high rate of SSI among patients undergoing colorectal surgery at a cancer centre, a comprehensive care bundle was implemented and its efficacy tested.

METHODS

A pragmatic study involving three phases (baseline, implementation and sustainability) was conducted on patients treated consecutively between 2013 and 2016. The intervention included 13 components related to: bowel preparation; oral and intravenous antibiotic selection and administration; skin preparation, disinfection and hygiene; maintenance of normothermia during surgery; and use of clean instruments for closure. SSI risk was evaluated by means of a preoperative calculator, and effectiveness was assessed using interrupted time-series regression.

RESULTS

In a population with a mean BMI of 30 kg/m² , diabetes mellitus in 17·5 per cent, and smoking history in 49·3 per cent, SSI rates declined from 11·0 to 4·1 per cent following implementation of the intervention bundle (P = 0·001). The greatest reductions in SSI rates occurred in patients at intermediate or high risk of SSI: from 10·3 to 4·7 per cent (P = 0·006) and from 19 to 2 per cent (P < 0·001) respectively. Wound care modifications were very different in the implementation phase (43·2 versus 24·9 per cent baseline), including use of an overlying surface vacuum dressing (17·2 from 1·4 per cent baseline) or leaving wounds partially open (13·2 from 6·7 per cent baseline). As a result, the biggest difference was in wound-related rather than organ-space SSI. The median length of hospital stay decreased from 7 (i.q.r. 5-10) to 6 (5-9) days (P = 0·002). The greatest reduction in hospital stay was seen in patients at high risk of SSI: from 8 to 6 days (P < 0·001). SSI rates remained low (4·5 per cent) in the sustainability phase.

CONCLUSION

Meaningful reductions in SSI can be achieved by implementing a multidisciplinary care bundle at a hospital-wide level.

Electronically Available Comorbidities Should Be Used in Surgical Site Infection Risk Adjustment

Background

Healthcare-associated infections such as surgical site infections (SSIs) are used by the Centers for Medicare and Medicaid Services (CMS) as pay-for-performance metrics. Risk adjustment allows a fairer comparison of SSI rates across hospitals. Until 2016, Centers for Disease Control and Prevention (CDC) risk adjustment models for pay-for-performance SSI did not adjust for patient comorbidities. New 2016 CDC models only adjust for body mass index and diabetes.

Methods

We performed a multicenter retrospective cohort study of patients undergoing surgical procedures at 28 US hospitals. Demographic data and International Classification of Diseases, Ninth Revision codes were obtained on patients undergoing colectomy, hysterectomy, and knee and hip replacement procedures. Complex SSIs were identified by infection preventionists at each hospital using CDC criteria. Model performance was evaluated using measures of discrimination and calibration. Hospitals were ranked by SSI proportion and risk-adjusted standardized infection ratios (SIR) to assess the impact of comorbidity adjustment on public reporting.

Results

Of 45394 patients at 28 hospitals, 573 (1.3%) developed a complex SSI. A model containing procedure type, age, race, smoking, diabetes, liver disease, obesity, renal failure, and malnutrition showed good discrimination (C-statistic, 0.73) and calibration. When comparing hospital rankings by crude proportion to risk-adjusted ranks, 24 of 28 (86%) hospitals changed ranks, 16 (57%) changed by ≥2 ranks, and 4 (14%) changed by >10 ranks.

Conclusions

We developed a well-performing risk adjustment model for SSI using electronically available comorbidities. Comorbidity-based risk adjustment should be strongly considered by the CDC and CMS to adequately compare SSI rates across hospitals.

Open Surgical Incisions After Colorectal Surgery Improve Quality Metrics, But Do Patients Benefit?

BACKGROUND

Surgical site infection is a frequent cause of morbidity after colorectal resection and is a quality measure for hospitals and surgeons. In an effort to reduce the risk of postoperative infections, many wounds are left open at the time of surgery for secondary or delayed primary wound closure.

OBJECTIVE

The purpose of this study was to evaluate the impact of delayed wound closure on the rate of surgical infections and resource use.

DESIGN

This retrospective propensity-matched study compared colorectal surgery patients with wounds left open with a cohort of patients with primary skin closure.

SETTINGS

The American College of Surgeons National Quality Improvement Program Participant Use file for 2014 was queried.

PATIENTS

A total of 50,212 patients who underwent elective or emergent colectomy, proctectomy, and stoma creation were included.

MAIN OUTCOME MEASURES

Rates of postoperative infections and discharge to medical facilities were measured.

RESULTS

Surgical wounds were left open in 2.9% of colorectal cases (n = 1466). Patients with skin left open were broadly higher risk, as evidenced by a significantly higher median estimated probability of 30-day mortality (3.40% vs 0.45%; p < 0.0001). After propensity matching (n = 1382 per group), there were no significant differences between baseline characteristics. Within the matched cohort, there were no differences in the rates of 30-day mortality, deep or organ space infection, or sepsis (all p > 0.05). Resource use was higher for patients with incisions left open, including longer length of stay (11 vs 10 d; p = 0.006) and higher rates of discharge to a facility (34% vs 27%; p < 0.001).

LIMITATIONS

This study was limited by its retrospective design and a large data set with a bias toward academic institutions.

CONCLUSIONS

In a well-matched colorectal cohort, secondary or delayed wound closure eliminates superficial surgical infections, but there was no decrease in deep or organ space infections. In addition, attention should be given to the possibility for increased resource use associated with open surgical incisions. See Video Abstract at http://links.lww.com/DCR/A560.

Risk Factors for Postoperative Complications Following Diverting Loop Ileostomy Takedown

INTRODUCTION

Diverting loop ileostomies are frequently created to divert the fecal stream in an effort to protect downstream anastomoses. These are later reversed to restore intestinal continuity. The goal of this study is to evaluate risk factors for postoperative complications following diverting loop ileostomy takedown.

MATERIALS AND METHODS

Patients who underwent diverting loop ileostomy takedown between January 1, 2010 and April 28, 2015 were identified in the Michigan Surgical Quality Collaborative registry. Candidate covariates were identified and a hierarchical logistic regression model was used to identify risk factors for postoperative complications.

RESULTS

1,737 patients met the inclusion criteria. Rates of postoperative complications were generally low. Mean length of stay (LOS) was 5.6 (± 4.5) days. Outcomes of interest were the following: overall morbidity, serious morbidity, extended LOS, SSI, UTI, pneumonia, readmission, reoperation, and mortality. Risk factors for these outcomes included the following: ASA class, bleeding disorder, prolonged operative time, race, tobacco use, gender, steroid use, peripheral vascular disease, weight loss, and functional status.

CONCLUSIONS

Diverting loop ileostomy takedown has a complication rate of approximately 20%. Higher ASA class, longer operative times, history of bleeding disorder, and functional status were identified as risk factors for most complications.

Does bowel preparation for inflammatory bowel disease surgery matter?

AIM

The purpose of this study was to determine if bowel preparation influences outcomes in patients with inflammatory bowel disease undergoing surgery.

METHODS

The database of the American College of Surgeons National Surgical Quality Improvement Program, Procedure Targeted Colectomy, from 2012 to 2014 was analyzed. Inflammatory bowel disease patients undergoing colorectal resection with or without bowel preparation were included in the study.

RESULTS

In all, 3679 patients with inflammatory bowel disease were identified. 42.5% had no bowel preparation, 21.5% had mechanical bowel preparation only, 8.8% had oral antibiotic bowel preparation only and 27.2% had combined mechanical and oral antibiotic preparation. Combined mechanical and oral antibiotic preparation is associated with lower rates of anastomotic leak, ileus, surgical site infection, organ space infection, wound dehiscence and sepsis/septic shock.

CONCLUSION

Combined mechanical and oral antibiotic preparation for inflammatory bowel disease patients undergoing colectomy is associated with decreased rates of surgical site infection, anastomotic leak, ileus. Combined bowel preparation should be the standard of care for inflammatory bowel disease patients undergoing colorectal resection.

Outcomes are Local: Patient, Disease, and Procedure-Specific Risk Factors for Colorectal Surgical Site Infections from a Single Institution

BACKGROUND

Colorectal surgical site infections (SSIs) contribute to postoperative morbidity, mortality, and resource utilization. Risk factors associated with colorectal SSI are well-documented. However, quality improvement efforts are informed by national data, which may not identify institution-specific risk factors.

METHOD

Retrospective cohort study of colorectal surgery patients uses institutional ACS-NSQIP data from 2006 through 2014. ACS-NSQIP data were enhanced with additional variables from medical records. Multivariable logistic regression identified factors associated with SSI development.

RESULTS

Of 2376 patients, 213 (9.0%) developed at least one SSI (superficial 4.8%, deep 1.1%, organ space 3.5%). Age < 40, BMI > 30, ASA3+, steroid use, smoking, diabetes, pre-operative sepsis, higher wound class, elevated WBC or serum glutamic-oxalocetic transaminase, low hematocrit or albumin, Crohn's disease, and prolonged incision-to-closure time were associated with increased SSI rate (all P < 0.01). After adjustment, BMI > 30, steroids, diabetes, and wound contamination were associated with SSI. Patients with Crohn's had greater odds of SSI than other indications.

CONCLUSION

Institutional modeling of SSI suggests that many previously suggested risk factors established on a national level do not contribute to SSIs at our institution. Identification of institution-specific predictors of SSI, rather than relying upon conclusions derived from external data, is a critical endeavor in facilitating quality improvement and maximizing value of quality investments.

Safety of Overlapping Surgery at a High-volume Referral Center

OBJECTIVE

To compare safety profiles of overlapping and nonoverlapping surgical procedures at a large tertiary-referral center where overlapping surgery is performed.

BACKGROUND

Surgical procedures are frequently performed as overlapping, wherein one surgeon is responsible for 2 procedures occurring at the same time, but critical portions are not coincident. The safety of this practice has not been characterized.

METHODS

Primary analyses included elective, adult, inpatient surgical procedures from January 2013 to September 2015 available through University HealthSystem Consortium. Overlapping and nonoverlapping procedures were matched in an unbalanced manner (m:n) by procedure type. Confirmatory analyses from the American College of Surgeons-National Surgical Quality Improvement Program investigated elective surgical procedures from January 2011 to December 2014. We compared outcomes mortality and length of stay after adjustment for registry-predicted risk, case-mix, and surgeon using mixed models.

RESULTS

The University HealthSystem Consortium sample included 10,765 overlapping cases, of which 10,614 (98.6%) were matched to 16,111 nonoverlapping procedures. Adjusted odds ratio for inpatient mortality was greater for nonoverlapping procedures (adjusted odds ratio, OR = 2.14 vs overlapping procedures; 95% confidence interval, CI 1.23-3.73; P = 0.007) and length of stay was no different (+1% for nonoverlapping cases; 95% CI, -1% to +2%; P = 0.50). In confirmatory analyses, 93.7% (3712/3961) of overlapping procedures matched to 5,637 nonoverlapping procedures. The 30-day mortality (adjusted OR = 0.69 nonoverlapping vs overlapping procedures; 95% CI, 0.13-3.57; P = 0.65), morbidity (adjusted OR = 1.11; 95% CI, 0.92-1.35; P = 0.27) and length of stay (-4% for nonoverlapping; 95% CI, -4% to -3%; P < 0.001) were not clinically different.

CONCLUSIONS

These findings from administrative and clinical registries support the safety of overlapping surgical procedures at this center but may not extrapolate to other centers.

Surgical data science: The new knowledge domain

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care, with the goal of maximizing the quality and value of care. Whereas innovations in diagnostic and therapeutic technologies have driven past improvements in the quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytical techniques, and translation or integration of research findings into patient care. We foresee the emergence of surgical/interventional data science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data are pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care, including prevention, diagnosis, intervention, or postoperative recovery. The existing literature already provides preliminary results, suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from preoperative, intraoperative, and postoperative contexts, how it could support intraoperative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

Patients at High-Risk for Surgical Site Infection

BACKGROUND

Surgical site infections (SSIs) are a significant healthcare quality issue, resulting in increased morbidity, disability, length of stay, resource utilization, and costs. Identification of high-risk patients may improve pre-operative counseling, inform resource utilization, and allow modifications in peri-operative management to optimize outcomes.

METHODS

Review of the pertinent English-language literature.

RESULTS

High-risk surgical patients may be identified on the basis of individual risk factors or combinations of factors. In particular, statistical models and risk calculators may be useful in predicting infectious risks, both in general and for SSIs. These models differ in the number of variables; inclusion of pre-operative, intra-operative, or post-operative variables; ease of calculation; and specificity for particular procedures. Furthermore, the models differ in their accuracy in stratifying risk. Biomarkers may be a promising way to identify patients at high risk of infectious complications.

CONCLUSIONS

Although multiple strategies exist for identifying surgical patients at high risk for SSIs, no one strategy is superior for all patients. Further efforts are necessary to determine if risk stratification in combination with risk modification can reduce SSIs in these patient populations.