Reproducibility and predictive value of three grading systems for intraoperative adverse events in a cohort of abdominal surgery

INTRODUCTION

Intraoperative adverse events (iAEs) are increasingly recognized for their impact on patient outcomes. The Kaafarani classification and Surgical Apgar Score (SAS) were developed to assess the intraoperative course; however, both have their drawbacks. ClassIntra was validated for iAEs of any origin. This study compares the Kaafarani and SAS to ClassIntra considering predictive value and interrater reliability in a cohort of abdominal surgery to support implementation of a classification in clinical practice.

METHODS

The authors made use of the LAParotomy or LAParoscopy and ADhesiolysis (LAPAD) study database of elective abdominal surgery. Detailed descriptions on iAEs were collected in real-time by a researcher. For the current research aim, all iAEs were graded according ClassIntra, Kaafarani, and SAS (score ≤4). The predictive value was assessed using univariable and multivariable linear regression and the area under the receiver operating curve (AUROC). Two teams graded ClassIntra and Kaafarani to assess the interrater reliability using Cohen's Kappa.

RESULTS

A total of 755 surgeries were included, in which 335 (44%) iAEs were graded according to ClassIntra, 228 (30%) to Kaafarani, and 130 (20%) to SAS. All classifications were significantly correlated to postoperative complications, with an AUROC of 0.67 (95% CI: 0.62-0.72), 0.64 (0.59-0.70), and 0.71 (0.56-0.76), respectively. For the secondary endpoint, the interrater reliability of ClassIntra with κ 0.87 (95% CI: 0.84-0.90) and Kaafarani 0.90 (95% CI: 0.87-0.93) was both strong.

CONCLUSION

ClassIntra, Kaafarani, and SAS can be used for reporting of iAEs in abdominal surgery with good predictive value for postoperative complications, with strong reliability. ClassIntra, compared with Kaafarani and SAS, included the most iAEs and has the most comprehensive definition suitable for uniform reporting of iAEs in clinical practice and research.

Appraisal of Intraoperative Adverse Events to Improve Postoperative Care

Background: Intraoperative adverse events (iAEs) are associated with adverse postoperative outcomes and cause a significant healthcare burden. However, a critical appraisal of iAEs is lacking. Considering the details of iAEs could benefit postoperative care. We comprehensively analyzed iAEs in a large series including all types of operations and their relation to postoperative complications. Methods: All patients enrolled in the multicenter ClassIntra® validation study (NCT03009929) were included in this analysis. The surgical and anesthesia team prospectively recorded all iAEs. Two researchers, blinded to each other’s ratings, appraised all recorded iAEs according to their origin into four categories: surgery, anesthesia, organization, or other, including subcategories such as organ injury, arrhythmia, or instrument failure. They further descriptively analyzed subcategories of all iAEs. Postoperative complications were assessed using the Comprehensive Complication Index (CCI®), a weighted sum of all postoperative complications according to the Clavien–Dindo classification. The association of iAE origins in addition to the severity grade of ClassIntra® on CCI® was assessed with a multivariable mixed-effects generalized linear regression analysis. Results: Of 2520 included patients, 778 iAEs were recorded in 610 patients. The origin was surgical in 420 (54%), anesthesia in 283 (36%), organizational in 34 (4%), and other in 41 (5%) events. Bleeding (n = 217, 28%), hypotension (n = 118, 15%), and organ injury (n = 98, 13%) were the three most frequent subcategories in surgery and anesthesia, respectively. In the multivariable mixed-effect analysis, no significant association between the origin and CCI® was observed. Conclusion: Analyzing the type and origin of an iAE offers individualized and contextualized information. This detailed descriptive information can be used for targeted surveillance of intra- and postoperative care, even though the overall predictive value for postoperative events was not improved by adding the origin in addition to the severity grade.

Classification of intraoperative adverse events in visceral surgery

BACKGROUND

Intraoperative adverse events (iAEs) are frequent in visceral surgery, but severity and related postoperative outcome are poorly investigated. A novel classification of intraoperative adverse events, ClassIntra, includes surgical and anesthesiologic intraoperative adverse events using 5 severity grades and showed a high criterion and construct validity across all surgical disciplines. ClassIntra was studied for reproducibility in a prespecified group of patients undergoing visceral surgery.

METHODS

iAEs were recorded in all patients enrolled in the ClassIntra validation study (NCT03009929). Postoperative complications were assessed daily according to the Clavien-Dindo classification. Results of the visceral group were compared with those of the non-visceral group and the full cohort. The risk-adjusted association between most severe intra and postoperative complications was investigated in a multivariable proportional odds model. Second, risk-adjusted association between ClassIntra grade and Comprehensive Complication Index, and postoperative length of stay was investigated.

RESULTS

In total, 1,270 out of 2,520 patients (50%) underwent visceral surgery. Compared with the nonvisceral group and full cohort, more intraoperative (337/1270 [27%] vs 273/1250 [22%] vs 610/2520 [24%] patients) and postoperative complications (457/1270 [36%] vs 381/1250 [30%] vs 838/2520 [33%] patients) occurred. The risk for a more severe postoperative complication increased with each ClassIntra grade (odds ratio [95% confidence interval] I vs 0 1.10 [0.73 to 1.66], II vs 0 1.69 [1.10 to 2.60], III vs 0 2.31 [1.21 to 4.41], IV vs 0 2.35 [0.69 to 8.06]). Accordingly, CCI and postoperative length of stay increased with each ClassIntra grade in the visceral group, comparable with the nonvisceral and full cohort.

CONCLUSION

Consistent results for the association of intraoperative adverse events and patient outcomes render ClassIntra a valuable instrument in visceral surgery.

Prospective validation of classification of intraoperative adverse events (ClassIntra): international, multicentre cohort study

OBJECTIVE

To prospectively assess the construct and criterion validity of ClassIntra version 1.0, a newly developed classification for assessing intraoperative adverse events.

DESIGN

International, multicentre cohort study.

SETTING

18 secondary and tertiary centres from 12 countries in Europe, Oceania, and North America.

PARTICIPANTS

The cohort study included a representative sample of 2520 patients in hospital having any type of surgery, followed up until discharge. A follow-up to assess mortality at 30 days was performed in 2372 patients (94%). A survey was sent to a representative sample of 163 surgeons and anaesthetists from participating centres.

MAIN OUTCOME MEASURES

Intraoperative complications were assessed according to ClassIntra. Postoperative complications were assessed daily until discharge from hospital with the Clavien-Dindo classification. The primary endpoint was construct validity by investigating the risk adjusted association between the most severe intraoperative and postoperative complications, measured in a multivariable hierarchical proportional odds model. For criterion validity, inter-rater reliability was evaluated in a survey of 10 fictitious case scenarios describing intraoperative complications.

RESULTS

Of 2520 patients enrolled, 610 (24%) experienced at least one intraoperative adverse event and 838 (33%) at least one postoperative complication. Multivariable analysis showed a gradual increase in risk for a more severe postoperative complication with increasing grade of ClassIntra: ClassIntra grade I versus grade 0, odds ratio 0.99 (95% confidence interval 0.69 to 1.42); grade II versus grade 0, 1.39 (0.97 to 2.00); grade III versus grade 0, 2.62 (1.31 to 5.26); and grade IV versus grade 0, 3.81 (1.19 to 12.2). ClassIntra showed high criterion validity with an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91) in the survey (response rate 83%).

CONCLUSIONS

ClassIntra is the first prospectively validated classification for assessing intraoperative adverse events in a standardised way, linking them to postoperative complications with the well established Clavien-Dindo classification. ClassIntra can be incorporated into routine practice in perioperative surgical safety checklists, or used as a monitoring and outcome reporting tool for different surgical disciplines. Future studies should investigate whether the tool is useful to stratify patients to the appropriate postoperative care, to enhance the quality of surgical interventions, and to improve long term outcomes of surgical patients.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03009929.