Competency-Based Assessment Tool for Pediatric Esophagoscopy: International Modified Delphi Consensus

Developing an Objective Structured Assessment of Technical Skills (OSATS) for Microlaryngoscopy

BACKGROUND

Microlaryngoscopy is a basic technical skill in Oto-HNS. It is essential for residency programs to have a competency-based assessment tool to evaluate residents' performance of this procedure. An Objective Structured Assessment of Technical Skills (OSATS) is a procedure-specific assessment, which consists of the following: (a) Operation-Specific Checklist and (b) Global Rating Scale (GRS).

OBJECTIVE

The objective of this study was to create an OSATS for adult microlaryngoscopy.

METHODS

This was a prospective study, with an initial qualitative phase for OSATS development (Phase I), and a clinical pilot phase (Phase II). In Phase I, interviews were conducted with three laryngologists to establish a stepwise description of adult microlaryngoscopy and review a previously validated GRS for relevance to microlaryngoscopy. Responses were used to create a framework for the OSATS. The OSATS was then presented to Oto-HNS residents and laryngologists in an alternating fashion, for review of clarity and relevance. A pilot study was then performed to evaluate the resident performance of adult microlaryngoscopy. Multiple regression analysis was carried out to investigate whether training level, case complexity, and previous OSATS exposure could predict participant scores.

RESULTS

Phase I of this study led to the creation of a 34-item OSATS. The pilot study (N = 28 procedures) revealed that training level was significantly correlated with increased OSATS scores. There was no statistically significant correlation between case complexity and resident scores. Assessors reported the perceived utility of the OSATS and intent for use in residency training.

CONCLUSION

Application of the proposed OSATS will allow for competency-based assessment of the resident performance of microlaryngoscopy.

LEVEL OF EVIDENCE

NA Laryngoscope, 133:2719-2724, 2023.

Documentation in Pediatric Microlaryngoscopy/Bronchoscopy: International Modified Delphi Consensus

BACKGROUND

Complete and accurate documentation of surgical procedures is essential for optimizing patient care, yet significant variation in operative notes persists within and across institutions. We sought to reach consensus on the most important components of an operative note for pediatric microlaryngoscopy and bronchoscopy.

METHODS

A modified Delphi consensus process was used. A checklist for operative documentation, created by fellowship-trained pediatric otolaryngologists-head and neck surgeons, was sent to surgeons identified as experts in pediatric laryngoscopy and bronchoscopy. In the first round, items were rated as "keep" or "remove". In the second round, each item was rated on a 7-point Likert scale for importance. The mean score of each item was calculated to determine if consensus was reached.

RESULTS

Overall, 43/74 (58.1%) surgeons responded to our survey. After two rounds of editing, 28 components reached consensus, 24 were near consensus, and 26 did not reach consensus. Items that reached final consensus had mean (SD) ratings of 6.12 (0.94) (range, 5.31-6.72).

CONCLUSION

Pediatric otolaryngologists identified as bronchoscopy experts were able to create a checklist of essential components of an operative note for pediatric laryngoscopy and bronchoscopy using a Delphi method. Items reaching consensus included procedure name, description of breathing, grade of airway view, description of normal anatomic structures, grade of subglottic stenosis if present, presence and description of tracheobronchomalacia, presence of fistulae, cleft and rings, and several special cases including foreign body and tracheostomy management, as well as end of procedure disposition and complications.

LEVEL OF EVIDENCE

5 Laryngoscope, 2022.