Validation of an Objective Assessment Tool for Tonsillectomy in Otolaryngology Resident Training

Incorporating Intraoperative Assessments of surgical Trainees Performing Open Surgery Into Summative Trainee Evaluations: A Scoping Review

OBJECTIVE

Real-world assessment of trainee surgical competence and autonomy must occur in the operative environment. It is unclear if or how workplace-based operative skills assessments are systematically incorporated into summative evaluations. In this study, intraoperative surgical skills assessment tools were reviewed for actual or proposed incorporation into summative evaluations.

DESIGN

A scoping review was conducted. The Cochrane Central Registrar of Clinical Trials, Embase, Medline, and Web of Science were searched for articles published from January 2012 to December 2021. Articles describing an educational assessment paradigm of a post-graduate surgical trainee performing an open surgical procedure were selected. Using the PRISMA Extension Guidelines for Scoping Reviews, two independent reviewers completed data extraction including literature quality, assessment paradigm, assessment quality, incorporation into summative evaluations, and facilitators and barriers to implementation. Finally, a narrative synthesis of extracted data was completed.

RESULTS

Twenty-two articles met inclusion criteria describing 11 different surgical assessments. All were cohort studies, of which 21 involved the evaluation of an assessment tool and one involved participant perceptions of the assessment tool. All studies discussed the possible use of their assessment to track progress over time, but no study explicitly outlined the incorporation of the tool into a summative review. Four types of assessments automatically integrated results, which reportedly were utilized at Clinical Competency Committee (CCC) meetings. Facilitators and barriers of surgical assessment integration into summative assessments was not described.

CONCLUSIONS

While proponents of formative intra-operative assessment have claimed that their potential use in supporting summative evaluation and tracking learning trajectories are major benefits, a systematic method of integrating workplace surgical assessments into summative resident evaluation has not been described or uniformly implemented.

Training and competence perception differences in otolaryngology and head and neck surgery training program - an anonymous electronic national survey

BACKGROUND

Otorhinolaryngology / Head and Neck Surgery consists of different sub-specialties, each comprising unique characteristics and challenges. Herein, we investigate the use of a uniform national electronic questionnaire for curriculum planning.

MAIN OUTCOME MEASURES

(1) Analyze the residents' perception of the different sub-specialties training programs and their competence capabilities. (2) Identify sub-specialties requiring attention. (3) Investigate the characteristics associated with competence perception.

METHODS

This is a national cross sectional study. An anonymous electronic questionnaire was emailed to all registered Otorhinolaryngology / Head and Neck Surgery residents.

RESULTS

63.5% registered residents responded to the questionnaire. Two sub-specialties, Rhinology and Laryngology, are located in the extremities of the residents' perceptions of competence and training (p < 0.0001), despite similar complexity perception (means 6.10 and 6.01, respectively). Rhinology is perceived as the most well-trained sub-specialty, both surgically and clinically (means 7.08 and 7.66, respectively), whereas Laryngology is bottom scaled (means 5.16 and 6.14, respectively). The same is true for perceived competence, surgical and clinical, in Rhinology (means 6.80 and 8.02, respectively) compared to Laryngology (means 5.04 and 6.75, respectively). Significant positive correlations were found between training, competence perception and workload ("golden training triangle").

CONCLUSIONS

Each ORL-HNS sub-specialty comprises different characteristics and a different learning curve, necessitating a tailored training program. Recognizing its sub-specialties distinctive features may assist in establishment of better-adapted learning curves in residency programs. Herein, we examine the use of anonymous electronic national survey. Laryngology, bottom ranked, is a prototype of a relatively new surgical discipline. Rhinology, ranked top by the residents, is an exemplar of a sub-specialty with an optimal 'educational environment'. Moreover, we have established golden training triangle, implicating, highlights the essential role of institutional and senior staff for proper residency teaching. We demonstrate and advocate the benefit of using an anonymous electronic questionnaire.

Gathering validity evidence for a 3D-printed simulator for training of myringotomy and ventilation tube insertion

Objectives

This study aimed to gather validity evidence according to Messick's framework for a novel 3D-printed simulator for myringotomy with ventilation tube insertion for use in technical skills training of otorhinolaryngology (ORL) residents.

Methods

The study included 15 junior ORL residents (trainees) and 13 experienced teaching otolaryngologists (experts). Experts and trainees first received an identically structured introduction to the procedure, simulator, and simulation setup. Five procedures performed by each participant were video-recorded and ordered randomly for blinded rating by two independent raters. The rating tools used were a global rating scale (GBRS) and a task-specific checklist. Validity evidence was collected according to Messick's framework. Differences in time consumption and performance scores were analyzed. Finally, a pass/fail standard was established using the contrasting groups' method.

Results

Trainees used significantly more time per procedure (109 s, 95% CI: 99-120) than experts (82 s, 95% CI: 71-93; p < .001). Adjusted for repetition and rater leniency, experts achieved an average GBRS score of 18.8 (95% CI: 18.3-19.2) out of 20 points, whereas trainees achieved an average of 17.1 points (95% CI: 16.6-17.5; p < .001). In contrast to the task-specific checklist, the GBRS score discriminated between repetition number and participant experience. The pass/fail standard for the GBRS was established at 18.4 points.

Conclusion

We established educational validity evidence for a novel 3D-printed model for simulation-based training of ventilation tube insertion and established a reliable pass/fail standard.

Level of Evidence

1b.

Does length of time between cases affect resident operative time for tonsillectomy and adenoidectomy?

OBJECTIVE

To evaluate the effect of prolonged time intervals between tonsillectomy and adenoidectomy (TA) on resident operative time and complications.

STUDY DESIGN

Retrospective cohort.

SETTING

Tertiary academic hospital.

METHODS

This retrospective study covers a five-year period from 2015 to 2020. Time intervals between isolated pediatric TA cases performed by eight otolaryngology residents were reviewed to assess effect on operative time (defined as prolonged if ≥ 30 min and non-prolonged if < 30 min). Intervals including a procedure involving either a tonsillectomy or adenoidectomy that was a non-isolated TA were excluded.

RESULTS

A total of 309 isolated TAs were identified with 67.3% of procedures performed under 30 min. The mean surgical time interval between procedures was 5.83 ± 10.02 days (range 0.02-69.82). Most TAs were performed on patients aged 7 years or younger. Surgical time interval between TA was not a significant factor in determining prolonged operative time on univariable logistic regression, OR 1.01 (CI: 0.98 to 1.03) (p = 0.63). Patient age at surgery, adenoid grade, tonsil size and total number of TAs performed to date were significant factors in determining prolonged operative time in both univariable and multivariable logistic regression models. Prolonged operative time did not have a significant effect on readmission, reoperation, or post-operative bleeding.

CONCLUSION

Extended time interval (up to 3 months) between routine TA does not affect operative time. Expansion of our methodology to more complex cases would be beneficial in designing resident training curriculum.