Conducting high-quality research in cardiothoracic surgical education: Recommendations from the Thoracic Education Cooperative Group

Reducing research wastage by starting off on the right foot: optimally framing the research question

PURPOSE

Strongly framed research questions are clear as to the population (P), the exposures or interventions (E/I), comparison groups (C), outcomes (O), time when relevant (T), and what the investigator wants to know. A solid framework sets up the measurement model, analysis, and anticipated results. The purpose of this study was to estimate the extent to which research questions in journals that focused on patient-reported outcome measures (PROM) and quality of life (QOL) are clear.

METHODS

All 440 research articles published in four PROM journals in 2020. excluding reviews, psychometric, and qualitative papers, were reviewed. Research questions were classified as: (i) adequately framed (ii) poorly framed; or (iii) unframed based on clarity criteria. Examples from each journal were presented and reframed to match results in the article.

RESULTS

Of 440 articles, 195 (44.3%) were classified as adequately framed; 230 (52.2%) as poorly framed; and 15 (3.4%) as unframed. There was heterogeneity across journals (Chi-square: 20.8; 6 df; p = 0.002). Only 29% were framed according to what the investigators wanted to know; 72% were framed like a "to do" list; and 6% were framed as a research agenda.

CONCLUSION

Almost half of the questions were poorly framed or unframed a practice that could contribute to research wastage. Even "adequately framed" questions rarely stated what they wanted to know a priori, increasing the risk of biased reporting. Researchers, reviewers, and editors should encourage the use established frameworks for research questions.

Multi-Institutional Evaluation of a Debate-Style Journal Club for Cardiothoracic Surgery Trainees

BACKGROUND

Traditional journal clubs address individual articles and are limited in terms of breadth and depth of content covered. The present study describes the outcomes of a novel debate-style journal club in a multi-institutional setting.

METHODS

Participating institutions were recruited through the Thoracic Education Cooperative Group (TECoG). The distributed curriculum included instructions, debate scenarios, suggested article lists, moderator slides, debate scoresheets, exams, and feedback surveys.

RESULTS

Six institutions participated in the study (2015-2019), consisting of a total of 10 years' worth of cumulative debates. Cardiothoracic surgery trainees participated in 10 monthly debates over each academic year. Trainee performance on the written examination in the realm of evidence-based medicine and critical appraisal improved over the course of the academic year (beginning 55.2% vs end 76.3%, p=0.040). Importantly, written examination after debates revealed a significant improvement in scores on questions relating to topics that were debated as compared to those that were not (+27.1% vs +2.5%, p=0.006), emphasizing the importance of the debates as compared to other sources of knowledge gain. Surveys completed by trainees and faculty overall favored the debate-style journal club as compared to the traditional journal club in gaining familiarity with seminal literature in the field, improving upon oral presentation skills, and applying published literature to questions encountered clinically.

CONCLUSIONS

In this multi-institutional prospective study, we demonstrate that the novel debate-style cardiothoracic surgery journal club is an effective educational intervention for cardiothoracic surgical trainees to acquire, retain, and gain practice in applying literature-based evidence to case-based scenarios.

The Unfulfilled Need for Technical Skill Assessments Among Academic Cardiothoracic Surgeons

Technical skill is a proven predictor of surgical outcomes, yet no platform exists for continual technical skill development following training. We aim to characterize the perceived need for feedback on technical skill among practicing thoracic surgeons. Under the Thoracic Education Cooperative Group, a panel of cardiothoracic surgeons and trainees developed and distributed an online survey for cardiothoracic surgery faculty in the Thoracic Surgery Directors Association database. The survey solicited demographics, perceived need for constructive feedback, barriers to sharing critiques, and preferences of desired peer reviewers. One hundred forty surgeons responded to our survey (response rate: 19.6% [140/713]). Most respondents had practiced for greater than 15 years (49.3%, 69/140). About 76.4% (107/140) of responders agreed or strongly agreed receiving feedback on their technical skills would help them improve, and 71.5% (100/140) desired individualized skills feedback. While 61.4% (86/140) of surgeons received meaningful technical skill feedback as attending surgeons, this was infrequent, with 63.3% (88/139) last receiving feedback over 12 months prior. Commonly cited barriers to sharing feedback included lack of common practice, time constraints, and hierarchical barriers. About 66.2% (92/139) of participants would spend at least 10 minutes providing peer feedback to receive feedback on their own skills, while 45.3% (63/139) would spend greater than 20 minutes. Attending thoracic surgeons identify an unmet desire for ongoing, constructive feedback on their technical skills following training. Surgeons feel critique fosters improvement and would devote significant time to engaging in peer feedback. A platform for exchange of technical skill feedback is warranted.

Are Esophagectomy Board Requirements Achievable? A Multi-Institutional Analysis

Duty-hour restrictions have implications on trainee operative exposure necessary to meet minimum case-volume requirements. We utilized a previously validated simulation model to evaluate the effect of program volume, trainee numbers and complement, and rotation schedule on the probability of achieving adequate esophagectomy case numbers for cardiothoracic surgery trainees. A ProModel simulator centered on probabilistic distributions of operative cases was utilized. Historical data from five 2-year cardiothoracic surgery training programs were obtained from 2016-2018 and used as inputs to the simulator that generated 10,000 "trainee 2-year periods" per program. Programs varied in annual average esophagectomy volume (12-91 per year), with 2-4 trainees graduating over a 2-year training period. If esophagectomy cases were distributed solely based on scheduling and institutional volume, only 60% of evaluated programs could adequately expose all trainees in esophagectomy to meet case requirements. The 3 programs with adequate esophagectomy volumes had averaged 3.3 times (range 3.0-3.6) the minimum number of board-required cases for their programs' trainees. The ability of programs to provide trainees with adequate esophagectomy volume is challenging based on institutional volume and scheduling. Through simulation, we demonstrate that programs need >2 times the expected minimum number of esophagectomies to ensure that >90% of trainees meet case-volume requirements. Programs may consider strategies such as allowing trainees to select cases based on personal need, train fewer fellows, or enable trainees to seek subspecialty exposure externally to achieve minimum esophagectomy case-load requirements.

Multi-institutional collaborative mock oral (mICMO) examination for cardiothoracic surgery trainees: Results from the pilot experience

Objective

The American Board of Thoracic Surgery–certifying examination is challenging for applicants. Single institutions have reported good results with a mock oral examination (MOE) for trainees. General surgery literature has demonstrated success with in-person multi-institutional MOE examinations. Due to small numbers of cardiothoracic training programs and significant geographic variability, we hypothesized that a multi-institutional, collaborative remotely administered MOE (mICMO) pairing faculty with trainees from different institutions would provide an important educational experience.

Methods

mICMOs were conducted via the Zoom virtual platform across 6 institutions. Descriptive results via post-experience survey were analyzed and reported.

Results

In total, 100% of trainees found mICMO useful. The majority would recommend to a peer, and 100% of faculty examiners found mICMO useful and would participate in another examination.

Conclusions

Faculty and trainees found the experience to be effective with respect to creating a high-stakes environment, educationally beneficial, and productive. These results support the continued use of mICMO and encourage expansion and collaboration with additional institutions across the country.

Surgical education research: How to move beyond the survey

BACKGROUND

Surgical education serves as the foundation for high-quality, efficient patient care. Because clinical knowledge is gained and newer surgical techniques developed, this material is incorporated into the curricula of surgical trainees. Methods for studying this integration vary widely, however, providing data that, at times, is challenging to interpret in the context of patient care and outcomes.

METHODS

A review of the literature was conducted to evaluate current practices in surgical education research, as well as topics areas of focus.

RESULTS

Several techniques in surgical education research currently exist, including surveys, knowledge assessments, tests of skill, and single-arm educational interventions, which may or may not include a prepost design with assessments administered both before and immediately following the intervention. The applicability of these measures to patient outcomes is variable.

CONCLUSION

Research in surgical education represents a field of great interest, with opportunity for novel investigations among a broad collection of topic areas. Educational research should be approached in a methodologically rigorous fashion with high investigational standards in order to advance the education of surgical trainees and the care of surgical patients.