Best Practice for Implementation of the SCORE Portal in General Surgery Residency Training Programs

Maintaining Effective Senior Resident-Led Intern Education through Virtual Curricular Transition

OBJECTIVE

The COVID-19 pandemic has resulted in a significant shift to virtual resident education. While novel methods for virtual resident training have been described, many of these demonstrate a substantial change from previous instructional methods and their efficacy cannot be directly compared to in-person teaching. We sought to determine if the conversion of our intern "summer school" from an in-person to online format (a) impacted the knowledge acquisition of interns, and (b) their preferences for senior resident-led didactics.

DESIGN

A senior-resident led intern summer curriculum was started in an in-person format with the 2019-2020 academic year. Interns underwent assessments of their knowledge and surveys of changes in subject confidence. After the COVID-19 pandemic, the curriculum was shifted to an online format for the academic year 2020-2021.

SETTING

Washington University in St. Louis, an academic medical center located in St. Louis, Missouri PARTICIPANTS: PGY1 general surgery residents during academic year 2019-2020 (n = 13) and 2020-2021 (n = 14).

RESULTS

In both years, interns demonstrated significant increases in confidence pre- and post-summer school in all domains (p <0.01). This was no different between the in-person and the virtual administration of the bootcamp (p 0.76). In both virtual and in-person curricula, interns demonstrated increased knowledge as measured by multiple choice, boards-style question quizzes. There were no significant differences between virtual and in-person formats. In both formats, interns reported a preference for senior residents as teachers (81% v. 77%) and increased comfort in asking questions in senior resident-led vs. attending-led didactics (91% v 100%).

CONCLUSION

Virtual senior-resident led intern educational sessions are equally as effective as in-person sessions for knowledge acquisition and improving confidence in intern-specific domains. In both virtual and in-person settings, interns prefer senior resident teachers to attendings. Virtual senior resident-led education is an effective and simple method for intern instruction, regardless of the format/approach.

Trainee Utilization of the SCORE Curriculum is Associated With Improved ABSITE Performance: A Multi-institutional Study

BACKGROUND

The Surgical Council on Resident Education's (SCORE) structured educational curriculum for general surgery residents uses a 2-year repeating cycle of modules and quizzes called "This Week in SCORE" (TWIS) to organize and sequence the curricular content on the SCORE portal. The purpose of this study was to evaluate the impact of using the SCORE content and TWIS curriculum on American Board of Surgery In-Training Examination (ABSITE) performance. To date, no multi-institutional studies have examined this effect.

METHODS

Eight residency programs participated, including university and community-based programs. SCORE usage overall, TWIS usage, and ABSITE percentile scores (adjusted for post-graduate year [PGY]) were analyzed for the academic years 2017 to 2020. SCORE usage was defined as number of SCORE logins annually per resident with "low usage" ≤10 times and "high usage" >10 times. TWIS usage was defined as "low usage" (no TWIS quizzes done) or "high usage" (≥1 TWIS quiz per year).

RESULTS

Four hundred and twenty-eight trainees were evaluated. Trainees with high SCORE and TWIS usage consistently achieved a higher ABSITE percentile score. This difference was maintained in subgroup analysis by PGY with the greatest impact in PGY-1 and PGY-2 levels.

CONCLUSIONS

Utilization of multiple aspects of SCORE content appears to have a significant positive impact on ABSITE performance across all levels of postgraduate training. This multi-institutional study of a large number of users is the first to demonstrate that increased usage of SCORE content appears to be a predictor of ABSITE performance success.

Multi-institutional Collaborative Surgery Education Didactics: Virtual Adaptations During a Global Pandemic

OBJECTIVE

The COVID-19 pandemic has disrupted graduate medical education, impacting Accreditation Council for Graduate Medical Education (ACGME)-mandated didactics. We aimed to study the utility of 2 methods of virtual learning: the daily National Surgery Resident Lecture Series (NSRLS), and weekly "SCORE School" educational webinars designed around the Surgical Council on Resident Education (SCORE) curriculum.

DESIGN AND SETTING

NSRLS: The National Surgery Resident Lecture Series was a daily virtual educational session initially led by faculty at an individual surgical residency program. Thirty-eight lectures were assessed for number of live viewings (March 23, 2020-May 15, 2020).

SCORE SCHOOL

Attendance at eleven weekly SCORE educational webinars was characterized into live and asynchronous viewings (May 13, 2020-August 5, 2020). Each 1-hour live webinar was produced by SCORE on a Wednesday evening and featured nationally recognized surgeon educators using an online platform that allowed for audience interaction.

RESULTS

NSRLS: There were a mean of 71 live viewers per NSRLS session (range 19-118). Participation began to decline in the final 2 weeks as elective case volumes increased, but sessions remained well-attended.

SCORE SCHOOL

There were a range of 164-3889 live viewers per SCORE School session. Sessions have most commonly been viewed asynchronously (89.8% of viewings). Live viewership decreased as the academic year ended and then rebounded with the start of the new academic year (range 4.9%-27%). Overall, the eight webinars were viewed 11,135 times. Each webinar continues to be viewed a mean of 43 times a day (range 0-102). Overall, the eleven webinars have been viewed a total of 22,722 times.

CONCLUSIONS

Virtual didactics aimed at surgical residents are feasible, well-attended (both live and recorded), and have high levels of viewer engagement. We have observed that careful coordination of timing and topics is ideal. The ability for asynchronous viewing is particularly important for attendance. As the COVID-19 pandemic continues to disrupt healthcare systems, training programs must continue to adapt to education via virtual platforms.

"Optimization of Surgical Resident Safety and Education During the COVID-19 Pandemic - Lessons Learned"

The COVID-19 pandemic has engendered rapid and significant changes in patient care. Within the realm of surgical training, the resultant reduction in clinical exposure and case volume jeopardizes the quality of surgical training. Thus, our general surgery residency program proceeded to develop a tailored approach to training that mitigates impact on resident surgical education and optimizes clinical exposure without compromising safety. Residents were engaged directly in planning efforts to craft a response to the pandemic. Following the elimination of elective cases, the in-house resident complement was effectively decreased to reduce unnecessary exposure, with a back-up pool to address unanticipated absences and needs. Personal protective equipment availability and supply, the greatest concern to residents, has remained adequate, while being utilized according to current guidelines. Interested residents were given the opportunity to work in designated COVID ICUs on a volunteer basis. With the decrease in operative volume and clinical duties, we shifted our educational focus to an intensive didactic schedule using a teleconferencing platform and targeted areas of weakness on prior in-service exams. We also highlighted critical COVID-19 literature in a weekly journal club to better understand this novel disease and its effect on surgical practice. The long-term impact of the COVID-19 pandemic on resident education remains to be seen. Success may be achieved with commitment to constant needs assessment in the changing landscape of healthcare with the goal of producing a skilled surgical workforce for public service.

Learning to Read: Successful Program-Based Remediation Using the Surgical Council on Resident Education (SCORE) Curriculum

BACKGROUND

The Surgical Council on Resident Education (SCORE) curriculum is aligned with the American Board of Surgery (ABS) objectives. Our program adopted the SCORE curriculum in 2015 after poor ABS In-Training Examination (ABSITE) performance and lowest quartile ABS Certifying Exam (CE) and Qualifying Exam (QE) first-time pass rates. We examined the association of SCORE use with ABSITE performance and ABS board exam first-time pass rate.

STUDY DESIGN

At a single institution, a retrospective review of surgery residents' SCORE metrics and ABSITE percentile was conducted for academic years 2015 to 2019. Metrics analyzed on the SCORE web portal were mean total minutes and total visits per resident for all residents using SCORE that year. First-time pass rates of the ABS QE and CE were examined from 2013 to 2019. Chi-square and linear regression analysis were performed, and a 95% level of confidence was assumed (alpha = 0.05).

RESULTS

Yearly data from categorical general surgery residents showed a significant increase in total minutes, total visits, and ABSITE percentile. Combined first time pass rates for the ABS QE and CE significant increased from 70.8% in 2013 to 2015 to 93.9% in 2016 to 2019 (p = 0.018).

CONCLUSIONS

Increased longitudinal use of the SCORE curriculum was associated with programmatic improvements in ABSITE performance and ABS board exam first-time pass rate.

Analyzing a follow-up needs assessment: Increased use of internet-based APSA educational programs by pediatric surgeons

PURPOSE

Needs assessment is a critical component of educational program design. Follow-up is important for improvement. Two electronic educational programs, Exam-based Pediatric surgery Educational Reference Tool (ExPERT) and Pediatric Surgery Not a Textbook (NaT), offered by the American Pediatric Surgical Association (APSA) have been functional for over three years, allowing for follow-up assessment.

METHODS

A 22-question survey was distributed via email to APSA members. Questions included practice demographics, learning preferences and APSA material use. Mann-Whitney analysis was performed (p<0.05).

RESULTS

294 members responded. 43% were in academic practice with a pediatric surgery fellowship. Top preferences for obtaining/maintaining medical knowledge were national meetings (27%), ExPERT (24%), and the NaT (20%). Comparatively, in a 2014 assessment, electronic programs were less desired (16%). Cost was cited by >1/3 for not subscribing to ExPERT or NaT. Question discussions were often read regardless of response. >86% would subscribe to APSA resources if there were no CME requirement. The most frequently cited knowledge gap was fetal therapy (30%).

CONCLUSIONS

This is the first publication documenting increased acceptance of electronic educational platforms for pediatric surgeons. Well-utilized and valued, the data justify and encourage continued development of electronic educational resources. Room for improvement exists in affordability, knowledge gaps, and individualizing curriculum development.

LEVEL OF EVIDENCE

IV.

General Surgery Resident Use of Electronic Resources: 15 Minutes a Day

BACKGROUND

General surgery resident performance on the American Board of Surgery In-Service Training Exam (ABSITE) has been used to predict American Board of Surgery (ABS) passage rates, selection for remediation programs, and ranking of fellowship applicants. We sought to identify electronic resource study habits of general surgery residents associated with successful test scores.

STUDY DESIGN

A single-institution, retrospective review of general surgery resident use of 2 electronic study resources, Surgical Council on Resident Education (SCORE) and TrueLearn (TL), were evaluated for the 7 months before the 2019 ABSITE. Metrics included TL question performance, SCORE use, and a survey about other reading sources. These metrics were evaluated in 3 ABSITE percentile groupings: ≥80^th, 31^st to 79^th, and ≤30^th.

RESULTS

The ≥80^th and 31^st to 79^th percentile groups scored higher on TL questions, at 69% and 67.7%, respectively, compared with 61.4% for the ≤30^th percentile group (p < 0.03). The ≥80^th percentile group spent on average 14.6 minutes/day on SCORE compared with 5.0 minutes/day and 4.7 minutes/day for the 31^st to 79^th and ≤30^th percentile groups, respectively (p < 0.04). The ≥80^th percentile group spent 34.8 minutes/session (77 sessions) compared with 19.2 minutes/session (49 sessions) and 20.7 minutes/session (43 sessions) in the 31^st to 79^th and ≤30^th percentile groups, respectively (p = 0.009).

CONCLUSIONS

Our nomogram incorporates time spent accessing an electronic content-based resource, SCORE, and performance on an electronic question-based resource as a novel method to provide individualized feedback and predict future ABSITE performance.