Program Director Opinions of Virtual Interviews: Whatever Makes my Partners Happy

Intern Experience Reflects Information Gathered During the Virtual Interview Process

OBJECTIVE

The onset of the coronavirus 2019 (COVID-19) pandemic brought many changes to the residency application process including transitioning to a virtual interview platform, which continues today. The transition brought many concerns from general surgery applicants about their ability to obtain adequate information about a program virtually. We sought to characterize how information presented by programs during the first ever virtual interview cycle matched the experience of general surgery interns after training at a program for 1 year.

DESIGN, SETTING, AND PARTICIPANTS

In May of 2022, a survey was distributed to 243 program directors who were asked to forward it to their general surgery categorical interns who matched during the 2021 virtual match cycle. Demographics, resources used to determine an impression of a program, and correlations between information presented virtually and what was subsequently experienced as an intern were collected.

RESULTS

Forty-six program directors confirmed forwarding the survey to their categorical interns. A total of 102 general surgery interns completed the survey. Most interns (88.2%) agreed that their experience matched expectations based on information received through the virtual interview process and 98% of interns were satisfied with their experience at their training program. Interviews with faculty (40.0%), residents (68.0%) and the program web site (29.0%) were the top 3 resources used to create the most accurate impression of a program. Interns felt they were well informed during the virtual interview experience about support from fellow residents (84.3%), culture (73.0%), surgical volume (72.5%), and intern operative experience (71.6%). In addition, 65.7% of participants thought they were able to obtain a good understanding of the program's culture from the virtual process. However, 16.7% thought that their program unintentionally misrepresented aspects of the training program.

CONCLUSIONS

The faculty and residency interviews were the most important factors in program ranking and most participants agreed that their virtual interview experience matched their expectations during their intern year. Most interns felt they were able to obtain a good understanding of the program's culture from the virtual process. In addition, a majority of interns felt well informed during the interview on aspects ranging from surgical volume, autonomy, and work hours to support from faculty and residents. If virtual interviews are to continue, residents can be satisfied that information gathered virtually will match the reality of their training. Programs should continue to make every effort to present their program realistically.

Reimagining the Rheumatology Fellowship Interview: Using Participatory Design-Thinking Process to Create a More Applicant-Centered Experience

OBJECTIVE

Design thinking is a creative problem-solving process used to better understand users' needs and experiences so that a product or service can be improved. Its emphasis on empathy, iterative prototyping, and participatory collaboration make it an ideal methodology for innovation in medical education. We apply this framework to the virtual rheumatology fellowship interview process so that interviews can become more applicant centered.

METHODS

This educational quality improvement project uses a design-thinking framework to identify opportunities and challenges for rheumatology fellowship applicants. The investigators use the 5-step process (Empathize, Define, Ideate, Prototype, Test) and incorporate rapid qualitative analysis of semistructured interviews to innovate the interview experience. The iterative and collaborative nature of this process has empowered participants to codesign an applicant-centered interview experience.

RESULTS

Interviews with fellowship applicants (n = 9), fellow physicians (n = 4), and faculty members (n = 3) identified three major dynamics of the interview process: (1) Is it a safe environment to ask questions? (2) How do I exchange information effectively? and (3) How do I fit all these data into the bigger picture? Creative brainstorming techniques at a series of three workshops yielded four prototypes emphasizing customization, hybridization, facilitation, and preparation. A finalized applicant-centered interview template was devised in preparation for the 2023-2024 application season.

CONCLUSION

Design thinking has yielded insights into three important dynamics that drive applicant experiences. These insights allow for a redesign of processes so that virtual interviews can be more applicant centered. This framework allows for further iterations and modifications as the needs of applicants and programs evolve over time.

General Surgery Residency Virtual Recruitment During the Pandemic: An Analysis of Applicant Surveys

INTRODUCTION

The COVID-19 pandemic forced a sudden change from in-person to virtual interviews for the general surgery residency match. General surgery programs and applicants adopted multiple strategies to best mimic in-person recruitment. The purpose of this study was to evaluate applicant opinions of the virtual recruitment format.

MATERIALS AND METHODS

Postinterview survey responses for applicants interviewing at a single general surgery residency program in the 2020-2021 and 2021-2022 cycles were evaluated. All interviewed applicants were sent an anonymous survey assessing the virtual interview structure, their impression of the program, and their opinions on recruitment in the future.

RESULTS

The response rate was 31.2% (n = 60). Most (88.4%) respondents reported a more favorable view of the program after a virtual interview. Factors that were most likely to create a favorable impression were residents (89.6%) and culture (81.0%). 50.8% of applicants favored virtual-only interviews. The majority of applicants (60.3%), however, preferred the virtual interview remain a component of the application process, 34.4% recommended that virtual interviews be used as an initial screen before in-person invites, while 19.0% suggested applicants should interview in-person or virtually without penalty. 62.1% favored capping the number of interviews offered by programs and accepted by applicants.

CONCLUSIONS

The virtual interview format for general surgery residency allows applicants to effectively evaluate a residency program. Applicants are in favor of a combination of virtual and in-person interviews in the future. Innovation in the recruitment process, including limiting the number of applications and incorporating virtual events, is supported by applicants.

Virtual Interviews: Assessing How Expectations Meet Reality

OBJECTIVE

This study aimed to determine the post-matriculation perceptions of interns and faculty who participated in the 2020-2021 virtual interview process and how their expectations of the program and the applicants, respectively, aligned with reality.

DESIGN, SETTING, AND PARTICIPANTS

Published surveys on virtual interviewing were reviewed and modified to design two surveys, for interns and for faculty who interviewed. Interns and faculty members from the Departments of Surgery and Medicine at one institution who participated in the 2020-2021 virtual interview process completed the surveys four to six months after the start of the academic year. Following survey completion, surgical interns from the same application cycle participated in one in-person focus group nine months after the start of the academic year to clarify points raised in the surveys.

RESULTS

Forty-six interns and faculty members responded to the survey (subgroup response rates ranging from 13-30%) and ten interns participated in the focus group (participation rate 100%). Most faculty and intern participants found that expectations formed during virtual interviews were accurate. However, our respondents noted limitations to virtual interviews that reduced their usefulness, including challenges with unnatural social interactions, understanding city fit, and getting a sense of resident and program culture. Participants provided possible solutions to address these challenges.

CONCLUSIONS

Overall, this mixed-methods study at a single institution found that perceptions from virtual interviews were generally accurate but with some limitations. We describe several opportunities to improve the virtual interview process and optimize the application experience.

Expectations versus reality: Understanding the accuracy of impressions made during virtual interviews of pediatric pulmonary fellowship programs

RATIONALE

As a result of the SARS-CoV-2 pandemic, all pediatric pulmonary fellowship programs conducted virtual interviews for the first time in the Fall of 2020. This study aimed to understand the accuracy of virtual-interview derived-impressions of fellowship programs, as well as applicant preference for future fellowship interview cycles.

METHODS

A group of pediatric pulmonary fellows and Program Directors designed a REDCap survey. The survey was distributed to all first-year pediatric pulmonary fellows who participated in the 2020-2021 virtual interview season.

RESULTS

23/52 (44%) of first-year pediatric pulmonary fellows completed the survey. 96% were able to form general impressions about fellowship programs during their virtual interviews. 96% reported that generally their fellowship experience matched their virtual-interview derived-impressions. 17 of the 19 factors applicants use to rank programs had no statistically significant change (p > 0.05) in impression from virtual interview to fellowship experience. The two factors with a statistically significant (p < 0.05) change in impression were patient care related-volume of "bread and butter" pediatric pulmonary patients and volume of tertiary care pediatric pulmonary patients. 87% prefer some form of in-person interview option in future application cycles. A tiered interview format in which applicants are first invited to a virtual interview day followed by an optional in-person second look day was the most popular preference for future interview cycles (48%).

CONCLUSIONS

Virtual interviews may provide accurate representations of pediatric pulmonary fellowship programs and applicants prefer some type of in-person interview option in future application cycles.

Virtual interviewing for graduate medical education recruitment and selection: A BEME systematic review: BEME Guide No. 80

BACKGROUND

The COVID-19 pandemic caused graduate medical education (GME) programs to pivot to virtual interviews (VIs) for recruitment and selection. This systematic review synthesizes the rapidly expanding evidence base on VIs, providing insights into preferred formats, strengths, and weaknesses.

METHODS

PubMed/MEDLINE, Scopus, ERIC, PsycINFO, MedEdPublish, and Google Scholar were searched from 1 January 2012 to 21 February 2022. Two authors independently screened titles, abstracts, full texts, performed data extraction, and assessed risk of bias using the Medical Education Research Quality Instrument. Findings were reported according to Best Evidence in Medical Education guidance.

RESULTS

One hundred ten studies were included. The majority (97%) were from North America. Fourteen were conducted before COVID-19 and 96 during the pandemic. Studies involved both medical students applying to residencies (61%) and residents applying to fellowships (39%). Surgical specialties were more represented than other specialties. Applicants preferred VI days that lasted 4-6 h, with three to five individual interviews (15-20 min each), with virtual tours and opportunities to connect with current faculty and trainees. Satisfaction with VIs was high, though both applicants and programs found VIs inferior to in-person interviews for assessing 'fit.' Confidence in ranking applicants and programs was decreased. Stakeholders universally noted significant cost and time savings with VIs, as well as equity gains and reduced carbon footprint due to eliminating travel.

CONCLUSIONS

The use of VIs for GME recruitment and selection has accelerated rapidly. The findings of this review offer early insights that can guide future practice, policy, and research.

LEGO®-based communication assessment in virtual general surgery residency interviews

Purpose

Effective communication skills are a critical quality and skill that is highly sought after for surgeons which largely impacts patient outcomes. Residency programs design their interview processes to select the best candidates. LEGO®-based activities have been frequently used to enhance communication skills and team building. This study investigates the effectiveness and reliability of a novel LEGO®-based communication assessment in interviews for surgical residencies and the feasibility of implementing it in a virtual setting.

Methods

This study conducted a retrospective analysis of a LEGO®-based communication assessment at the program's 2020/2021 residency interviews. Each applicant was assessed on a different model. The total scores were analyzed for consistency among raters and correlated to faculty interviews. Furthermore, the impact of the assessment structure, scoring criteria, and range of models' difficulties on the total scores were explored.

Results

A total of 54 categorical and 55 preliminary applicants interviewed on 2 days. The assessment on different models and had no impact on applicants' total scores for either categorical and preliminary groups (p = 0.791 and 0.709, respectively). The communication components of the assessment showed high consistency between the raters. The two applicant groups displayed a statistically significant difference (p = 0.004) in the communication evaluation and model accuracy components. Total scores did not correlate with the faculty interviews of standardized questions in either group.

Conclusion

This novel LEGO®-based communication assessment showed high reliability and promising results as a tool to assess communication and problem solving for residency interviews that can be readily implemented in a virtual setting.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44186-022-00021-4.

Experiences, perspectives, and advice for using virtual interviews in post-graduate trainee selection: a national survey of CCFP (EM) program directors

Objective

Due to the COVID pandemic, restrictions were put in place mandating that all residency interviews be transitioned to a virtual format. Canadian CCFP(EM) programs were among the first to embark on this universal virtual interview process for resident selection. Although there have been several recent publications suggesting best practice guidelines for virtual interviews in trainee selection, pragmatic experiences and opinions from Program Directors (PDs) are lacking. This study aimed to elicit the experiences and perspectives of CCFP(EM) PDs after being amongst the first to conduct universal virtual interviews in Canada.

Methods

A 17-item online survey was created and distributed to all CCFP(EM) PDs (n = 17). It explored the virtual interview format employed, perceived advantages and disadvantages of a virtual configuration, confidence in determining a candidate’s rank order, and PD preference for employing a virtual interview format in the future. It also elicited practical advice to conduct a smooth and successful virtual interview day.

Results

The survey response rate was 76.5% (13/17). Nine respondents (69.2%) agreed that the virtual interview format enabled them to confidently determine a candidate’s rank order. With respect to preference for future use of virtual interviews, 23.1% agreed, 38.5% disagreed and 38.5% neither agreed nor disagreed. Inductive thematic analysis of free text responses revealed themes related to virtual interview advantages (time, financial, and resource costs), disadvantages (difficulty promoting smaller programs, getting a ‘feel’ for candidates and assessing their interpersonal skills), and practical tips to facilitate virtual interview processes.

Conclusion

Once restrictions are lifted, cost-saving advantages must be weighed against suggested disadvantages such as showcasing program strengths and assessing interpersonal skills in choosing between traditional and virtual formats. Should virtual interviews become a routine part of resident selection, the advice suggested in this study may be considered to help optimize a successful virtual interview process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43678-022-00312-1.

Otolaryngology residency programs' perspectives on virtual interviews during the COVID ‐19 pandemic

Objective

To evaluate perceptions of otolaryngology residency program directors (PDs) and department chairs (DCs) on virtual interviews (VIs) at the conclusion of the 2021 residency match.

Study design

Cross‐sectional survey study.

Methods

An anonymous and voluntary survey was administered to PDs and DCs of U.S. otolaryngology residency programs from March 1, 2021 to April 11, 2021. Utilizing Likert scale ratings, the survey measured overall satisfaction with the VI format, perceived ability to assess key candidate attributes, relative importance of traditional candidate metrics, and likelihood to implement VIs in future cycles.

Results

Ninety‐four surveys were completed by PDs (48.9%) and DCs (51.1%). Sixty (63.8%) respondents found the VI experience to be satisfactory or strongly satisfactory. Sixty‐six (70.2%) respondents felt confident in their ability to assess a candidate's professionalism and communication skills through VIs; however, confidence in gauging an applicant's clinical skills/knowledge (41.2%) and overall program fit (47.3%) was lower. Regarding candidate metrics, 64 (68.1%) respondents believed that personal prior knowledge of the applicant gained increased importance with VIs. Forty‐four (46.8%) and 45 (47.9%) participants believed that letters of recommendation in the specialty and perceived commitment to their program similarly attained increased significance in evaluating applicants, respectively.

Conclusion

Most PDs and DCs were satisfied with their VI experiences and expect the quality of interns to be unaffected by virtual assessment modalities. Majority opinion on the relative importance of traditional applicant metrics remained largely unchanged, the exception being an increase in importance of prior knowledge of the applicant.

Level of Evidence: N/A.

Faculty Members’ Perspective on Virtual Interviews for Medical Residency Matching during the COVID-19 Crisis: A National Survey

During the COVID-19 pandemic, conducting face-to-face medical residency interviews was challenging due to infection prevention precautions, social distancing, and travel restrictions. Virtual interviews were implemented by the Saudi Commission for Health Specialties (SCFHS) as an alternative process for residency matching while striving to maintain the same quality standards. This national survey was conducted to assess the satisfaction and perceptions of faculty members’ virtual interview performance in the assessment for the medical training residency programs. Among the participating 173 faculty members, 34.1% did not have previous experience with video-conferencing. The Zoom application was the most commonly used platform (65.9%). Most (89.6%) of the faculty perceived virtual interviews as “adequate” platforms on which the candidates could express themselves, while almost half of the faculty (53.8%) agreed that virtual interviews allowed them to accurately reach an impression about the candidates. Overall, 73.4% of faculty felt comfortable ranking the virtually interviewed candidates. We conclude that the acceptance of participating faculty members in the first Saudi medical residency training matching cycle virtual interviewing event was well-perceived. This study provides evidence for future application and research of virtual interviews in residency candidates’ assessment, especially after the pandemic crisis resolves.