The Virtual Urology Residency Match Process: Moving Beyond the Pandemic

The New Urology Match: How Recent Innovations Including Virtual Interviews and Preference Signaling Have Changed Match Outcomes

OBJECTIVES

To determine how recent changes in the urology match occurring from 2021 to 2023, including virtual interviews (VIs) and preference signals (PS), affected match outcomes.

METHODS

The American Urological Association (AUA) match data from 2021 to 2023 was compared to the 15 years prior. This was obtained from the AUA website and a previous study of public AUA match data. Self-reported applicant characteristics and outcomes from the Urology Residency Applicant Spreadsheet 2021-2023 were compared to the four years prior.

RESULTS

Between 2021 and 2023, residency programs offered 43 interviews each, compared to an average of 35 in the 15 years prior. Programs have been receiving more applications each year, from a low of 225 in 2019 to a peak of 347 in 2022. This resulted in an interview offer rate of 13% between 2021 and 2023, compared to 16% in the five years prior. Applicants applied to a mean of 88 programs in 2023, increasing each year since 40 in 2006. Applicants attended 12 interviews on average between 2021 and 2023, compared to 13 in the two years prior. Self-reported applicant data similarly demonstrated that, compared to the four years prior, applicants between 2021 and 2023 applied to more programs (81 vs. 70), had a lower interview offer rate (22% vs. 32%), and a higher interview acceptance rate (90% vs. 75%).

CONCLUSIONS

During the years with VIs, programs offered more interviews and applicants attended fewer on average, indicating a larger applicant pool was interviewed. Despite the introduction of PS, applicants applied to more programs in 2022 and 2023 than ever before.

The Impact of the Virtual Interview on Candidate Experience and Outcomes: Society of Academic Urologists National Survey of Interns Who Participated in the 2020-2021 Urology Match

OBJECTIVE

To examine retrospective experiences with the virtual interview (VI) process among postgraduate year 1 (PGY1) urology residents who participated in the 2020-2021 American Urologic Association (AUA) Match cycle.

METHODS

A 27-question survey created by a Society of Academic Urologists Taskforce on VI was distributed to PGY1 residents from 105 institutions between February 1, 2022 and March 7, 2022. The survey asked respondents to reflect on the VI process, cost concerns, and how experiences at their current program aligned with prior VI representation.

RESULTS

A total of 116 PGY-1 residents completed the survey. The majority felt the VI represented the following domains well: (1) institution/program culture and strengths (74%), (2) representation of all faculty/disciplines (74%), (3) resident quality of life (62%), (4) personal fit (66%), (5) quality of surgical training and volume (63%), and (6) opportunities to meet residents (60%). Approximately 71% of respondents did not match at their home program or a program they visited in-person. Within this cohort, 13% agreed important aspects of their current program were not translated virtually, and they would not have prioritized the program had they been able to visit in-person. In total, 61% ranked programs they ordinarily might not have listed during an in-person interview season. Overall, 25% deemed financial costs a "very important" consideration during the VI process.

CONCLUSION

The majority of PGY1 urology residents reported key components of their current program translated well from the VI process. This platform offers a method of overcoming conventional geographic and financial barriers associated with the in-person interview process.

Effect of Holistic Review, Interview Blinding, and Structured Questions in Resident Selection: Can we Predict Who Will Do Well in a Residency Interview?

OBJECTIVE

To examine the Urology residency application process, particularly the interview. Historically, the residency interview has been vulnerable to bias and not determined to be a predictor of future residency performance. Our goal is to determine the relationship between pre-interview metrics and post-interview ranking using best practices for Urology resident selection including holistic review, blinded interviews, and structured behaviorally anchored questions.

METHODS

Applications were assessed on cognitive (Alpha Omega Alpha, class rank, junior year clinical clerkship grades) and non-cognitive attributes (letters of recommendation [LOR], personal statement [PS], demographics, research, personal characteristics) by reviewers blinded to USMLE scores and photograph. Interviewers were blinded to the application other than PS and LORs. Interviews consisted of a structured behaviorally anchored question (SBI) and an unstructured interview (UI). Odds ratios were determined comparing pre-interview and interview impressions.

RESULTS

Fifty-one applicants were included in the analysis. USMLE step 1 score (average 245) was associated with Alpha Omega Alpha, class rank, junior year clinical clerkship, and PS. The UI score was associated with the LOR (P = .04) whereas SBI scores were not (P = .5). Faculty rank was associated with SBI, UI, and overall interview (OI) scores (P < .001). Faculty rank was also associated with LOR. Resident impression of interviewees were associated with faculty interview scores (P = .001) and faculty rank (P < .001).

CONCLUSION

Traditional interviews may be biased toward application materials and may be balanced with behavioral questions. While Step 1 score does not offer additional information over other PI metrics, blinded interviews may offer discriminant validity over a PI rubric.

Influencing Factors of Preference Signaling in the 2022 Urology Residency Match

OBJECTIVE

To evaluate influencing factors of preference signaling (PS) among urology residency applicants during the 2022 American Urological Association (AUA) Match.

METHODS

We emailed an anonymous, deidentified questionnaire survey to applicants to our institution for the 2022 AUA Match. The main question asked to applicants was "What factor(s) went into your decision to send 'Program X' a preference signal?" Certain questions allowed the selection of multiple options, and applicants were further asked to specify these by selecting a single most important option. Descriptive statistical analyses were conducted using IBM SPSS software.

RESULTS

Out of 601 registrants to the AUA match, 324 individuals applied to our institution and therefore received a survey; 77 responded for a 24% response rate. A total of 383 PS were sent by the 77 applicants. Overall, 73% and 49% of the total 383 PS had program location and reputation, respectively, selected as an influencing factor. More than 1 influencing factor was considered in 73% of PS selections, with program location (45%) considered the most important factor. In relation to applicant competitiveness, 35% of PS were sent to perceived "target" programs, 31% to "reach" programs, and 8% to "safety" programs. Among respondents who matched, 75% matched at a home, away, or signaled program.

CONCLUSION

Program location appears to be the most influential factor in sending a program a PS. Programs were also signaled based on applicant's perception of their own competitiveness. PS appears to have a possible beneficial impact on obtaining interviews and successfully matching.

Preference Signaling and Virtual Interviews: The New Urology Residency Match

OBJECTIVE

To define applicant response to the preference signaling program and continuing virtual aspects of the 2022 Urology Residency Match to guide future decisions surrounding this process.

METHODS

We emailed an anonymous, de-identified 20-question, multiple choice survey to all applicants to our institution for the 2022 Urology Residency Match (RedCap). Where appropriate, comparisons were made to already published data collected in an identical manner from applicants to our institution for the 2021 Urology Residency Match.

RESULTS

Of the 418 survey recipients, 155 (37%) responded to our survey. A majority of applicants (83%) thought that preference signaling should remain in future years, and 66% of applicants matched to a program to which they had signaled or where they completed a subinternship. Geographic location of programs was ranked to have the highest impact on choice of programs for preference signaling. Fifty-two percent of 2022 applicants thought that interviews should remain virtual compared with 39% of 2021 applicants (P = .03). Twenty-one percent of 2022 applicants agreed that pre/post-interview socials were well-replicated virtually compared with 10% of 2021 applicants (P = .04).

CONCLUSION

A majority of urology applicants were satisfied with the preference signaling program, suggesting that preference signaling should remain in future matches. A majority of urology applicants now favor the virtual interview platform. While it is gaining greater acceptance among applicants, the virtual platform generally still carries deficiencies. Further research of the urology match process is necessary for continued optimization of the program for all stakeholders.

A survey of pediatric urology fellowship applicants: past priorities to guide future possibilities

BACKGROUND

The number of applicants to pediatric urology fellowships is often lower than the available positions (chart), giving applicants significant influence over where they ultimately match. Historically, interviews were conducted at individual hospitals, in-person, with residents bearing most costs. The objective of this study was to understand the factors associated with where applicants decide to apply, interview, and match for fellowship, as well as barriers within this process.

METHODS

A 24-question survey was sent via email to all applicants who successfully matched into pediatric urology fellowship from 2013 to 2019. Questions included: demographics; factors associated with where they applied, interviewed, and ranked; and barriers within the application process.

RESULTS

A total of 126 recent and current fellows were contacted, and 73 (60%) completed the full survey (51% male and 49% female). On average, respondents applied to 10 programs, interviewed at 9, and ranked 8. The most important factors in choosing where to apply/interview were: volume of surgical cases, diversity of surgical cases, and advice from mentors. The most important factors when making a rank list were: clinical autonomy, reputation of program, and structure of program. Hospital facilities were only rated "important" by 12% of respondents. 82% (60 respondents) faced at least one personal or professional barrier during the application process. The most common barrier was "cost of interviewing" (59%, 43 respondents). Personal vacation time was used by 61% of applicants during interviews, with 37% using more than 5 days.

DISCUSSION

This study is the first to explore the factors that applicants consider when choosing where to apply, interview, and rank for pediatric urology fellowship. This information is important to understand due to the current supply and demand of fellowship positions. We are limited by extrapolating more general conclusions about applicants as a whole from a survey with a 60% response rate and the lack of an available validated survey in this realm.

CONCLUSIONS

This study has shown that most pediatric urology fellowship applicants apply to programs primarily based on perceived surgical volume and reputational factors. These same factors are used when making a rank list. Many applicants faced personal or professional barriers during the application process, largely due to costs and time away from work and family. While recent interviews have transitioned to a virtual format by necessity, prior applicants did not rate hospital facilities as important to them. Overall, there is room to improve this process based on such feedback.

No difference in subsequent trainee satisfaction associated with in-person exposure prior to remote interviews

BACKGROUND

In 2020-2021, residency and fellowship applicants participated in virtual interviews. There was concern that trainees who had not been to the area before would potentially have different satisfaction with their new workplace and community.

OBJECTIVE

To compare satisfaction and likelihood to recommend work and community among new trainees with or without prior exposure to a single academic center or its environs.

METHODS

We conducted an IRB-approved cross-sectional survey of new trainees. An electronic survey included demographic items, self-report of prior exposure to the area, satisfaction with the program and area, and likelihood to recommend the program and area. Descriptive statistics were used for responses and Chi square tests for comparisons.

RESULTS

In September 2021 and May 2022 electronic surveys were sent to all 173 trainees who started residency or fellowship in July 2021, which had 87 responses (50.3% response rate) and 31 (18.0% response rate) responses, respectively. At both times, most respondents were interns. The majority of the September group (55.6%), while 38.7% of the May group had prior exposure to the area. Overall, the majority were satisfied with Penn State Health and would recommend their workplace. The majority also agreed they were satisfied with their new community and would recommend it to others. There were no significant differences in the proportions of satisfied trainees for any of the four outcomes at either timepoint.

CONCLUSIONS

Satisfaction with training and the community were not significantly different for trainees with or without prior in-person exposure to the institution or surrounding area.

Virtual interviewing for graduate medical education recruitment and selection: A BEME systematic review: BEME Guide No. 80

BACKGROUND

The COVID-19 pandemic caused graduate medical education (GME) programs to pivot to virtual interviews (VIs) for recruitment and selection. This systematic review synthesizes the rapidly expanding evidence base on VIs, providing insights into preferred formats, strengths, and weaknesses.

METHODS

PubMed/MEDLINE, Scopus, ERIC, PsycINFO, MedEdPublish, and Google Scholar were searched from 1 January 2012 to 21 February 2022. Two authors independently screened titles, abstracts, full texts, performed data extraction, and assessed risk of bias using the Medical Education Research Quality Instrument. Findings were reported according to Best Evidence in Medical Education guidance.

RESULTS

One hundred ten studies were included. The majority (97%) were from North America. Fourteen were conducted before COVID-19 and 96 during the pandemic. Studies involved both medical students applying to residencies (61%) and residents applying to fellowships (39%). Surgical specialties were more represented than other specialties. Applicants preferred VI days that lasted 4-6 h, with three to five individual interviews (15-20 min each), with virtual tours and opportunities to connect with current faculty and trainees. Satisfaction with VIs was high, though both applicants and programs found VIs inferior to in-person interviews for assessing 'fit.' Confidence in ranking applicants and programs was decreased. Stakeholders universally noted significant cost and time savings with VIs, as well as equity gains and reduced carbon footprint due to eliminating travel.

CONCLUSIONS

The use of VIs for GME recruitment and selection has accelerated rapidly. The findings of this review offer early insights that can guide future practice, policy, and research.

Thriving in the Virtual Era: An Organized Approach to Improve Program Visibility and Virtual Residency Interview Process

OBJECTIVE

Since residency interviews became virtual due to COVID-19, and likely continue in the future, programs must find ways to improve their non-traditional recruiting methods. The objective of this study was to evaluate effectiveness of a structured, non-traditional approach on visibility and perception of the program as well as virtual interview experience.

METHODS

The focus of our approach was to ensure constant engagement while maintaining all pre-interview communication as resident-led and informal. The program focused on improving visibility and outreach through an organized utilization of social media platforms highlighting people and local culture. The virtual interview process was restructured with resident-led virtual meet and greets followed by small group discussions and providing virtual hospital tours, videos, and slides of the program's culture and expectationson the interview day. Perception of the program and the new approach to the interview process was assessed via an anonymous survey.

RESULTS

The program's visibility was measured via social media analytics with an increase in reach on Facebook from 0/post to as high as 4200/post and engagement 2/post to nearly 600/post. Tweet Impressions from approximately 350/mo to 11,000/mo with the increase in new Followers/month by 532.5%. Increase in total number of applicants in 2021 of 16% compared to average between 2018 and 2020. Survey response rate was 66.1%; of those 53.8% of interviewees attended a virtual meet and greet session. Perceptions of interviewees on our program was exclusively positive. Specific characteristics of the program that would make students rank us higher were program's culture, people, academics, and clinical experiences they would get as residents.

CONCLUSIONS

The exponential increase in our program's visibility and exclusively positive program assessment suggest that a structured approach utilizing social media and virtual technologies could improve both the recruitment and the virtual interview process while maintaining positive perceptions of the program.

The Impact of Visiting Rotations on the Urology Residency Match: Insights and Perspectives from the 2021-2022 Applicants

OBJECTIVE

To survey 2022 Urology Residency Match applicants on their away rotation experiences to better understand the impact of pandemic changes, including the one in-person away rotation limit, on interviews and match outcomes.

METHODS

An anonymous post-match online survey was distributed to applicants of study-participating institutions. Data was prospectively collected regarding respondent demographics, away rotation experiences, interviews, utilization of preference signaling, and overall match outcomes.

RESULTS

In total, 230 applicants completed the survey. Respondents prioritized geography (28.1%), program reputation (26.1%), and early notification (25.6%) as the top factors in determining whether to accept a rotation offer. The majority (93%) participated in a single away rotation, with nearly all respondents being offered a virtual interview from the program where they completed a visiting clerkship. Of those who declined away rotation offers, 56% were not offered an interview at the program they declined an offer from. The majority matched at either their home institution (20.8%), a program where they submitted a preference signal (22.2%), or a program where they completed an in-person away rotation (13%). Despite 46.3% of respondents reporting that their match outcome was negatively affected by the ability to rotate at only one away rotation, the majority were satisfied (70%) with their rotation selection based on match results.

CONCLUSIONS

For applicants, participation in visiting rotations plays a significant role in the match process. Further research regarding mechanisms to optimize the away rotation application process is needed as the capacity to do visiting electives expands in the post-pandemic environment.