Virtual “Matchmaking”: Stakeholder Perspectives on the Future of the Urology Residency Match

The New Urology Match: How Recent Innovations Including Virtual Interviews and Preference Signaling Have Changed Match Outcomes

OBJECTIVES

To determine how recent changes in the urology match occurring from 2021 to 2023, including virtual interviews (VIs) and preference signals (PS), affected match outcomes.

METHODS

The American Urological Association (AUA) match data from 2021 to 2023 was compared to the 15 years prior. This was obtained from the AUA website and a previous study of public AUA match data. Self-reported applicant characteristics and outcomes from the Urology Residency Applicant Spreadsheet 2021-2023 were compared to the four years prior.

RESULTS

Between 2021 and 2023, residency programs offered 43 interviews each, compared to an average of 35 in the 15 years prior. Programs have been receiving more applications each year, from a low of 225 in 2019 to a peak of 347 in 2022. This resulted in an interview offer rate of 13% between 2021 and 2023, compared to 16% in the five years prior. Applicants applied to a mean of 88 programs in 2023, increasing each year since 40 in 2006. Applicants attended 12 interviews on average between 2021 and 2023, compared to 13 in the two years prior. Self-reported applicant data similarly demonstrated that, compared to the four years prior, applicants between 2021 and 2023 applied to more programs (81 vs. 70), had a lower interview offer rate (22% vs. 32%), and a higher interview acceptance rate (90% vs. 75%).

CONCLUSIONS

During the years with VIs, programs offered more interviews and applicants attended fewer on average, indicating a larger applicant pool was interviewed. Despite the introduction of PS, applicants applied to more programs in 2022 and 2023 than ever before.

Effect of Holistic Review, Interview Blinding, and Structured Questions in Resident Selection: Can we Predict Who Will Do Well in a Residency Interview?

OBJECTIVE

To examine the Urology residency application process, particularly the interview. Historically, the residency interview has been vulnerable to bias and not determined to be a predictor of future residency performance. Our goal is to determine the relationship between pre-interview metrics and post-interview ranking using best practices for Urology resident selection including holistic review, blinded interviews, and structured behaviorally anchored questions.

METHODS

Applications were assessed on cognitive (Alpha Omega Alpha, class rank, junior year clinical clerkship grades) and non-cognitive attributes (letters of recommendation [LOR], personal statement [PS], demographics, research, personal characteristics) by reviewers blinded to USMLE scores and photograph. Interviewers were blinded to the application other than PS and LORs. Interviews consisted of a structured behaviorally anchored question (SBI) and an unstructured interview (UI). Odds ratios were determined comparing pre-interview and interview impressions.

RESULTS

Fifty-one applicants were included in the analysis. USMLE step 1 score (average 245) was associated with Alpha Omega Alpha, class rank, junior year clinical clerkship, and PS. The UI score was associated with the LOR (P = .04) whereas SBI scores were not (P = .5). Faculty rank was associated with SBI, UI, and overall interview (OI) scores (P < .001). Faculty rank was also associated with LOR. Resident impression of interviewees were associated with faculty interview scores (P = .001) and faculty rank (P < .001).

CONCLUSION

Traditional interviews may be biased toward application materials and may be balanced with behavioral questions. While Step 1 score does not offer additional information over other PI metrics, blinded interviews may offer discriminant validity over a PI rubric.

Preference Signaling Pilot in the Urology Match: Outcomes and Perceptions

OBJECTIVE

To assess the perceptions and outcomes of the A preference signaling (PS) pilot implemented by the Society of Academic Urology (SAU) in the 2021-2022 Urology Match (UM) cycle.

METHODS

5 non-weighted signals were provided to each applicant, and signals were delivered to programs by the AUA to provide an applicant-centered formal and equitable process to express genuine interest in residency programs. Applicant and program perception and behavior was assessed through surveys. Signal distribution, mean, range, and interview offer rates for overall, signaled, and non-signaled programs were calculated.

RESULTS

In the 2021-2022 UM cycle, 566 applicants completed signaling; 2829 total signals were sent to programs with 97% applicant and 100% program participation. Each program received a mean number of 19 signals (range of 1 to 62), and 25% of programs received 49% of all signals. The overall interview-offer rate for the cohort was 12.5% (6,019 interviews held/47,989 applications received); the signaled interview rate was 51% (1,443/2,829), and the non-signaled interview rate was 10% (4,576/45,160) with an approximate 4-fold increase in rate of interview for signaled programs compared to non-signaled programs. There was moderate to high levels of satisfaction from applicants and programs on the overall process. 48% of programs incorporated PS into initial application review.

CONCLUSIONS

Preference signaling was demonstrated to be feasible and successful in providing a novel applicant-directed, formal, equitable and credible structured process for applicants to express genuine interest in programs. Additionally, programs were able to incorporate PS into their interview selection and recruitment process.