Factors Influential in the Selection of Radiology Residents in the Post-Step 1 World: A Discrete Choice Experiment

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis

There are currently no studies examining differences in perceptions and expected impact of the Step 1 score change to pass/fail between surgical and non-surgical program directors (PDs). We conducted a systematic review in May 2023 of PubMed, Scopus, Web of Science, and PSYCInfo to evaluate studies examining PDs' perspectives regarding the Step 1 score change. We performed random-effects meta-analyses to determine differences in perspectives among surgical and non-surgical PDs. Surgical PDs (76.8% [95% CI, 72.1%-82.0%], I2 = 52%) reported significantly greater rates of disagreement with the score change compared to non-surgical (65.1% [95% CI, 57.9%-73.1%], I2 = 69.7%) (P = .01). Surgical PDs also reported significantly greater rates of agreement that the score change will increase the difficulty in objectively comparing applicants (88.1% [95% CI, 84.6%-91.7%], I2 = 16.4%), compared to non-surgical (81.0% [95% CI, 75.6%-86.8%], I2 = 72.6%) (P = .04). There was less heterogeneity among non-surgical PDs (88.7% [95% CI, 86.2%-91.2%], I2 = 0%), compared to surgical (84.7% [95% CI, 79.0%-90.8%], I2 = 67.3%), regarding expected increases in emphasis on Step 2, although the difference in rates of agreement was not statistically significant. Overall, there is significant heterogeneity in the literature regarding expected changes in the residency application review process. Most PDs reported significant disagreement with the score change, greater expected difficulty in objectively evaluating applicants, and greater emphasis on Step 2, with surgical PDs reporting greater rates of disagreement, greater expected difficulty, and heterogeneity regarding expected increases in emphasis on Step 2, compared to non-surgical. Additionally, there is significant heterogeneity in the overall literature regarding expected changes in the residency application review process. Further research is needed to establish evidence-based guidelines that improve the overall residency application process for all stakeholders.

The Influence of Extracurricular Activities on Radiology Resident Selection Decisions

PURPOSE

Extracurricular activities (EAs) listed on radiology residency applications can signal traits and characteristics desired in holistic reviews. The authors conducted an objective analysis to determine the influence of EAs on resident selection decisions.

METHODS

A discrete-choice experiment was designed to model radiology resident selection and determine the relative weights of EAs among academic and demographic application factors. Faculty members involved in resident selection at 30 US radiology programs chose between hypothetical pairs of applicant profiles between October 2021 and February 2022. Each applicant profile included one of 22 EAs chosen for study. A conditional logistic regression model assessed the relative weights of the attributes and odds ratios (ORs) were calculated.

RESULTS

Two hundred forty-four participants completed the exercise. Community-service EAs were ranked most highly by participants. LGBTQ Pride Alliance (OR, 1.56; 95% confidence interval [CI], 1.14-2.15; P = .006) and Young Republicans (OR, 0.60; 95% CI, 0.43-0.82; P = .001) significantly influenced decisions. The highest ranked EAs were significantly preferred over the lowest ranked EAs (OR, 1.916; 95% CI, 1.671-2.197; P < .001). Participants preferred EAs that reflected active over passive engagement (OR, 1.154; 95% CI, 1.022-1.304; P = .021) and progressive over conservative ideology (OR, 1.280; 95% CI, 1.133-1.447; P < .001). Participants who ranked progressive EAs more highly preferred applicants with progressive EAs (P < .05 for all).

CONCLUSIONS

The influence of EAs on resident selection decisions is significant and likely to gain importance in resident selection as medical student performance metrics are further eliminated. Applicants and selection committees should consider this influence and the bias that EAs can bring to resident selection decisions.

Unintended Consequence: Diversity as a Casualty of Eliminating United States Medical Licensing Examination Step 1 Scores

PURPOSE

The purpose of this study was to use a discrete-choice experiment to model the trade-offs evaluators make between academic attributes and demographics when the United States Medical Licensing Examination (USMLE) Step 1 switches to pass/fail.

METHODS

A discrete-choice experiment was administered to faculty members from a geographically diverse mix of 14 academic and community radiology departments in the United States from August through November 2020. Reviewers reviewed 10 applicant pairs with numeric Step 1 scores (part 1) and 10 applicant pairs with a pass Step 1 result (part 2). Applicant attributes included medical school rank, gender, race/ethnicity, USMLE Step 1 score, USMLE Step 2 score, class rank, clerkship honors, and publications. Conditional logistic regression modeled the influence of attribute levels.

RESULTS

Two hundred twelve evaluators completed the study (response rate 59%). The most influential attribute was Step 1 score in part 1 and medical school rank in part 2. The relative importance of race/ethnicity and gender decreased by 25% and 29%, respectively, when Step 1 switches to pass/fail. Evaluators weigh race/ethnicity the strongest when applicants have the same Step 1 score (preference weights of 0.85 for African American, 1.42 for Hispanic, and 0 for White and Asian applicants). Race/ethnicity is relatively more important when Step 1 scores are higher (preference weights of 1.58 for African American, 0.90 for Hispanic, and 0 for White and Asian applicants).

CONCLUSIONS

The loss of numeric Step 1 scores reduced the residency evaluator preference for diversity. Reviewers prioritize underrepresented-in-medicine applicants when Step 1 scores are higher and comparable with White and Asian applicants.

A 5-Year Update on the IR Residency Match: 2022 National Survey Results of Program Directors and Matched Applicants Compared with 2017

PURPOSE

To characterize the experiences of matched applicants (MAs) and program directors (PDs) in the 2022 interventional radiology (IR) residency Match and compare with 2017 data.

METHODS

Surveys were distributed to IR PDs and MAs from the 2022 Match. Findings were compared with those of 2017 using the 2-sample t test and Fisher exact test.

RESULTS

In total, 68 MAs (40%) and 47 PDs (52%) responded. Collected demographic traits were similar, including ongoing male predominance (77% of MAs, 83% of PDs). Moreover, 86% of MAs and 87% of PDs were "satisfied" with Match outcomes. Compared with those in 2017, MAs applied to more IR programs (P < .001). MAs reported more research (P = .003) and abstracts/publications (P < .001) and ranked these as more important than PDs did (P < .001 for both). Approximately 82% of PDs gave special attention to candidates who completed a visiting rotation at their institution; 60% of MAs and 95% of PDs believed virtual interviews resulted in overinterviewing (P < .001); both agreed they provided convenience and accessibility. Furthermore, 63% of MAs believed a Step 1 pass/fail system will be less equitable for applicants. Additional data on demographics, medical school experiences, applications, interviews, intern year, and rank process were reported.

CONCLUSIONS

Satisfaction with Match results remained high from 2017 to 2022, although efforts are needed to improve applicants' ability to navigate the application process, address overapplying, and evaluate concerns regarding the Step 1 pass/fail system. These survey findings will help inform applicants and PDs for future match cycles.

An Elite Privilege: Top-Ranked Medical Schools Provide Fewer Comparative Performance Data on Their Students

PURPOSE

The objective of this study was to determine differences in the reporting of performance data on medical student performance evaluations (MSPEs) by medical school ranking.

METHODS

MSPEs from all US allopathic and osteopathic medical schools received by a single diagnostic radiology residency program during the 2021-2022 application cycle were retrospectively reviewed. Preclinical class and core clerkship grades were categorized as pass/fail or multitiered. Comparative summative assessments provided in the MSPEs were recorded. Medical schools were grouped by their US News & World Report rankings, and the proportion of reported performance metrics for each group was compared.

RESULTS

Information from 95% of US allopathic medical schools (148 of 155) and 73% of osteopathic medical schools (27 of 37) was collected, on the basis of 1,046 applications received. For preclinical classes, multitiered grading was reported by no schools ranked in the top 10, 17% of schools ranked 11th to 50th, 52% of schools ranked 51st to 100th, and 59% of unranked schools (P < .001). For core clinical clerkships, multitiered grades were reported by 70% of the top 10 ranked schools, 90% of schools ranked 11th to 50th, 94% of those ranked 51st to 100th, and 94% of unranked schools (P = .0463). Comparative summative assessments were reported by none of the top 10 ranked schools, 56% of schools ranked 11th to 50th, 80% of those ranked 51th to 100th, and 81% of unranked schools (P < .001).

CONCLUSIONS

Higher ranked medical schools are less likely to provide comparative assessment data on their MSPEs, which may disadvantage students from lower ranked medical schools.

Maintenance of Certification in Radiology: Eliciting Radiologist Preferences Using a Discrete Choice Experiment

OBJECTIVE

To quantitatively assess radiologists' preferences for Maintenance of Certification (MOC) and Continuing Certification (CC) using a survey of attitudes and perceptions.

METHODS

A questionnaire that assessed attitudes and perceptions and included a discrete choice or trade-off task was developed by ACR staff in conjunction with an independent market research agency and the Survey Subcommittee of the ACR Task Force on Certification in Radiology. The trade-off exercise was integrated into this methodology to better understand the underlying utilities or preferences of the components of MOC-CC among respondents and to better enable specific recommendations on how to optimize the current program. The survey was administered via e-mail to 17,305 ACR members. The demographic and practice characteristics of the 1,994 (11.5%) respondents were similar to the ACR radiologist membership and correspond to a normal distribution. At a 95% confidence level, with a margin of error 2.1%, we believe that the respondent population fairly reflects the actual population.

RESULTS

Similar proportions judged the existing program as excellent or very good (36%), or fair or poor (35%), with 27% neutral. MOC-CC was perceived more often as excellent or very good by those who were grandfathered yet still participating in MOC, were in academic practice, were in an urban setting, were older, or had a role with the ABR. In contrast, MOC-CC was more often judged as fair or poor by those who were not grandfathered, were in private practice, were in a rural setting, or were younger. The current MOC-CC program is not well regarded by diplomates, with few showing preference or acceptability. The program's reception is most sensitive to the following attributes: absence or presence of a practice quality improvement requirement, Online Longitudinal Assessment content including or excluding general radiology in addition to one's specialty and inclusion or exclusion of self-assessment as part of the CME.

CONCLUSION

ACR members diverged in their attitudes toward MOC, with differences among specific demographic and practice characteristics. The current package of features of MOC-CC was widely viewed as unsatisfactory, and a more optimal feature set arose from a simulation exercise.

For What It's Worth: Medical School Reputation in The Integrated Plastic Surgery Match

Integrated Plastic and Reconstructive Surgery residency programs may use medical school reputation to help fill the gap of a pass/fail USMLE Step 1 in the match. The main objective of this manuscript was to consider if this shifting emphasis is warranted. Herein, a cross-sectional analysis of academic plastic surgeons found that medical school reputation did not predict career achievement. In the absence of evidence demonstrating its worth, residency programs should exercise caution in using medical school reputation in the match.