Innovation in Residency Selection: The AAMC Standardized Video Interview

The Inconspicuous Learner Handover: An Exploratory Study of U.S. Emergency Medicine Program Directors' Perceptions of Learner Handovers from Medical School to Residency

Phenomenon: Central to competency-based medical education is the need for a seamless developmental continuum of training and practice. Trainees currently experience significant discontinuity in the transition from undergraduate (UME) to graduate medical education (GME). The learner handover is intended to smooth this transition, but little is known about how well this is working from the GME perspective. In an attempt to gather preliminary evidence, this study explores U.S. program directors (PDs) perspective of the learner handover from UME to GME. Approach: Using exploratory qualitative methodology, we conducted semi-structured interviews with 12 Emergency Medicine PDs within the U.S. from October to November, 2020. We asked participants to describe their current perception of the learner handover from UME to GME. Then we performed thematic analysis using an inductive approach. Findings: We identified two main themes: The inconspicuous learner handover and barrier to creating a successful UME to GME learner handover. PDs described the current state of the learner handover as "nonexistent," yet acknowledged that information is transmitted from UME to GME. Participants also highlighted key challenges preventing a successful learner handover from UME to GME. These included: conflicting expectations, issues of trust and transparency, and a dearth of assessment data to actually hand over. Insights: PDs highlight the inconspicuous nature of learner handovers, suggesting that assessment information is not shared in the way it should be in the transition from UME to GME. Challenges with the learner handover demonstrate a lack of trust, transparency, and explicit communication between UME and GME. Our findings can inform how national organizations establish a unified approach to transmitting growth-oriented assessment data and formalize transparent learner handovers from UME to GME.

Virtual Residency Interviews- A Survival Guide and Lessons Learnt

With the outbreak of Coronavirus Disease 2019 (COVID-19) pandemic, the residency programs were required to conduct virtual/online interviews for recruitment of candidates for their residency programs. While both programs and the candidates had challenges, with the abrupt transition of interviews to online platform, there were some perceived benefits by the applicants. This paper will further review the pros and cons, the challenges and the changes brought about by the online transformation of residency interviews and conclude with tips to the residency applicants and lessons learnt from this transition. Although, residency programs are considering going back to in-person interviews, they may continue to offer virtual interview as well to the candidates in the future.

Video-based interviewing in medicine: a scoping review

BACKGROUND

The Coronavirus 2019 pandemic necessitated a rapid uptake of video-based interviewing within the personnel selection process in healthcare. While video-based interviews have been evaluated previously, we identified a gap in the literature on the implementation of video-based interviews and how they compare to their face-to-face counterparts.

METHODS

A scoping review was conducted to consolidate the available literature on the benefits and limitations of video-based interviews and to understand the perceived barriers associated with transitioning away from face-to-face interviews. A search strategy, developed in concert with an academic health sciences librarian, was run on Ovid MEDLINE, Embase, PsycInfo, and Cochrane Central. The search was performed on March 31, 2020, and updated on February 21, 2021. Studies that implemented and evaluated the impact of video-based interviewing in healthcare were included in our study. Review articles and editorials were excluded.

RESULTS

Forty-three studies were included in our scoping review, of which 17 were conference abstracts and 26 were peer-reviewed manuscripts. The risk of bias was moderate or high in most studies, with only four studies having a low risk of bias. Both financial costs and opportunity costs associated with the selection process were reported to be improved with video-based interviewing, while no studies explored the impact on environmental costs. Technical limitations, which were not prevalent, were easily managed during the interview process. Overall, video-based interviews were well received by both applicants and interviewers, although most participants still reported a preference for face-to-face interviews.

CONCLUSIONS

While video-based interviewing has become necessary during the Coronavirus 2019 era, there are benefits from a financial, opportunistic, and environmental point of view that argue for its continued use even after the pandemic. Despite its successful implementation with minimal technical issues, a preference still remains for face-to-face interviews. Reasons for this preference are not clear from the available literature. Future studies on the role of nonverbal communication during the video-based interview process are important to better understand how video-based interviewing can be optimized.

SYSTEMATIC REVIEW REGISTRATION

This scoping review was registered with Open Science Framework.

Adapting the Admissions Interview During COVID-19: A Comparison of In-Person and Video-Based Interview Validity Evidence

COVID-19 physical distancing limited many medical schools' abilities to conduct in-person interviews for the 2020 admissions cycle. The University of Toronto (U of T) Temerty Faculty of Medicine was already in the midst of its interview process, with two-thirds of applicants having completed the in-person modified personal interview (MPI). As the university and surrounding region were shut down, the shift was made in the middle of the application cycle to a semisynchronous video-based MPI interview (vMPI) approach. U of T undertook the development, deployment, and evaluation of the 2 approaches mid-admissions cycle. Existing resources and tools were used to create a tailored interview process with the assistance of applicants. The vMPI was similar in content and process to the MPI: a 4-station interview with each station mapped to attributes relevant to medical school success. Instead of live interviews, applicants recorded 5-minute responses to questions for each station using their own hardware. These responses were later assessed by raters asynchronously. Out of 627 applicants, 232 applicants completed the vMPI. Validity evidence was generated for the vMPI and compared with the MPI on the internal structure, relationship to other variables, and consequential validity, including applicant and interviewer acceptability. Overall, the vMPI demonstrated similar reliability and factor structure to the MPI. As with the MPI, applicant performance was predicted by nonacademic screening tools but not academic measures. Applicants' acceptance of the vMPI was positive. Most interviewers found the vMPI to be acceptable and reported confidence in their ratings. Continuing physical distancing concerns will require multiple options for admissions committees to select medical students. The vMPI is an example of a customized approach that schools can implement and may have advantages for selection beyond the COVID-19 pandemic. Future evaluation will examine additional validity evidence for the tool.

Innovation in Resident Selection: Life Without Step 1

INTRODUCTION

The announcement of Step 1 shifting to a Pass/Fail metric has prompted resident selection committees (RSCs) to pursue objective methods of evaluating prospective residents. Regardless of the program's specialty or affiliated hospital/school, RSCs universally aim to recognize and choose applicants who are an "optimal fit" to their programs.¹ An optimal fit can be defined as a candidate who thrives in the clinical and academic setting, both contributing to and benefiting from their respective training environments.

OBJECTIVE

The objective of this scoping review is to evaluate alternative, innovative methods by which RSCs can evaluate applicants and predict success during residency. Objective methods include: Step 2 scores, Traditionally Used Metrics (core clerkship scores), interview performance, musical talent, sports involvement, AOA membership, research publications, unprofessional behavior, Dean's letters, Rank list, judgement testing, and specialty-specific shelf exams.^13-15.

METHODS

A scoping review was performed in compliance with the guidelines indicated by the PRISMA Protocol for scoping review.¹⁸ 9308 results were identified in the original PubMed search for articles with the key words "Resident Success". Abstract screening and application of inclusion and exclusion criteria yielded 97 articles that were critically appraised via review of full manuscript.

RESULTS

Of the articles that focused on personality traits, situational judgement testing, and specialty specific pre-assessment, all of them demonstrated some level of predictability for resident success. Standardized Letter of Recommendations, Traditionally Used Metrics, and STEP 2 did not show a unanimous consensus in demonstrating predictability of a resident's success, this is because some articles suggested predictability and some articles disputed predictability.

CONCLUSION

The authors found personality traits, situational judgement testing, and specialty specific assessments to be predictive in selecting successful residents. Further research should aim to analyze exactly how RSCs utilize these assessment tools to aid in screening their large and competitive applicant pools to find residents that will be successful in their program.

Resource Utilization and Emergency Medicine Advisors' Approach to Video Interview Preparation

Introduction The Standardized Video Interview (SVI) was a residency application component introduced by the Association of American Medical Colleges (AAMC) as a supplement to the existing process, which aimed to measure knowledge of professional behaviors and interpersonal skills. Given its novelty in both aim and execution, little advice or experience was available to inform preparation strategies. We sought to perform a cross-sectional analysis to explore advisors' practices in guiding students' preparation for the SVI. Methods An electronic questionnaire was developed and piloted for flow and usability, then distributed to all members of the Council of Residency Directors in Emergency Medicine (CORD EM), the professional society for emergency medicine educators, via listserv, comprised of 270 residency programs. Questions were both open- and closed-ended and therefore analyzed in a mixed-method fashion.  Results We received 56 responses from a listserv representing 270 residency programs. Respondents cited personal experience and consensus opinions from national organizations as the primary sources for their advice. The most common resources offered to students were space for completing the SVI (41%) or technical support for completing the SVI (47%). The time committed to student advising specifically for the SVI ranged from zero to 20 hours. Estimated associated costs of preparation ranged from zero up to $10,000 (time plus resources). Two individuals reported recommending commercial preparation resources to students.  Conclusion The SVI was a novel attempt to augment the resident application process. We found variability in resources and advice offered to students, including broad ranges of time dedicated, the monetary value of resources contributed, and the types of resources utilized. As the global COVID-19 pandemic has inspired a wave of innovation and process changes, we present this data for consideration as a snapshot of the variable responses to a single uniform process change.

Virtual Ophthalmology Fellowship Interviews: Perceptions of U.S. Ophthalmology Fellowship Applicants in 2020

Objective This study aimed to evaluate the experiences and preferences of ophthalmology fellowship applicants utilizing a virtual interview format.

Design Present study is a cross-sectional study.

Subjects All fellowship applicants to Wills Eye Hospital during 2020 to 2021 application cycle were included.

Methods A nonvalidated, online survey was conducted, and surveys were distributed at the conclusion of the interview process after rank list submission.

Main Outcome Measures Applicant demographics, application submissions, interview experiences, financial considerations, and suggestions for improvement of the virtual interview process were the primary outcomes of this cross-sectional study.

Results Survey responses were received from 68 fellowship applicants (34% response rate). Thirty (44%) applicants preferred in-person interviews, 25 (36%) preferred virtual interviews, and 13 (19%) would like to prefer the option to choose either. Fifty-five of 68 (80%) applicants attended the same range of interviews for which they received interview invitations. Reduced costs were reported as the highest ranked strength of virtual interviews in 44 (65%) applicants, with a majority of respondents (68%) spending less than U.S. $250 throughout the entire process. The highest ranked limitation for virtual interviews was limited exposure to the culture/environment of the program in 20 (29%) respondents. On a scale of 0 to 100, the mean (standard deviation [SD]) satisfaction level with the fellowship application process was 74.6 (18.3) and mean (SD) perceived effectiveness levels of virtual interviews was 67.4 (20.4).

Conclusion Respondents were generally satisfied with virtual interviews and noted reduced costs and increased ability to attend more fellowship interviews as the strengths of the virtual interview format. Limited exposure to the culture/environment of the program was cited as the most important limitation.

A National Survey Evaluating the Impact of the COVID-19 Pandemic on Students Pursuing Careers in Neurosurgery

Background: The COVID-19 pandemic has profoundly disrupted medical education and the residency application process. Methods: We conducted a descriptive observational study in April 2020 of medical students and foreign medical graduates considering or pursuing careers in neurosurgery in the United States to examine the impact of the pandemic. Results: A total of 379 respondents from 67 medical schools completed the survey. Across all participants, 92% (n = 347) stopped in-person didactic education, and 43% (n = 161) experienced basic science and 44% (n = 167) clinical research delays. Sixty percent (n = 227) cited a negative impact on academic productivity. Among first year students, 18% (n = 17) were less likely to pursue a career in neurosurgery. Over half of second year and third year students were likely to delay taking the United States Medical Licensing Examination Steps I and II. Among third year students, 77% (n = 91) reported indefinite postponement of sub-internships, and 43% (n = 53) were unsatisfied with communication from external programs. Many fourth-year students (50%, n = 17) were graduating early to participate in COVID-19-related patient care. Top student-requested support activities included access to student-focused educational webinars and sessions at upcoming conferences. Conclusions: Medical students pursuing careers in neurosurgery faced unique academic, career, and personal challenges secondary to the pandemic. These challenges may become opportunities for new initiatives guided by professional organizations and residency programs.

Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

BACKGROUND

Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged.

OBJECTIVE

This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process.

METHODS

Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories.

RESULTS

Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13).

CONCLUSIONS

This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change.

Computer-based test (CBT) and OSCE scores predict residency matching and National Board assessment results in Japan

CONTEXT

The Japan Residency Matching Program (JRMP) launched in 2003 and is now a significant event for graduating medical students and postgraduate residency hospitals. The environment surrounding JRMP changed due to Japanese health policy, resulting in an increase in the number of unsuccessfully-matched students in the JRMP. Beyond policy issues, we suspected there were also common characteristics among the students who do not get a match with residency hospitals.

METHODS

In total 237 out of 321 students at The University of Tokyo Faculty of Medicine graduates from 2018 to 2020 participated in the study. The students answered to the questionnaire and gave written consent for using their personal information including the JRMP placement, scores of the pre-clinical clerkship (CC) Objective Structured Clinical Examinations (OSCE), the Computer-Based Test (CBT), the National Board Examination (NBE), and domestic scores for this study. The collected data were statistically analyzed.

RESULTS

The JRMP placements were correlated with some of the pre-CC OSCE factors/stations and/or total scores/global scores. Above all, the result of neurological examination station had most significant correlation between the JRMP placements. On the other hand, the CBT result had no correlation with the JRMP results. The CBT results had significant correlation between the NBE results.

CONCLUSIONS

Our data suggest that the pre-clinical clerkship OSCE score and the CBT score, both undertaken before the clinical clerkship, predict important outcomes including the JRMP and the NBE. These results also suggest that the educational resources should be intensively put on those who did not make good scores in the pre-clinical clerkship OSCE and the CBT to avoid the failure in the JRMP and the NBE.

A Novel Ticket System for Capping Residency Interview Numbers: Reimagining Interviews in the COVID-19 Era

The 2019 novel coronavirus (COVID-19) pandemic has led to dramatic changes in the 2020 residency application cycle, including halting away rotations and delaying the application timeline. These stressors are laid on top of a resident selection process already under duress with exploding application and interview numbers-the latter likely to be exacerbated with the widespread shift to virtual interviewing. Leveraging their trainee perspective, the authors propose enforcing a cap on the number of interviews that applicants may attend through a novel interview ticket system (ITS). Specialties electing to participate in the ITS would select an evidence-based, specialty-specific interview cap. Applicants would then receive unique electronic tickets-equal in number to the cap-that would be given to participating programs at the time of an interview, when the tickets would be marked as used. The system would be self-enforcing and would ensure each interview represents genuine interest between applicant and program, while potentially increasing the number of interviews-and thus match rate-for less competitive applicants. Limitations of the ITS and alternative approaches for interview capping, including an honor code system, are also discussed. Finally, in the context of capped interview numbers, the authors emphasize the need for transparent preinterview data from programs to inform applicants and their advisors on which interviews to attend, learning from prior experiences and studies on virtual interviewing, adherence to best practices for interviewing, and careful consideration of how virtual interviews may shift inequities in the resident selection process.

Planning for Virtual Interviews: Residency Recruitment During a Pandemic

Recent directives from The Coalition for Physician Accountability, Association of Pediatric Program Directors, Council on Medical Student Education in Pediatrics, and Association of Medical School Pediatric Department Chairs recommend virtual recruitment for graduate medical education (GME) programs this year. Driven by concerns for safety, equity and financial pressures, virtual interviewing will require a thoughtful approach to achieve the desired goals of recruitment. Drawing on the fields of communication and human resources management, we examine the potential effects of web-based interviews on communication through the lens of several communication theories: media richness, media naturalness, and signaling. We then review the literature on virtual interviews in GME training programs. Finally, we will provide best practices compiled from the literature.

Preparing for the interviewing process during Coronavirus disease-19 pandemic: Virtual interviewing experiences of applicants and interviewers, a systematic review

Purpose

Coronavirus disease-19 (COVID-19) has forced upon all academic institutions to conduct virtual interviewing (VI) instead of face-to-face interviewing (FTFI) this interviewing cycle. The purpose of this systematic review was to understand the process of VI, its effectiveness as an alternative to FTFI, and the experiences of applicants and institutions with VI. We also share best practice strategies for applicants and institutions in VI preparation.

Method

PubMed/MEDLINE, Cochrane Library of Systematic Reviews, Web of Science Core Collection, Scopus and CINAHL databases were searched through May 2020. Articles in English evaluating the effectiveness of VI were included, without applying any date limits. Two reviewers selected articles and extracted data.

Results

Of the 934 articles screened, 22 articles underwent full-text article analysis to include 15 studies. There were 4 studies that reported the use of VI as a screening tool. 11 studies completely replaced FTFI with VI. Most applicants could appropriately convey themselves through VI. Most applicants and interviewing programs expressed reservations about VI’s use as an alternative to FTFI.

Conclusion

There is dearth of evidence supporting the efficacy of VI. There is an opportunity for potential research at multi-institutional level to gain better understanding of the efficacy of VI. The knowledge obtained from this systematic review has the potential of helping applicants and institutions in preparing for VI process. Additionally, authors propose supportive strategies to help prepare applicants and institutions for VI.

Videoconference Interviews: A Timely Primary Care Residency Selection Approach

BACKGROUND

Videoconference interviews (VCIs) are increasingly being used in the selection process of residency program candidates across a number of medical specialties, but nevertheless remain an underutilized approach, particularly in the field of primary care.

OBJECTIVE

This retrospective data review with cost analysis explores financial and acceptability outcomes of VCI implementation over a 9-year period.

METHODS

VCIs were incorporated into the recruitment process at a community-based academic family medicine residency program in 2011, whereby suitable candidates were selected for VCIs after Electronic Residency Application Service (ERAS) application review. Based on the outcome of VCI, candidates were invited via a structured interview tool for a subsequent in-person interviews to determine final rank decisions. Costs of the interview process were tracked, as well as perceptions of VCIs.

RESULTS

VCI implementation over 9 years demonstrated a median 48% reduction of in-person interviews-or 95 applicants eliminated out of a total 195 VCIs performed. This represents a mean annual direct cost savings estimated at $9,154, equating to a 55% reduction in allocated program costs, in addition to indirect cost savings to both applicants and the program.

CONCLUSIONS

Compared to exclusively in-person interviewing, the utilization of VCIs is potentially more cost-effective for residency programs and candidates, while creating a more personal experience for applicants early in the recruitment process. Limited data of acceptability among faculty and candidates is generally favorable but remains mixed.

Reimagining Residency Selection: Part 3-A Practical Guide to Ranking Applicants in the Post-COVID-19 Era

Collapse

Resident Selection in the Wake of United States Medical Licensing Examination Step 1 Transition to Pass/Fail Scoring

INTRODUCTION

The numeric score for the United States Medical Licensing Examination Step 1 is one of the only universal, objective, scaled criteria for comparing the many students who apply to orthopaedic surgery residency. However, on February 12, 2020, it was announced that Step 1 would be transitioning to pass/fail scoring. The purpose of this study was to (1) determine the most important factors used for interview and resident selection after this change and (2) to assess how these factors have changed compared with a previous report on resident selection.

METHODS

A survey was distributed to the program directors (PDs) of all 179 orthopaedic surgery programs accredited by the Accreditation Council for Graduate Medical Education. Questions focused on current resident selection practices and the impact of the Step 1 score transition on expected future practices.

RESULTS

A total of 78 PDs (44%) responded to the survey. Over half of PDs (59%) responded that United States Medical Licensing Examination Step 2 clinical knowledge (CK) score is the factor that will increase most in importance after Step 1 transitions to pass/fail, and 90% will encourage applicants to include their Step 2 CK score on their applications. The factors rated most important in resident selection from zero to 10 were subinternship performance (9.05), various aspects of interview performance (7.49 to 9.01), rank in medical school (7.95), letters of recommendation (7.90), and Step 2 CK score (7.27). Compared with a 2002 report, performance on manual skills testing, subinternship performance, published research, letters of recommendations, and telephone call on applicants' behalf showed notable increases in importance.

DISCUSSION

As Step 2 CK is expected to become more important in the residency application process, current applicant stress on Step 1 scores may simply move to Step 2 CK scores. Performance on subinternships will remain a critical aspect of residency application, as it was viewed as the most important resident selection factor and has grown in importance compared with a previous report.

The role of the interview in residency selection: A mixed-methods study

CONTEXT

Residency programmes invest considerable time and resources in candidate interviews as a result of their perceived ability to reveal important social traits. However, studies examining the ability of interviews to predict resident performance have shown mixed findings, and the role of the interview in candidate evaluation remains unclear. This mixed-methods study, conducted in an anaesthesiology residency programme at a large academic medical centre, examined how interviews contributed to candidate assessment and whether the addition of behavioural questions to interviews altered their role in the evaluation process.

METHODS

During the 2018-2019 residency selection season in the Department of Anesthesiology and Critical Care at the University of Pennsylvania, independent ratings for each interviewee were collected from faculty interviewers. Consensus ratings subsequently established by committee were also collected. Committee meetings were audiorecorded and transcribed for qualitative analysis. Behavioural questions were integrated into half of interview days. Ratings of candidates interviewed on behavioural question days were compared statistically with those of candidates interviewed on non-behavioural question days.

RESULTS

Qualitative analysis showed that interviewers heavily emphasised candidates' application files in evaluating the interviews. Interviewers focused on candidates' academic records and favoured candidates whose interview behaviours were consistent with their applications and whose applications demonstrated similarities to interviewers' traits. The addition of behavioural questions demonstrated little ability to alter these dynamics. Quantitatively, there were no significant differences in candidate rating outcomes between behavioural and non-behavioural interviewing days, whereas a higher medical school rating and higher score on the United States Medical Licensing Examination Step 1 were associated with a more favourable consensus rating.

CONCLUSIONS

Residency candidates' application files predisposed interviewers' experience and evaluation of interviews, preventing the interviews from providing discrete assessments of interpersonal qualities, even when behavioural questions were included. In the continued effort to perform well-rounded assessments of residency candidates, further research and reflection on the role of interviewing in evaluation are necessary.

The AAMC Standardized Video Interview: Lessons Learned From the Residency Selection Process

Calls to change the residency selection process have increased in recent years, with many focusing on the need for holistic review and alternatives to academic metrics. One aspect of applicant performance to consider in holistic review is proficiency in behavioral competencies. The Association of American Medical Colleges (AAMC) developed the AAMC Standardized Video Interview (SVI), an online, asynchronous video interview that assesses applicants' knowledge of professionalism and their interpersonal and communication skills. The AAMC worked with the emergency medicine community to pilot the SVI. Data from 4 years of research (Electronic Residency Application Service [ERAS] 2017-2020 cycles) show the SVI is a reliable, valid assessment of these behavioral competencies. It provides information not available in the ERAS application packet, and it does not disadvantage individuals or groups. Yet despite the SVI's psychometric properties, the AAMC elected not to renew or expand the pilot in residency selection.In this Invited Commentary, the authors share lessons learned from the AAMC SVI project about introducing a new tool for use in residency selection. They recommend that future projects endeavoring to find ways to support holistic review engage all stakeholders from the start; communicate the value of the new tool early and often; make direct comparisons with existing tools; give new tools time and space to succeed; strike a balance between early adopters and broad participation; help stakeholders understand the limitations of what a tool can do; and set clear expectations about both stakeholder input and pricing. They encourage the medical education community to learn from the SVI project and to consider future partnerships with the AAMC or other specialty organizations to develop new tools and approaches that prioritize the community's needs. Finding solutions to the challenges facing residency selection should be a priority for all stakeholders.

Perception of medical students and residents about virtual interviews for residency applications in the United States

INTRODUCTION

Residency applications via virtual-interview could potentially mitigate the extensive cost and time required for customary in-person interviews. We outline the perception of medical students and residents on the use of virtual-interview for residency applications in lieu of in-person interviews.

METHODS

We obtained 1824 responses from medical students and residents through an online questionnaire between March2019-Feb2020 in Texas-United States. The survey had 11 statements (five in favor of in-person interviews and 6 in favor of virtual interviews) that respondents could rank on a 5-point Likert scale. All statements' scores were summed based on the response given by each participant to create a total score between 11 and 55. The perception of the two groups was analyzed using an independent sample T-test and ANOVA.

RESULTS

We received a total of 1711 responses from medical students and 113 from medical residents. Respondents were more female (82.2% of medical students and 47.8% of residents), with a mean age of 22.87±3.42 years old for medical students and 28.72±4.35 years old for residents. Both groups preferred in-person interviews; however, the residents were significantly more in favor (P = 0.03). Both groups agree that virtual-interviews should be as an option, though this was considerably higher in the medical students (P = 0.001). In the multivariate analysis, "travel distance" and "type of medical school" had a significant impact on choosing the virtual-interviews in both groups (p<0.01).

CONCLUSIONS

In-person interviews are favored by both medical students and residents compared to virtual-interview services in normal circumstances. However, both groups agree that programs should offer the option of having virtual-interviews as an available choice. Distance to an interview location and the type of medical school were the factors that had a significant impact on perception of using virtual-interviews. Knowing about the applicants' attitude toward residency interviews and the national circumstances are essential when preparing the interview guides. Our findings are limited by the small sample size and the low response rate. Further extensive studies are warranted to better understand the perception of residency applicants toward virtual-interviews to improve the interview process in the United States.

Postbaccalaureate and graduate applicants to dental school: An analysis of holistic review

Although they are valued for their perceived maturity, resiliency, and diverse insight into dentistry, postbaccalaureate and graduate (PBGR) applicants face significant challenges in the admissions process. This study looks at how PBGR applicants are evaluated during the selection process at a US dental school. An analysis of metrics associated with PBGR applicants was performed, focusing on the demographic makeup, academic performance, and total experience hours compared to traditional applicants. Our results suggest that PBGR applicants who are successful in their postbaccalaureate/graduate course of study are also more likely to be admitted if they have a history of strong undergraduate performance. Our results also suggest that PBGR applicants with high self-disclosed employment hours are strongly considered. Taken together, these results suggest that holistic review has helped PBGR applicants in the admissions process but that further adjustments need to be implemented.

Which Applicant Factors Predict Success in Emergency Medicine Training Programs? A Scoping Review

BACKGROUND

Program directors (PDs) in emergency medicine (EM) receive an abundance of applications for very few residency training spots. It is unclear which selection strategies will yield the most successful residents. Many authors have attempted to determine which items in an applicant's file predict future performance in EM.

OBJECTIVES

The purpose of this scoping review is to examine the breadth of evidence related to the predictive value of selection factors for performance in EM residency.

METHODS

The authors systematically searched four databases and websites for peer-reviewed and gray literature related to EM admissions published between 1992 and February 2019. Two reviewers screened titles and abstracts for articles that met the inclusion criteria, according to the scoping review study protocol. The authors included studies if they specifically examined selection factors and whether those factors predicted performance in EM residency training in the United States.

RESULTS

After screening 23,243 records, the authors selected 60 for full review. From these, the authors selected 15 published manuscripts, one unpublished manuscript, and 11 abstracts for inclusion in the review. These studies examined the United States Medical Licensing Examination (USMLE), Standardized Letters of Evaluation, Medical Student Performance Evaluation, medical school attended, clerkship grades, membership in honor societies, and other less common factors and their association with future EM residency training performance.

CONCLUSIONS

The USMLE was the most common factor studied. It unreliably predicts clinical performance, but more reliably predicts performance on licensing examinations. All other factors were less commonly studied and, similar to the USMLE, yielded mixed results.

Applicant Attitudes Toward the Association of American Medical Colleges' Standardized Video Interview

BACKGROUND

The Standardized Video Interview (SVI) was developed by the American Association of Medical Colleges to allow applicants to include objective data about professional behaviors and interpersonal and communication skills. Although the SVI pilot was administered to individuals applying to emergency medicine (EM) residency programs during the 2018 Electronic Residency Application Service (ERAS) cycle, little data have been published evaluating the applicant's perceptions. This survey aims to assess EM residency applicant attitudes toward the SVI.

METHODS

During the 2018 ERAS application season an anonymous survey was administered to interviewees at one urban Accreditation Council for Graduate Medical Education-approved EM residency. Respondents were asked questions regarding the production of their video interviews, thoughts regarding the additive value of the SVI, and individual demographic data such as ethnicity and sex. Participation was optional.

RESULTS

A total of 219 of 238 candidates completed the survey representing a 92% response rate. While the majority of applicants did not feel that their ethnicity impacted their application, 58.1% of those who did self-identified as African American or Asian. A total of 8.7% of respondents felt the SVI added information about their professional behaviors and 11% felt that it added information about interpersonal and communication skills. Only 2.8% of survey respondents felt the SVI should remain a portion of the ERAS application.

CONCLUSIONS

Most respondents felt that the SVI was not an accurate representation of their interpersonal and communication skills or their professionalism and that it did not add value to their applications. While most cohorts were not concerned about bias regarding sex, ethnicity, sex, or age, a small subset felt that there was a potential for the SVI to bias the party reviewing their applications. Very few applicants felt the SVI should remain a part of the ERAS application. Applicant attitudes toward the SVI are largely negative and require further investigation prior to becoming a standard part of applicants' ERAS files.

The AAMC Standardized Video Interview: Reactions and Use by Residency Programs During the 2018 Application Cycle

PURPOSE

To evaluate how emergency medicine residency programs perceived and used Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI) total scores and videos during the Electronic Residency Application Service 2018 cycle.

METHOD

Study 1 (November 2017) used a program director survey to evaluate user reactions to the SVI following the first year of operational use. Study 2 (January 2018) analyzed program usage of SVI video responses using data collected through the AAMC Program Director's Workstation.

RESULTS

Results from the survey (125/175 programs; 71% response rate) and video usage analysis suggested programs viewed videos out of curiosity and to understand the range of SVI total scores. Programs were more likely to view videos for attendees of U.S. MD-granting medical schools and applicants with higher United States Medical Licensing Examination Step 1 scores, but there were no differences by gender or race/ethnicity. More than half of programs that did not use SVI total scores in their selection processes were unsure of how to incorporate them (36/58; 62%) and wanted additional research on utility (33/58; 57%). More than half of programs indicated being at least somewhat likely to use SVI total scores (55/97; 57%) and videos (52/99; 53%) in the future.

CONCLUSIONS

Program reactions on the utility and ease of use of SVI total scores were mixed. Survey results indicate programs used the SVI cautiously in their selection processes, consistent with AAMC recommendations. Future user surveys will help the AAMC gauge improvements in user acceptance and familiarity with the SVI.

Applicant Reactions to the AAMC Standardized Video Interview During the 2018 Application Cycle

PURPOSE

This study examined applicant reactions to the Association of American Medical Colleges Standardized Video Interview (SVI) during its first year of operational use in emergency medicine (EM) residency program selection to identify strategies to improve applicants' SVI experience and attitudes.

METHOD

Individuals who self-classified as EM applicants applying in the Electronic Residency Application Service 2018 cycle and who completed the SVI in summer 2017 were invited to participate in 2 surveys. Survey 1, which focused on procedural issues, was administered immediately after SVI completion. Survey 2, which focused on applicants' SVI experience, was administered in fall 2017, after SVI scores were released.

RESULTS

The response rates for surveys 1 and 2 were 82.3% (2,906/3,532) and 58.7% (2,074/3,532), respectively. Applicant reactions varied by aspect of the SVI studied and their SVI total scores. Most applicants were satisfied with most procedural aspects of the SVI, but most applicants were not satisfied with the SVI overall or with their total SVI scores. About 20% to 30% of applicants had neutral opinions about most aspects of the SVI. Negative reactions to the SVI were stronger for applicants who scored lower on the SVI.

CONCLUSIONS

Applicants had generally negative reactions to the SVI. Most were skeptical of its ability to assess the target competencies and its potential to add value to the selection process. Applicant acceptance and appreciation of the SVI will be critical to the SVI's acceptance by the graduate medical education community.

The AAMC Standardized Video Interview and the Electronic Standardized Letter of Evaluation in Emergency Medicine: A Comparison of Performance Characteristics

PURPOSE

To compare the performance characteristics of the electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI), a new tool designed by the AAMC to assess interpersonal and communication skills and professionalism knowledge.

METHOD

The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships.

RESULTS

The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = -0.21) and US-MD applicants (d = 0.23-0.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = -0.20) and white applicants over black applicants (approaching d = 0.40).

CONCLUSIONS

Small positive correlations between SVI score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance.