Clinical experience can compensate for inferior academic achievements in an undergraduate objective structured clinical examination

BACKGROUND

Practical and non-cognitive skills are essential to medical professions; yet, success in medical studies is primarily assessed with cognitive criteria. We show that practical exams can benefit students who have only average high school final grades, but working experience in medical professions.

METHODS

With a cross-sectional study, we compare the performance of undergraduate medical students with working experience in adjacent health-care professions (and below-average school leaving-grades) with students who entered medical school directly based on their excellent school records in an Objective Structured Clinical Examination (OSCE). For a sample of more than 1,200 students, we use information on OSCE scores in medical and practical skills, doctor-patient communication/interaction, performance in MC-exams, and core sociodemographic variables.

RESULTS

Waiting list students outperformed their classmates in the demonstration of practical skills. Students admitted via their excellent school grades scored best overall. This difference vanishes once we control for school-leaving grade and age, the two main factors separating the analysed groups. Students from the waiting list have a significantly smaller overall chance to reach excellent grades in the first two years of study.

CONCLUSIONS

Students who gathered experiences in health-care professions before enrolling at medical school can benefit from an expanded role of practical elements in medical studies. Student selection instruments should take these different starting positions and qualities of applicants into account, for example with a quota for the professionally experienced.

Can selection interviews predict OSCE performance? Evidence from Hannover Medical School

OBJECTIVES

We analyze whether the student selection process at Hannover Medical School (MHH), which combined a semi-structured interview with school leaving grades, can predict performances in an Objective Structured Clinical Examination (OSCE). We also check whether there are differences between assessments of clinical knowledge, practical skills, and communication abilities.

METHODS

We use data from 525 medical students who were admitted after a successful selection process and who completed the OSCE in the years 2015-2019. We employ multivariate regressions and a mediation analysis approach to learn whether study success after admission and prior to the OSCE mediates the outcome of the latter.

RESULTS

A better performance in the MHH's selection interview is unrelated to success in the OSCE. However, there is a small but significant influence of school grades on OSCE results in each part except for the assessment of communication skills. The impact of the school grade is partially mediated by performances in written and oral exams preceding the OSCE.

DISCUSSION

School grades matter for the OSCE outcome, albeit to different degrees for more learning-based vs. practical parts of the examination. The interview at MHH was purely informative and unrelated to study success, also in the assessment of communication skills. Better structured interview tools may yield better results.

CONCLUSION

Students' cognitive abilities predict study success in an undergraduate OSCE. Performances in a semi-structured selection interview have no impact, not even the assessment of communication skills.

Student selection in radiography education. A narrative review

OBJECTIVES

Due to effects on study success, radiography student selection has a major impact on higher education institutions and applicants. However, there is very little research to demonstrate which selection methods and contents are most successful in radiography education. This study aimed to describe the methods and contents used in radiography student selection and factors related to study success.

KEY FINDINGS

A narrative review was undertaken. A computerized search in four databases limited to studies published between January 2000 and June 2021. Ten quantitative, mainly retrospective, studies were included. The review identified 23 selection methods; of these, interview (n = 4), Scholastic Aptitude Test (n = 3), American College Test (n = 2) and reference letter (n = 2) were used more than once in radiography student selection. The content of the selection methods was identified in four categories including 44 factors. The most often assessed content was category of learning skills while the least often assessed concerned categories of social skills, personality traits and career choice. Regarding study success, factors of learning skills, namely mathematics, physics, biology, anatomy, physiology, natural sciences, a composite of factors comprising electronics and a composite of factors comprising mechanics predicted study success. Factors of social skills, personality traits and career choice were not related to study success.

CONCLUSION

The methods used and contents assessed vary greatly in radiography student selection. The results suggest using the content in the four categories in the selection of radiography students.

IMPLICATIONS FOR PRACTICE

Further research is needed to clarify the methods, with knowledge of the reliability and validity and the contents for the suggested categories, and to demonstrate their relationship to study success and identify the core content of radiography student selection especially in European context.

[Selection interviews at Hanover Medical School (MHH): determinants of student selection in medical studies]

INTRODUCTION

In order to incorporate social and communicative skills in its student admissions process, Hanover Medical School (MHH) has conducted selection interviews (in combination with the high-school GPA) to choose 60 % of its freshmen in medical studies. The present article analyses if applicants' performances in the interviews were the determining criterion of student selection, despite a higher weighting of school grades in the admission process. Furthermore, this article checks whether the grading of the interviews was independent of the applicants' gender, age, origin and educational background.

METHODS

For a sample of more than 3,000 successful and unsuccessful participants in the MHH student admission process in the years 2010-2017, we employ variance analysis and logistic regression analysis to determine those factors that have contributed to the chances of being offered a place at the MHH after a successful interview.

RESULTS

The scores received in the selection interview were the sole determinant of being offered a place at the MHH; neither the applicants'age nor their gender or origin biased the decisions of the selection committees. The grading of the interview was also not affected by school GPAs.

DISCUSSION

The selection interviews at the MHH have been costly in terms of both financial and human resources. However, this selection method has been popular among students and lecturers alike, not least because its elements fit the MHH's focus on early bedside teaching and social skills of prospective physicians. However, there is no evidence that selection interviews are effective in predicting academic success.

CONCLUSIONS

The present article has shown that the selection interviews acted up to the principles defined by the MHH: very homogenous high school GPAs were complemented by differentiated interview assessments that did not discriminate by sociodemographic characteristics. It is, however, unclear if the MHH will resume the interviews after the end of a federally mandated halt to stand-alone selection methods at medical schools in Germany.

Nursing applicants' reasoning skills and factors related to them: A cross-sectional study

BACKGROUND

The assessment of reasoning skills is recommended in undergraduate nursing student selection. Reasoning skills are crucial for sound decision-making, improving patient safety and are necessary from the very beginning of studies. Nursing applicants' reasoning skills based on the reasoning process have not been previously measured.

OBJECTIVES

To assess undergraduate nursing applicants' reasoning skills and factors related to them.

DESIGN

A cross-sectional study.

SETTING AND PARTICIPANTS

Undergraduate nursing applicants (n = 1056, response rate 55.4%), who consented to the study and performed a joint electronic entrance examination to six Finnish Universities of Applied Sciences in spring 2019, participated in the study.

METHODS

The Reasoning Skills (ReSki) test, based on the steps of the reasoning process, was used, comprising three question sections (collecting information, processing information, and identifying the problem and establishing goals). Background variables were collected through a questionnaire and the Positive System Usability Scale (P-SUS). The data were analysed with descriptive statistics, Pearson correlation coefficients and analysis of covariance with Tukey's test in post-hoc multiple group comparisons.

RESULTS

Applicants' total reasoning skills mean scores were above the centre of the range of possible scores (2.72/4.5, SD = 0.80). The applicants scored higher in collecting and processing information than in identifying the problem and establishing goals. Standard deviations demonstrated variance between the applicants' ability. Age, gender, and previous education were statistically significantly related to applicants' reasoning skills. Previous work experience was statistically significantly related to success only in the step of identifying the problem and establishing goals.

CONCLUSIONS

Nursing applicants' reasoning skills vary in the student selection phase. Applicants are less able to identify the problem and establish goals than to collect and process information. Vocational education does not necessarily develop adequate reasoning skills and thus prepare students for higher education studies. The results have implications for educational institutions and further research.

Benefits and Feasibility of Using Videos to Assess Medical School Applicants' Empathetic Abilities in Multiple Mini Interviews

PURPOSE

We sought to evaluate the feasibility and benefits of using video-based scenarios in Multiple Mini Interviews (MMIs) to assess candidate's empathic abilities by investigating candidate perceptions and the acceptability, fairness, reliability, and validity of the test.

METHODS

The study sample was candidates for admission interviews held in the MMI format at a medical school in South Korea. In this six-station MMI, one station included a 2-min video clip of a patient-doctor communication scenario to assess candidate emphatic abilities, whereas paper-based scenarios were used in the other stations. Candidate's perceptions and acceptability of using the video-based scenario in the empathy station were examined using a 41-item post-MMI questionnaire. Fairness of the test was assessed by means of differences in candidate perceptions and performance across different demographics or backgrounds. Construct validity was assessed by examining the relationship of candidate performances in the empathy station with those in other stations. The G-coefficient was analyzed to estimate the reliability of the test.

RESULTS

Eighty-two questionnaires were returned, a 97.6% response rate. Candidates showed overall positive perceptions of the video-based scenario and they found it authentic and interesting. The test was fair as there were no differences in candidates' perceptions of the patient-doctor relationship presented in the video clip and neither in their performance nor in their perceived difficulty of the station across demographics or backgrounds. Construct validity was established as candidate performance in the empathy station was not associated with that of any other stations. The G-coefficient was 0.74.

CONCLUSIONS

The present study demonstrates that the video-based scenario is a feasible tool to assess candidate's empathy in the MMI.

Adaptation of internet-based multiple mini-interviews in a limited-resource medical school during the coronavirus disease 2019 pandemic

The coronavirus disease 2019 pandemic has affected many aspects of life, including medical education. The selection of students using multiple mini-interviews (MMI) should be adapted promptly as internet-based MMI (iMMI) in the current setting. The current transition from MMI to iMMI is supported by experience in conducting conventional MMI in previous years; hence, the station and assessor preparation could be completed without significant issues. The greater challenges to the use of iMMI are due to technical issues, including poor internet connections and adaptations necessary for assessing candidates' nonverbal expressions. This paper aims to describe the experience of conducting iMMI in the current pandemic situation, with further highlights on contextual challenges in a limited-resource setting.

Admission policies and methods at crossroads: a review of medical school admission policies and methods in seven Asian countries

Selecting the right applicants is an important part of medical student admission. While one universally accepted selection criterion is academic capacity, there are other criteria such as communication skills and local criteria (e.g., socio-cultural values) that are no less important. This article reviews the policies and methods of selection to medical schools in seven countries with varying socio-economic conditions and healthcare systems. Senior academics involved in medical education in Indonesia, Japan, Malaysia, the Philippines, Singapore, Sri Lanka, and Taiwan completed a pre-agreed pro-forma per each country to describe the country's admission policies and methods. The details were then compared and contrasted. This review identifies tension between many of the policies and methods used in medical school admissions, such as between the need to assess non-cognitive abilities and widen access, and between the need for more medical professionals and the requirement to set high entry standards. Finding the right balance requires careful consideration of all variables, including the country's human resource needs; socio-economic status; graduates' expected competencies; and the school's vision, mission, and availability of resources.

Medical student selection criteria and junior doctor workplace performance

BACKGROUND

Medical school selection decisions have consequences beyond graduation. With generally low attrition rates, most medical students become junior doctors. Universities are therefore not just selecting students into a medical course; they are choosing the future medical workforce. Understanding the relationship between selection criteria and outcomes beyond the successful completion of a medical degree may inform approaches to student selection.

METHODS

A retrospective data matching study was conducted involving 39 interns employed by a South Australian local health network in 2017 who had originally entered Flinders University's medical school through a graduate pathway. Student selection data were matched with internship workplace performance scores (measured by supervising consultants' reports across five clinical rotations using a standardised assessment). Correlational analyses then examined associations between these two sets of variables.

RESULTS

An overall selection rank (equal thirds of weighted Grade Point Average from a prior degree, a panel interview, and a national selection test) was moderately associated with all performance measures, accounting for up to 25% of variance. Both weighted Grade Point Average and the interview had multiple and mostly moderate correlations with performance. An increasing number of years taken to complete the course was associated with poorer workplace performance across multiple outcome measures (moderate to strong negative associations with 31 to 62% of shared variance), as was age to a lesser extent (7 to 14%). The national selection test contributed a single and small relationship accounting for 5% of variance with one outcome measure.

CONCLUSIONS

Selection into medicine is a critical assessment given that most students become doctors. This study found multiple associations between selection scores and junior doctor workplace performance measures in the internship year, with weighted Grade Point Average from a prior degree and an interview appearing more important than the national selection test. Future collaborative research should map desired workplace performance outcomes to initial student selection and explore the impact of changes to selection which focus on assessment of these domains. The association between slower course progression and poorer workplace performance should also be examined.

Predictive validity of a Uniform Entrance Test for the health professionals

Objective

To investigate the predictive validity of Uniform Entrance Test for academic performance in the first two years in various health science degree programs.

Methods

A retrospective analysis of admissions data and academic performance of students admitted in under-graduate programs of medicine, dentistry and pharmacy of three cohorts was taken. The independent and dependent variables were entry test scores and semester scores respectively. Spearman's Correlation co-efficient was computed to determine the association between entrance test scores and semester scores for three groups.

Results

Majority of the students were from the MBBS degree program (61%) with majority of female students (65%) in all three programs. In MBBS the highest correlation coefficient between entry test and semester scores was observed for semester one rs = 0.334 and lowest in semester four rs= 0.208. In BDS degree program both highest and lowest correlations were in semester one. In the Pharm-D degree program, a significant correlation was only seen in cohort 1 but not in the subsequent cohorts.

Conclusion

The uniform entrance test has an incremental predictive validity for the MBBS and BDS programs as compared to Pharm-D. Better performance in the entrance test predicts higher semester scores and more likelihood of achieving higher scores in the first year as compared to the second year.

[Selection of medical students : Measurement of cognitive abilities and psychosocial competencies]

The German Constitutional Court is currently reviewing whether the actual study admission process in medicine is compatible with the constitutional right of freedom of profession, since applicants without an excellent GPA usually have to wait for seven years. If the admission system is changed, politicians would like to increase the influence of psychosocial criteria on selection as specified by the Masterplan Medizinstudium 2020.What experiences have been made with the actual selection procedures? How could Situational Judgement Tests contribute to the validity of future selection procedures to German medical schools?High school GPA is the best predictor of study performance, but is more and more under discussion due to the lack of comparability between states and schools and the growing number of applicants with top grades. Aptitude and knowledge tests, especially in the natural sciences, show incremental validity in predicting study performance. The measurement of psychosocial competencies with traditional interviews shows rather low reliability and validity. The more reliable multiple mini-interviews are superior in predicting practical study performance. Situational judgement tests (SJTs) used abroad are regarded as reliable and valid; the correlation of a German SJT piloted in Hamburg with the multiple mini-interview is cautiously encouraging.A model proposed by the Medizinischer Fakultätentag and the Bundesvertretung der Medizinstudierenden considers these results. Student selection is proposed to be based on a combination of high school GPA (40%) and a cognitive test (40%) as well as an SJT (10%) and job experience (10%). Furthermore, the faculties still have the option to carry out specific selection procedures.

The impact of the program structure at Hannover Medical School on academic success in medical studies

Aim: The classical course structure for medicine in Germany is separated by three sections of the medical state examination. This structure is generally regarded as sensible and unchangeable. Because the special program structure at Hannover Medical School (MHH) has one integrated, rather than two separate study blocks, it is possible to examine the influence of structural modifications on the study success of different admission groups. Methods: The data was obtained from students admitted to the MHH between 2006 and 2008 in different admission quotas. Study success was defined as the successful completion of the entire program, but completion of the first section of the state examination was also analysed. Results: More students from the best "Abitur" (school leaving examinations) quota successfully completed their studies than those accepted via the selection process of the universities. The latter were more successful than students from the waiting list quota. However the successful graduates of this last group completed their studies more often within the prescribed period of study, although they needed more time for completing all parts of the first section of the state examination. Conclusion: The data shows that an integrated course structure can offer, in particular, students from the waiting list quota, the opportunity to compensate for delays in the first years of study. However, they do not provide any evidence which applicants are best suited to meet the social and professional requirements of trained doctors. Due to the complex structure of such longitudinal studies, our results allow more than one plausible interpretation.

Correcting the predictive validity of a selection test for the effect of indirect range restriction

BACKGROUND

The validity of selection tests is underestimated if it is determined by simply calculating the predictor-outcome correlation found in the admitted group. This correlation is usually attenuated by two factors: (1) the combination of selection variables which can compensate for each other and (2) range restriction in predictor and outcome due to the absence of outcome measures for rejected applicants.

METHODS

Here we demonstrate the logic of these artifacts in a situation typical for student selection tests and compare four different methods for their correction: two formulas for the correction of direct and indirect range restriction, expectation maximization algorithm (EM) and multiple imputation by chained equations (MICE). First we show with simulated data how a realistic estimation of predictive validity could be achieved; second we apply the same methods to empirical data from one medical school.

RESULTS

The results of the four methods are very similar except for the direct range restriction formula which underestimated validity.

CONCLUSION

For practical purposes Thorndike's case C formula is a relatively straightforward solution to the range restriction problem, provided distributional assumptions are met. With EM and MICE more precision is obtained when distributional requirements are not met, but access to a sophisticated statistical package such as R is needed. The use of true score correlation has its own problems and does not seem to provide a better correction than other methods.

Student nurse selection and predictability of academic success: The Multiple Mini Interview project

BACKGROUND

With recent reports of public enquiries into failure to care, universities are under pressure to ensure that candidates selected for undergraduate nursing programmes demonstrate academic potential as well as characteristics and values such as compassion, empathy and integrity. The Multiple Mini Interview (MMI) was used in one university as a way of ensuring that candidates had the appropriate numeracy and literacy skills as well as a range of communication, empathy, decision-making and problem-solving skills as well as ethical insights and integrity, initiative and team-work.

OBJECTIVES

To ascertain whether there is evidence of bias in MMIs (gender, age, nationality and location of secondary education) and to determine the extent to which the MMI is predictive of academic success in nursing.

DESIGN

A longitudinal retrospective analysis of student demographics, MMI data and the assessment marks for years 1, 2 and 3.

SETTINGS

One university in southwest London.

PARTICIPANTS

One cohort of students who commenced their programme in September 2011, including students in all four fields of nursing (adult, child, mental health and learning disability).

METHODS

Inferential statistics and a Bayesian Multilevel Model.

RESULTS

MMI in conjunction with MMI numeracy test and MMI literacy test shows little or no bias in terms of ages, gender, nationality or location of secondary school education. Although MMI in conjunction with numeracy and literacy testing is predictive of academic success, it is only weakly predictive.

CONCLUSIONS

The MMI used in conjunction with literacy and numeracy testing appears to be a successful technique for selecting candidates for nursing. However, other selection methods such as psychological profiling or testing of emotional intelligence may add to the extent to which selection methods are predictive of academic success on nursing.

Relationship between student selection criteria and learner success for medical dosimetry students

Medical dosimetry education occupies a specialized branch of allied health higher education. Noted international shortages of health care workers, reduced university funding, limitations on faculty staffing, trends in learner attrition, and increased enrollment of nontraditional students force medical dosimetry educational leadership to reevaluate current admission practices. Program officials wish to select medical dosimetry students with the best chances of successful graduation. The purpose of the quantitative ex post facto correlation study was to investigate the relationship between applicant characteristics (cumulative undergraduate grade point average (GPA), science grade point average (SGPA), prior experience as a radiation therapist, and previous academic degrees) and the successful completion of a medical dosimetry program, as measured by graduation. A key finding from the quantitative study was the statistically significant positive correlation between a student׳s previous degree and his or her successful graduation from the medical dosimetry program. Future research investigations could include a larger research sample, representative of more medical dosimetry student populations, and additional studies concerning the relationship of previous work as a radiation therapist and the effect on success as a medical dosimetry student. Based on the quantitative correlation analysis, medical dosimetry leadership on admissions committees could revise student selection rubrics to place less emphasis on an applicant׳s undergraduate cumulative GPA and increase the weight assigned to previous degrees.