Reliability and acceptability of six station multiple mini-interviews: past-behavioural versus situational questions in postgraduate medical admission

Development, Validity and Reliability of Objective Structured Clinical Examination in Nursing Students

Introduction

The adoption of measurement instruments such as the Objective Structured Clinical Examination (OSCE) is essential to assess clinical competencies in nursing students.

Objective

The purpose of this study is to develop an OSCE, analyze its validity and reliability in the nursing curriculum and measure self-assessment, stress and satisfaction.

Methods

The observational validation study of a measurement instrument was carried out in two phases: the design and development of the OSCE and validity and reliability analysis.

Results

A total of 118 students participated in the study. Ten scenarios were designed that incorporated six competency components extracted from the curriculum. Good results were obtained in face validity, content validity (CVI .82-.95), criterion validity (r = .71, p < .001), and reliability (α Cronbach .84). Satisfaction and stress scores were high, and self-assessment scores were lower than the scores obtained.

Conclusion

A rigorously designed OSCE provides a reliable and valid method for assessing the clinical competence of nursing students.

Beyond a Hashtag: Strategies to Move Toward a More Inclusive Physiatry Workforce

ABSTRACT

The field of physical medicine and rehabilitation should strive for a physician workforce that is ethnically/racially, sex, and ability diverse. Considering the recent realities of disparities in health outcomes related to COVID-19 and in racial injustice in the United States, we are called to be champions for antiracism and equity. The specialty of physical medicine and rehabilitation should be the leaders in fostering a culture of inclusion and pay special attention to the population of applicants who are underrepresented in medicine. The specialty needs tools to start addressing these disparities. This article aims to provide strategic and intentional evidence-based recommendations for programs to follow. Holistic review, implicit bias training, structured interviews, and targeted outreach for those underrepresented in medicine are some of the tools that will help students enter and become successful in our specialty. Furthermore, this article provides novel guidance and considerations for virtual interviews during the COVID-19 pandemic.

The objective structured clinical examination as an assessment strategy for clinical competence in novice nursing practitioners in Taiwan

Background

The conventional written tests and professional assessment have limitation in fair judgement of clinical competence. Because the examiners may not have total objectivity and may lack standardization throughout the assessment process. We sought to design a valid method of competence assessment in medical and nursing specialties. This work was aimed to develop an Objective Structured Clinical Exam (OSCE) to evaluate novice nursing practitioners’ clinical competency, work stress, professional confidence, and career satisfaction.

Methods

A Quasi-experimental study (pre-post). Fifty-five novice nursing practitioners received the OSCE three-months following their graduation, which consisted of four stations: history taking, physical examination, problem-directed management, interpersonal communication, and the required techniques of related procedures. The examiners had to complete an assessment checklist, and the participants had to complete a pre-post questionnaire (modified from a Nursing Competency Questionnaire, a Stress scale, and Satisfaction with Learning scale).

Results

Among the novice nursing practitioners, 41 of them (74.5 %) passed the exam with a mean score of 61.38 ± 8.34. There was a significantly higher passing rate among nurses who were working in medical-surgical wards (85.7 %) and the intensive care unit-emergency department (77.8 %) compared to novice nursing practitioners working in other units. All the novice nursing practitioners at Station A had poor performance in assessing patients with a fever. OSCE performance was more associated with educational attainment and work unit, rather than the gender. Finally, the participants showed statistically significant increases in their clinical competency, confidence in their professional competence, satisfaction with the clinical practice, and decreased work stress after the OSCE.

Conclusions

We found that the OSCE process had a positive educational effect, in providing a meaningful and accurate assessment of the competence of novice nursing practitioners. An appropriate OSCE program is vital for novice nursing practitioners, educators, and administrators. The effective application of OSCEs can help novice nursing practitioners gain confidence in their clinical skills.

Rethinking the Admissions Interview: Piloting Multiple Mini-Interviews in a Graduate Psychology Program

Health profession programs routinely utilize traditional interviews in admissions as a means of assessing important non-academic characteristics (e.g., critical thinking, interpersonal skills, judgment) of candidates. However, the reliability and validity of traditional interviews is highly questionable. Given this, multiple health profession programs (e.g., medicine, nursing, pharmacy, physical therapy) have implemented multiple mini-interviews as an alternative for assessing non-academic characteristics. This paper describes the development and implementation of multiple mini-interviews in the admissions process for a doctoral clinical psychology program, one of the health professions yet to use multiple mini-interviews. This paper also examines the feasibility and acceptability of the multiple mini-interviews in this program. Results of a mixed-method survey of all 120 candidates who participated in admissions days are presented along with discussion of factors associated with satisfaction and dissatisfaction. Recommendations for program refinement and application to other graduate psychology programs for improved admissions processes are discussed.

Multiple Mini-Interviews: Current Perspectives on Utility and Limitations

The growing role of healthcare professionals urged admissions committees to restructure their selection process and assess key personal attributes rather than academic achievements only. Multiple mini interviews (MMIs) were designed in 2002 to assess such domains in prospective healthcare professions. Being a high-stake assessment, the utility and limitations of MMI need to be explored. The purpose of this article is to review the available evidence to establish its utility. The claim of the reliability is verified by the studies assessing the effect of number of stations, duration of stations, format and scoring systems of stations and number of raters assessing the applicants. Similarly, by gathering evidence concerning its content validity, convergent/divergent correlation and predictive ability, validity is ensured. Finally, its acceptability and feasibility along with limitations is discussed. This article concludes by providing recommendations for further work required to deal with the limitations and enhance its utility.

Using the Multiple Mini Interview for selection into vocational general practice training

INTRODUCTION Interviews for selection into post graduate training courses are an accepted method of selection. There is the presumption that such interview processes are fair to both candidates and the training scheme. AIM Due to concerns over unconscious bias and a desire to move to best practice, the Royal New Zealand College of General Practitioners introduced the Mini Multiple Interview (MMI) process as the selection process for doctors wishing to enter vocational training in general practice. METHODS Aspects assessed during the interviews were developed through wide consultation and included: reason for wanting to undertake training, managing poor compliance, addressing issues of equity, managing complaints, insight and understanding the role of general practitioners in chronic care. There were 218 applicants who took the MMI. Demographic data as well as scores were collected. RESULTS The MMI process has good reliability and performs well in several aspects of validity. All three interview venues had similar results. There was no gender difference in overall result or scores. New Zealand graduates scored higher than overseas graduates. Of the 218 candidates, 12 were considered not yet ready to enter training. DISCUSSION The MMI process appears to have acceptable reliability and good validity. The structure of the MMI is likely to have reduced unconscious bias. Further research will study the predictive validity of the MMI for this cohort of candidates.

Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

OBJECTIVES

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017.

DATA SOURCES

Twelve electronic bibliographic databases.

REVIEW METHODS

Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique.

RESULTS

A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies.

CONCLUSIONS

Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures.