Purposeful assessment

A Comparative Case Study Analysis of Cultural Competence Training at 15 U.S. Medical Schools

PURPOSE

Twenty years have passed since the Liaison Committee on Medical Education (LCME) mandated cultural competence training at U.S. medical schools. There remain multiple challenges to implementation of this training, including curricular constraints, varying interpretations of cultural competence, and evidence supporting the efficacy of such training. This study explored how medical schools have worked to implement cultural competence training.

METHOD

Fifteen regionally diverse public and private U.S. medical schools participated in the study. In 2012-2014, the authors conducted 125 interviews with 52 administrators, 51 faculty or staff members, and 22 third- and fourth-year medical students, along with 29 focus groups with an additional 196 medical students. Interviews were recorded, transcribed, and imported into NVivo 10 software for qualitative data analysis. Queries captured topics related to students' preparedness to work with diverse patients, engagement with sociocultural issues, and general perception of preclinical and clinical curricula.

RESULTS

Three thematic areas emerged regarding cultural competence training: formal curriculum, conditions of teaching, and institutional commitment. At the formal curricular level, schools offered a range of courses collectively emphasizing communication skills, patient-centered care, and community-based projects. Conditions of teaching emphasized integration of cultural competence into the preclinical years and reflection on the delivery of content. At the institutional level, commitment to institutional diversity, development of programs, and degree of prioritization of cultural competence varied.

CONCLUSIONS

There is variation in how medical schools approach cultural competence. Among the 15 participating schools, longitudinal and experiential learning emerged as important, highlighting the needs beyond mere integration of cultural competence content into the formal curriculum. To determine efficacy of cultural competence programming, it is critical to conduct systematic assessment to identify and address gaps. While LCME standards have transformed aspects of medical education, further research is needed to clarify evidence-based, effective approaches to this training.

Development of a comprehensive clinical performance assessment system for nursing students: A programmatic approach

AIM

Evaluation of achievement of learning objectives needs an accurate assessment program. Hence, nursing educators should move away from the use of individual assessment methods to apply a programmatic approach. The aim of this study was to develop a comprehensive assessment system for nursing students in their critical care rotation based on a programmatic approach.

METHODS

The population of this study was nursing students in their critical care course. The learning objectives of the course were determined using an expert panel and classified into three categories. Suitable assessment methods were identified for each category according to the consensus of experts. Then, the assessment tools were designed and the content validity was established using content validity ratio (CVR) and index (CVI). The reliability was determined by Cronbach's alpha coefficient. The satisfaction of the participants was investigated using a questionnaire.

RESULTS

According to the findings, all items of the assessment system had a high CVR (P < 0.05) and CVI ranged 0.93-0.97. The alpha coefficient of the whole system was more than 0.90 and for subsystems ranged 0.72-0.96. The findings showed that 87.5% of the instructors and 89.47% of students believed that the new assessment system had a positive impact on learning. In addition, the majority of them were satisfied with the new assessment system.

CONCLUSION

A programmatic approach should be used for effective evaluation of clinical performance of nursing students in critical care settings because of high validity and reliability, multidimensionality, positive educational impact, and acceptability.

The rationale for and use of assessment frameworks: improving assessment and reporting quality in medical education

An assessment framework provides a structured conceptual map of the learning outcomes of a programme of study along with details of how achievement of the outcomes can be measured. The rationale for using frameworks to underpin the targeting of essential content components is especially relevant for the medical education community. Frameworks have the capacity to improve validity and reliability in assessment, allowing test developers to more easily create robust assessment instruments. The framework used by the Australian Medical Assessment Collaboration (AMAC) is an interesting and relevant case study for the international community as it draws and builds on established processes in higher education assessment. The AMAC experience offers an insight into important considerations for designing assessment frameworks and implementing frameworks in differing contexts. There are lessons which have the potential to improve assessment and reporting practice and quality in not only medical education, but in other domains of assessment. Prior to implementing any programme of assessment, the framework considerations outlined here will hopefully improve the quality of assessment and reporting practice by making implicit assumptions explicit, and allowing more critical reflection and evaluation throughout assessment processes.