Psychometric evaluation of a cultural competency assessment instrument for health professionals

An innovative gamification tool to enhance intercultural competence and self-efficacy among healthcare professionals caring for vulnerable migrants and refugees

BACKGROUND

The growing number of vulnerable migrants and refugees (VMRs) in the European Union presents challenges to healthcare systems, emphasizing the need for enhanced intercultural competence training for healthcare professionals. Educational escape rooms, using gamification-based principles, may offer an innovative solution to improve these competencies.

OBJECTIVE

This pilot study evaluates the acceptability and preliminary effectiveness of an educational escape room aimed at improving intercultural competence, self-efficacy, and knowledge among healthcare students and professionals caring for VMRs.

METHODS

A pre-post, single-group pilot study was conducted with 101 healthcare students and professionals, recruited through convenience sampling. Participants engaged in an educational escape room simulating a migratory crisis, designed to foster collaborative problem-solving under pressure. A newly validated questionnaire was administered before and after the intervention to measure changes in intercultural competence, self-efficacy, and knowledge. Paired t-tests were used to analyze pre-post differences, and thematic analysis explored participant feedback on the learning experience and the acceptability of the intervention.

RESULTS

Significant improvements were observed in intercultural competence (d = 1.13, p < 0.001), self-efficacy (d = 0.38, p = 0.001), and knowledge (d = 1.19, p < 0.001). Participants reported high engagement, satisfaction, and an enhanced understanding of healthcare challenges related to VMRs. The escape room was deemed acceptable.

CONCLUSIONS

This pilot study provides evidence of the acceptability and effectiveness of an educational escape room in enhancing intercultural competence, self-efficacy, and knowledge. Further research with larger, more rigorous studies is recommended to confirm these findings and explore scalability.

Measuring attitudes towards ethnic minority patients: the revalidated REMP-3 instrument for graduate healthcare practitioners

OBJECTIVE

Measuring intercultural attitudes can aid in understanding and addressing persistent inequities in healthcare. Instead of creating new instruments, several sources call for a more rigorous revalidation of existing instruments towards a more broad population. As an example of such an existing instrument, the EMP-3 (Ethnic Minority Patients) focuses on the attitudes of physicians towards ethnic minority patients. Starting from a robust theoretical underpinning and a rigorous methodological setup, the present study revalidates the EMP-3 instrument for physicians towards the REMP-3 instrument for graduate healthcare practitioners.

METHODS

We assessed the reliability and validity of the old EMP-3, which we then updated to a new REMP-3 instrument. We used structural equation modeling to model the framework of intercultural effectiveness on two waves of independent data, N2021 = 368 and N2022 = 390. Within this framework, we tested the new REMP-3 instrument as an operationalization of intercultural attitudes. We conducted a confirmatory factor analysis on the first wave, after which we made adaptations to the original EMP-3 instrument to obtain a new REMP-3 instrument. The new REMP-3 instrument was then cross-validated using the data of the second wave.

RESULTS

The new REMP-3 instrument is a psychometric upgrade compared to the EMP-3. The REMP-3 now has a cross-validated structure, with three subscale dimensions (i.e., task perception, background perception and the perceived need to communicate) and an overarching higher-order, full-scale dimension. Both the subscales as well as the full instrument show acceptable to good internal consistency reliability, with a reduced number of items from eighteen to ten. As theoretically predicted, the REMP-3 also functions as a measure of intercultural attitudes in an intercultural competence framework.

CONCLUSION

Ultimately, the REMP-3 instrument can contribute to more equity in healthcare by concisely and reliably assessing and monitoring attitudes in healthcare practitioners. This attitude assessment represents the potential of learning new skills and knowledge to address interactions with ethnic minority patients, which is especially useful during training situations like an internship.

A Randomized Crossover Trial Evaluating the Impact of Cultural Dexterity Training on Surgical Residents' Knowledge, Cross-Cultural Care, Skills, and Beliefs: The Provider Awareness and Cultural Dexterity Training for Surgeons (PACTS) Trial

OBJECTIVES

This trial examines the impact of the Provider Awareness and Cultural dexterity Toolkit for Surgeons (PACTS) curriculum on surgical residents' knowledge, cross-cultural care, skills, and beliefs.

BACKGROUND

Cross-cultural training of providers may reduce health care outcome disparities, but its effectiveness in surgical trainees is unknown.

METHODS

PACTS focuses on developing skills needed for building trust, working with patients with limited English proficiency, optimizing informed consent, and managing pain. The PACTS trial was a randomized crossover trial of 8 academic general surgery programs in the United States: The Early group ("Early") received PACTS between periods 1 and 2, while the Delayed group ("Delayed") received PACTS between periods 2 and 3. Residents were assessed preintervention and postintervention on Knowledge, Cross-Cultural Care, Self-Assessed Skills, and Beliefs. χ 2 and Fisher exact tests were conducted to evaluate within-intervention and between-intervention group differences.

RESULTS

Of 406 residents enrolled, 315 were exposed to the complete PACTS curriculum. Early residents' Cross-Cultural Care (79.6%-88.2%, P <0.0001), Self-Assessed Skills (74.5%--85.0%, P <0.0001), and Beliefs (89.6%-92.4%, P =0.0028) improved after PACTS; knowledge scores (71.3%-74.3%, P =0.0661) were unchanged. Delayed resident scores pre-PACTS to post-PACTS showed minimal improvements in all domains. When comparing the 2 groups in period 2, Early residents had modest improvement in all 4 assessment areas, with a statistically significant increase in Beliefs (92.4% vs 89.9%, P =0.0199).

CONCLUSIONS

The PACTS curriculum is a comprehensive tool that improved surgical residents' knowledge, preparedness, skills, and beliefs, which will help with caring for diverse patient populations.

A Systematic Review of Assessment Tools Measuring Cultural Competence Outcomes Relevant to Pharmacy Education

Objective. To identify and describe validated assessment tools that measure cultural competence and are relevant to pharmacy education.Methods. A systematic approach was used to identify quantitative cultural competence assessment tools relevant to pharmacy education. A systematic search of the literature was conducted using the OVID and EBSCO databases and a manual search of journals deemed likely to include tools relevant to pharmacy education. To be eligible for the review, the tools had to be developed using a study sample from the United States, have at least one peer-reviewed validated publication, be applicable to the pharmacy profession, and be published since 2010.Results. The systematic literature and manual search identified 27 tools. Twelve assessment tools met the criteria to be included in the summary and their relevancy to pharmacy education is discussed.Conclusion. A review of literature demonstrates that assessment tools vary widely and there is no one tool that can effectively assess all aspects of cultural competence in pharmacy students or the Doctor of Pharmacy curriculum. As cultural competence is a priority within the accreditation standards for pharmacy education, PharmD programs are encouraged to develop additional tools that measure observed performance.

Implicit Bias and Caring for Diverse Populations: Pediatric Trainee Attitudes and Gaps in Training

The objective of this study was to determine the attitudes, skill level, and preferred educational interventions of pediatric residents related to implicit bias and caring for diverse patient populations. A cross-sectional survey of pediatric residents at a single, large urban residency program was utilized. Surveys were completed by 88 (55%) residents who were 69% female and 35% non-White or mixed race. Almost all residents felt that it was very or extremely important to receive training on health disparities, diverse patient populations, and implicit bias. Self-assessment of skill level revealed that residents felt confident in areas often covered by cultural competency curricula, such as interpreter use, but were less confident in other areas. The top 3 areas identified for further training included implicit bias, working with transgender and gender nonconforming patients, and weight bias. For the majority of diversity and bias-related skills, prior training was significantly correlated with higher skill level (P < .05).

Psychometric properties of instruments used to measure the cultural competence of nurses: A systematic review

BACKGROUND

Cultural competence is a key component of culturally congruent nursing care. In order to reduce healthcare inequalities and to identify potentials for improvement in nursing practice, researchers need to be able to assess cultural competence properly. Although many instruments for the assessment of cultural competence have been developed, their measurement properties have not yet been reviewed systematically. Such an overview of existing instruments, however, would allow researchers to identify the most valid and reliable instrument for nursing practice.

OBJECTIVE

The purpose of conducting this review is to identify and critically appraise the psychometric properties of instruments used to measure the cultural competence of nurses.

METHODS

A systematic literature search was performed in November 2019 in the following electronic databases: Cumulative Index of Nursing and Allied Health Literature, Embase, PsycINFO and PubMed. Studies that were conducted to assess any measurement property of instruments used to measure the cultural competence of nurses were included. Two reviewers independently screened the articles and assessed the risk of bias using the COnsensus-based Standards for the selection of health Measurement INstruments checklist. The quality of included instruments was assessed on the basis of the updated criteria for good measurement properties, and the quality of the summarised results was graded based on the principles of Grading of Recommendations Assessment, Development and Evaluation.

RESULTS

In total, 44 studies describing 21 instruments were included in this study. We found that most instruments were tested for at least some forms of validity, but seldom for reliability. The quality of the psychometric properties was evaluated using the criteria for good measurement properties for the following: content validity, structural validity, internal consistency, reliability, measurement error and construct validity. No studies were found in which cross-cultural validity, criterion validity, or the responsiveness of the included instruments were evaluated. The Transcultural Self-Efficacy Tool, the Cultural Competence Assessment, and the Cultural Competence Health Practitioner Assessment showed sufficient levels of quality for psychometric properties and can be recommended for the assessment of cultural competence in nurses.

CONCLUSION

Given the broad availability of self-administered instruments to assess cultural competence, the development of new instrument is not recommended. A particular need was identified to conduct further psychometric evaluation studies on existing instruments and to adapt them accordingly, and especially on less frequently evaluated properties, such as reliability, measurement error and responsiveness.

Development and validation of an organizational readiness to change instrument focused on cultural competency

OBJECTIVE

To validate a brief survey developed to be used in hospitals nationwide to assess organizational readiness to change to increase cultural competency.

DATA SOURCES/STUDY SETTING

Analysis of primary data collected as part of a 125-item Organizational Assessment Survey conducted in the ten US hospitals participating in the Robert Wood Johnson Foundation Expecting Success program in 2005-2006.

STUDY DESIGN

The study utilized a cross-sectional survey.

DATA COLLECTION

Surveys were distributed to participants in the ten hospitals based on job title and role within the organization (including clinicians, clinical administrators, other clinical professionals, and those in relevant nonclinical roles; respondents = 513; response rate = 31%). Missing data were deleted listwise. We computed internal consistency reliability via Cronbach's alpha and interrater agreement using the r_wg(j) index, and conducted exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to examine validity of the survey. We subsequently conducted ANOVAs to examine whether the instrument adequately distinguished between hospitals.

PRINCIPAL FINDINGS

Across 408 complete responses, a scree plot generated by the EFA and a follow-up CFA indicated a 2-factor solution (RMSEA = 0.06; CFI = 0.96; GFI = 0.96; RMSR = 0.08). We identified these primary factors as two scales, a 12-item Readiness to Address Quality scale (α = 0.85; r_wg(j)  = 0.93) and an 11-item Readiness to Address Disparities scale (α = 0.65; r_wg(j)  = 0.89). ANOVAs suggested that these scales distinguished between hospitals (RTAQ: F[9, 428] = 3.70, P < .001; RTAD: F[9, 435] = 3.02, P = .002).

CONCLUSIONS

This survey can help identify an organization's readiness to change to increase cultural competency.

[Questionnaires of person centered care in primary care. A systematic review]

Objetivo

Pese a la importancia de la «atención centrada en la persona» (ACP), hay evidentes limitaciones para valorarla y medirla, debidas probablemente a la dificultad para definirla. El objetivo del estudio fue identificar herramientas validadas que midiesen la ACP o algunos de sus aspectos en el ámbito de la atención primaria.

Diseño

Revisión sistemática.

Fuentes de datos

MEDLINE, Embase, Cochrane, PSYCinfo, IME-Biomedicina, LILACS y TESEO hasta del 31 de mayo de 2018.

Selección de estudios

Los criterios de inclusión fueron: a) estudios de validación de cuestionarios, revisiones sistemáticas sobre estudios de validación u otros estudios descriptivos, b) realizados todos ellos en el ámbito de la atención primaria y c) que midieran aspectos de la ACP en profesionales y/o pacientes. Dos investigadores revisaron de forma independiente los artículos y sus discrepancias fueron resueltas por un tercer investigador.

Extracción de datos

Se recogieron datos sobre los aspectos de la ACP medidos, población a la que se dirige, tipo de cuestionario, y datos sobre validez y fiabilidad.

Resultados

Se localizaron 1.415 artículos a los que se añadieron 54 referencias adicionales identificadas a través de referencias de los artículos de la revisión sistemática. Tras una depuración finalmente fueron 75 los artículos que cumplieron todos los criterios y 39 las herramientas identificadas y clasificadas según las dimensiones analizadas.

Conclusiones

Debido a la dificultad de medir la ACP en su conjunto, la mayor parte de los artículos hacen referencia solo a alguno de sus aspectos o dimensiones, predominando la perspectiva del paciente frente a la del profesional. Estas herramientas son, no obstante, un importante punto de partida para futuros cuestionarios que intenten valorar de forma integral la ACP.

Identifying People With Intellectual and Developmental Disabilities in National Population Surveys

Public health and policy planning for adults with intellectual and developmental disabilities (IDD) is imperiled by the lack of ongoing national surveillance data on prevalence and health status. In 2018, the Administration on Intellectual and Developmental Disabilities appointed a workgroup of representatives from key federal agencies and national experts to recommend strategies to improve prevalence estimates and health surveillance for people with IDD. This article presents the workgroup findings on the availability of prevalence and health surveillance data for adults with IDD and suggested items that could identify respondents with IDD on national surveys with special attention to modifications in the National Health Interview Survey. We identify core constructs that must be measured to identify sample members with IDD in population surveys, and additional constructs which, if measured, would support more comprehensive identification of sample members and enhance ongoing surveillance of the health status, outcomes, and unmet needs of this population. We conclude with a brief review of methodological considerations to improve IDD national surveillance including cultural and linguistic sensitivity and the inclusion of U.S. territories in national surveillance protocols.

Applying modern measurement approaches to constructs relevant to evidence-based practice among Canadian physical and occupational therapists

BACKGROUND

Evidence-based practice (EBP) is a complex process. To quantify it, one has to also consider individual and contextual factors using multiple measures. Modern measurement approaches are available to optimize the measurement of complex constructs. This study aimed to develop a robust measurement approach for constructs around EBP including practice, individual (e.g. knowledge, attitudes, confidence, behaviours), and contextual factors (e.g. resources).

METHODS

One hundred eighty-one items arising from 5 validated EBP measures were subjected to an item analysis. Nominal group technique was used to arrive at a consensus about the content relevance of each item. Baseline questionnaire responses from a longitudinal study of the evolution of EBP in 128 new graduates of Canadian physical and occupational therapy programmes were analysed. Principles of Rasch Measurement Theory were applied to identify challenges with threshold ordering, item and person fit to the Rasch model, unidimensionality, local independence, and differential item functioning (DIF).

RESULTS

The nominal group technique identified 70/181 items, and modified Delphi approach identified 68 items that fit a formative model (2 related EBP domains: self-use of EBP (9 items) and EBP activities (7 items)) or a reflective model (4 related EBP domains: attitudes towards EBP (17 items), self-efficacy (9 items), knowledge (11 items) and resources (15 items)). Rasch analysis provided a single score for reflective construct. Among attitudes items, 65% (11/17) fit the Rasch model, item difficulties ranged from - 7.51 to logits (least difficult) to + 5.04 logits (most difficult), and person separation index (PSI) = 0.63. Among self-efficacy items, 89% (8/9) fit the Rasch model, item difficulties ranged from - 3.70 to + 4.91, and PSI = 0.80. Among knowledge items, 82% (9/11) fit the Rasch model, item difficulties ranged from - 7.85 to 4.50, and PSI = 0.81. Among resources items, 87% (13/15) fit the Rasch model, item difficulties ranged from - 3.38 to 2.86, and PSI = 0.86. DIF occurred in 2 constructs: attitudes (1 by profession and 2 by language) and knowledge (1 by language and 2 by profession) arising from poor wording in the original version leading to poor translation.

CONCLUSIONS

Rasch Measurement Theory was applied to develop a valid and reliable measure of EBP. Further modifications to the items can be done for subsequent waves of the survey.

Assessment of Psychometric Properties of an Oral Health Care Measure of Cultural Competence Among Dental Students Using Rasch Partial Credit Model

Reliability, validity, and feasibility of the only validated oral health care measure of cultural competence, the Knowledge, Efficacy, and Practices Instrument (KEPI), have been confirmed. However, the instrument's psychometric properties including item and person reliability, category response functioning, and scale targeting, as well as differential scale functioning for subgroups, have not yet been examined. The aim of this study was to test the psychometric properties of KEPI among dental students using Rasch Partial Credit Model to determine if this model provided broader valid information that cannot be demonstrated according to Classical Test Theory. A total 1,290 dental students in the first or final semester at four U.S. dental schools were invited to participate in the study in 2016. Of those, 1,231 individuals completed the survey, for a 95.4% response rate. The participants were 613 males and 618 females and 889 non-underrepresented minority (URM) and 342 URM students. The Rasch Partial Credit Model assessed the psychometric properties of KEPI's 20 items/three subscales. Differential scale functioning was found in the Culture-Centered Practice and Efficacy of Assessment subscales. Four items were endorsed differentially by gender; four items were endorsed differentially by URM/non-URM students. This study examined the psychometric properties of the KEPI using Rasch analysis to assess differential item functioning by dental student gender and race. The results provided valid evidence for the high internal reliability, measurement properties, and unidimensionality for the KEPI domains, ideal targeting, and well response category functioning, showing that the KEPI is a reliable instrument for measuring the subscales Knowledge of Diversity, Culture-Centered Practice skills, and Efficacy of Assessment for health care providers.

Evaluation of the psychometric properties of the Korean version of the Cultural Competence Assessment

AIM

The 25 item Cultural Competence Assessment assesses the cultural competence of multiple types of healthcare providers. This study aimed to examine the validity and reliability of the Korean version of the questionnaire (KCCA) and to determine the need for changes to improve its validity and reliability.

METHODS

Data from 161 hospital nurses were used for the item analysis and to assess the reliability and construct validity of the KCCA before and after the deletion of nine items.

RESULTS

The KCCA did not demonstrate acceptable construct validity and subscale internal reliability. Nine items with high interitem correlations, high modification indices, and relatively lower factor loadings were deleted. The 16 item Modified KCCA showed improved construct validity, convergent and discriminant validity, and reliability.

CONCLUSION

While further psychometric evaluation of the Modified KCCA should be undertaken with larger samples and diverse professionals, the study's data provide evidence that the Modified KCCA might be a more suitable measure for use among Korean healthcare providers.

Development and psychometric assessment of the healthcare provider cultural competence instrument

This study presents the measurement properties of 5 scales used in the Healthcare Provider Cultural Competence Instrument (HPCCI). The HPCCI measures a health care provider's cultural competence along 5 primary dimensions: (1) awareness/sensitivity, (2) behaviors, (3) patient-centered communication, (4) practice orientation, and (5) self-assessment. Exploratory factor analysis demonstrated that the 5 scales were distinct, and within each scale items loaded as expected. Reliability statistics indicated a high level of internal consistency within each scale. The results indicate that the HPCCI effectively measures the cultural competence of health care providers and can provide useful professional feedback for practitioners and organizations seeking to increase a practitioner's cultural competence.

A students' survey of cultural competence as a basis for identifying gaps in the medical curriculum

BACKGROUND

Assessing the cultural competence of medical students that have completed the curriculum provides indications on the effectiveness of cultural competence training in that curriculum. However, existing measures for cultural competence mostly rely on self-perceived cultural competence. This paper describes the outcomes of an assessment of knowledge, reflection ability and self-reported culturally competent consultation behaviour, the relation between these assessments and self-perceived cultural competence, and the applicability of the results in the light of developing a cultural competence educational programme.

METHODS

392 medical students, Youth Health Care (YHC) Physician Residents and their Physician Supervisors were invited to complete a web-based questionnaire that assessed three domains of cultural competence: 1) general knowledge of ethnic minority care provision and interpretation services; 2) reflection ability; and 3) culturally competent consultation behaviour. Additionally, respondents graded their overall self-perceived cultural competence on a 1-10 scale.

RESULTS

86 medical students, 56 YHC Residents and 35 YHC Supervisors completed the questionnaire (overall response rate 41%; n= 177). On average, respondents scored low on general knowledge (mean 46% of maximum score) and knowledge of interpretation services (mean 55%) and much higher on reflection ability (80%). The respondents' reports of their consultation behaviour reflected moderately adequate behaviour in exploring patients' perspectives (mean 64%) and in interaction with low health literate patients (mean 60%) while the score on exploring patients' social contexts was on average low (46%). YHC respondents scored higher than medical students on knowledge of interpretation services, exploring patients' perspectives and exploring social contexts. The associations between self-perceived cultural competence and assessed knowledge, reflection ability and consultation behaviour were weak.

CONCLUSION

Assessing the cultural competence of medical students and physicians identified gaps in knowledge and culturally competent behaviour. Such data can be used to guide improvement efforts to the diversity content of educational curricula. Based on this study, improvements should focus on increasing knowledge and improving diversity-sensitive consultation behaviour and less on reflection skills. The weak association between overall self-perceived cultural competence and assessed knowledge, reflection ability and consultation behaviour supports the hypothesis that measures of sell-perceived competence are insufficient to assess actual cultural competence.