Cross-validation of a learning climate instrument in a non-western postgraduate clinical environment

Educational climate of a pathology residency program at a tertiary care hospital

Evaluating educational climate (EC) is imperative for ensuring postgraduate trainees' competencies and quality in residency training programs. This study assessed the EC experiences of pathology postgraduate residents (PGRs) during their postgraduate training in pathology residency programs-a cross-sectional study design assigned EC scores in the pathology residency program at a prestigious institution in Islamabad, which were measured using the Dutch Residency Educational Climate Test (D-RECT) questionnaire. Scores from the D-RECT were employed to conduct descriptive statistics and comparison of means across groups to evaluate EC scores by years of training and compared to assess where the differences were located. Among FCPS-II pathology residents, most of whom were females (94.4%), the mean age was 28.11±2.91 years. A mean positive score was observed among all pathology residents (M≥3.6) for all D-RECT subscales except for the feedback subscale: the average score for feedback was below the average mean score of 3.6 (M = 3.19). A significant difference p = 0.016 was observed in EC scores across different groups through the Analysis of Variances (ANOVA) test. The most significant difference was between less than two and greater than two groups p = 0.027, followed by the difference between equal to two groups and greater than two groups p = 0.052. Overall, positive scores for EC in the pathology residency program were observed. Thus, targeted interventions are needed to increase feedback scores and address observed differences in EC scores by years of training.

The postgraduate medical educational climate assessed by the Danish Residency Educational Climate Test (DK-RECT): a validation and cross-sectional observational study

BACKGROUND

A good educational climate is essential for delivering high-quality training for medical trainees, professional development, and patient care. The aim of this study was to (1) validate the Dutch Residency Educational Climate Test (D-RECT) in a Danish setting and (2) describe and evaluate the educational climate among medical trainees.

METHODS

D-RECT was adopted in a three-step process: translation of D-RECT into Danish (DK-RECT), psychometric validation, and evaluation of educational climate. Trainees from 31 medical specialties at Copenhagen University Hospital - Rigshospitalet, Denmark were asked to complete an online survey in a cross-sectional study.

RESULTS

We performed a forward-backward translation from Dutch to Danish. Confirmatory factor analysis showed that DK-RECT was robust and valid. The reliability analysis showed that only seven trainees from one specialty were needed for a reliable result. With 304 trainees completing DK-RECT, the response rate was 68%. The subsequent analysis indicated a positive overall educational climate, with a median score of 4.0 (interquartile range (IQR): 3.0-5.0) on a five-point Likert scale. Analysis of the subscales showed that the subscale Feedback received the lowest ratings, while Supervision and Peer collaboration were evaluated highest.

CONCLUSIONS

Psychometric validation of D-RECT in a Danish context demonstrated valid results on the educational climate in specialist training. DK-RECT can be used to evaluate the effectiveness of interventions in the future and can facilitate the conversation on the educational climate.

Validation of the French translation of the Dutch residency educational climate test

BACKGROUND

The learning environment is one of the most influential factors in training of medical residents. The Dutch Residency Educational Climate Test (D-RECT) is one of the strongest instruments for measuring the learning environment. However, it has not been translated in French. The objective of this study is the psychometric validation of the DRECT French version.

MATERIAL AND METHODS

After translation of the D-RECT questionnaire into French, residents of five Moroccan hospitals were invited to complete the questionnaire between July and September 2018. Confirmatory factor analysis was used to evaluate the validity of the construct using the standardized root mean square residual (SRMR), the root mean square error approximation (RMSEA), the Comparative Fit Index (CFI) and the Tucker- Lewis Index (TLI). Reliability analysis was analysed using Internal consistency and Test-retest.

RESULTS

During the study period, 211 residents completed the questionnaire. Confirmatory factor analysis showed an adequate model fit with the following indicators: SRMR = 0.058 / RMSEA = 0.07 / CFI = 0.88 / TLI = 0.87. The French translation had a good internal consistency (Cronbach alpha score >  0.7 for all subscales) and a good temporal stability (correlation score between two measurements = 0.89).

CONCLUSION

This French version has an acceptable validity of the construct, a good internal consistency and good temporal reliability, and may be used to evaluate the learning climate. Additional research is necessary in other French-speaking contexts, in order to confirm these results.

The Dutch residency educational climate test: construct and concurrent validation in Spanish language

OBJECTIVES

To translate the 35-item version of the Dutch Residency Educational Climate Test (D-RECT), and assess its reliability, construct validity and concurrent validity in the Spanish language.

METHODS

For this validation study, the D-RECT was translated using international recommendations. A total of 220 paper-based resident evaluations covering two Colombian universities were cross-sectionally collected in 2015. A Confirmatory Factor Analysis (CFA) was used to assess the internal validity of the instrument using the Comparative fit index (CFI), Tucker-Lewis index (TLI), Standardized root mean square residual (SRMSR), and Root mean square error of approximation (RMSA). Cronbach's α was used to assess reliability. The concurrent validity was investigated through Pearson correlations with the Spanish version of the Postgraduate Hospital Educational Environment Measure (PHEEM).

RESULTS

The original 9-factor structure showed an appropriate fit for the Spanish version of the instrument (CFI = 0.84, TLI = 0.82, SRMSR = 0.06, and RMSA = 0.06). The reliability coefficients were satisfactory (>0.70). The mean total scores of the D-RECT and the PHEEM showed a significant correlation (r = 0.7, p<0.01).

CONCLUSIONS

This study confirms the validity and reliability of the Spanish version of the Dutch Residency Educational Climate Test, indicating that the instrument is suitable for the evaluation of departments' learning climate in the Spanish context. Future research is needed to confirm these findings in other Spanish speaking countries.

Cross-validation of a learning climate instrument in a non-western postgraduate clinical environment

BACKGROUND

In postgraduate training, there is a need to continuously assess the learning and working conditions to optimize learning. Students or trainees respond to the learning climate as they perceive it. The Dutch Residency Educational Climate Test (D-RECT) is a learning climate measurement tool with well-substantiated validity. However, it was originally designed for Dutch postgraduate trainees and it remains to be shown whether extrapolation to non-Western settings is viable. The dual objective of this study was to revalidate D-RECT outside of a Western setting and to evaluate the factor structure of a recently revised version of the D-RECT containing 35 items.

METHODS

We invited Filipino internal medicine residents from 96 hospitals to complete the revised 35-item D-RECT. Subsequently, we performed a confirmatory factor analysis to check the fit of the 9 scale model of the revised 35-item D-RECT. Inter-rater reliability was assessed using generalizability theory.

RESULTS

Confirmatory factor analysis unveiled that the factor structure of the revised 35-item D-RECT provided a reasonable fit to the Filipino data, after removal of 7 items. Five to seven evaluations of individual residents were needed per scale to obtain a reliable result.

CONCLUSION

Even in a non-Western setting, the D-RECT exhibited psychometric validity. This study validated the factor structure of the revised 35-item D-RECT after some modifications. We recommend that its application be extended to other Asian countries and specialties.