RIME Foreword: What Constitutes Science in Educational Research? Applying Rigor in Our Research Approaches

Research Methodologies in Health Professions Education Publications: Breadth and Rigor

PURPOSE

Research methodologies represent assumptions about knowledge and ways of knowing. Diverse research methodologies and methodological standards for rigor are essential in shaping the collective set of knowledge in health professions education (HPE). Given this relationship between methodologies and knowledge, it is important to understand the breadth of research methodologies and their rigor in HPE research publications. However, there are limited studies examining these questions. This study synthesized current trends in methodologies and rigor in HPE papers to inform how evidence is gathered and collectively shapes knowledge in HPE.

METHOD

This descriptive quantitative study used stepwise stratified cluster random sampling to analyze 90 papers from 15 HPE journals published in 2018 and 2019. Using a research design codebook, the authors conducted group coding processes for fidelity, response process validity, and rater agreement; an index quantifying methodological rigor was developed and applied for each paper.

RESULTS

Over half of research methodologies were quantitative (51%), followed by qualitative (28%), and mixed methods (20%). No quantitative and mixed methods papers reported an epistemological approach. All qualitative papers that reported an epistemological approach (48%) used social constructivism. Most papers included participants from North America (49%) and Europe (20%). The majority of papers did not specify participant sampling strategies (56%) or a rationale for sample size (80%). Among those reported, most studies (81%) collected data within 1 year.The average rigor score of the papers was 56% (SD = 17). Rigor scores varied by journal categories and research methodologies. Rigor scores differed between general HPE journals and discipline-specific journals. Qualitative papers had significantly higher rigor scores than quantitative and mixed methods papers.

CONCLUSIONS

This review of methodological breadth and rigor in HPE papers raises awareness in addressing methodological gaps and calls for future research on how the authors shape the nature of knowledge in HPE.

Use of the CPD-REACTION Questionnaire to Evaluate Continuing Professional Development Activities for Health Professionals: Systematic Review

BACKGROUND

Continuing professional development (CPD) is essential for physicians to maintain and enhance their knowledge, competence, skills, and performance. Web-based CPD plays an essential role. However, validated theory-informed measures of their impact are lacking. The CPD-REACTION questionnaire is a validated theory-informed tool that evaluates the impact of CPD activities on clinicians' behavioral intentions.

OBJECTIVE

We aimed to review the use of the CPD-REACTION questionnaire, which measures the impact of CPD activities on health professionals' intentions to change clinical behavior. We examined CPD activity characteristics, ranges of intention, mean scores, score distributions, and psychometric properties.

METHODS

We conducted a systematic review informed by the Cochrane review methodology. We searched 8 databases from January 1, 2014, to April 20, 2021. Gray literature was identified using Google Scholar and Research Gate. Eligibility criteria included all health care professionals, any study design, and participants' completion of the CPD-REACTION questionnaire either before, after, or before and after a CPD activity. Study selection, data extraction, and study quality evaluation were independently performed by 2 reviewers. We extracted data on characteristics of studies, the CPD activity (eg, targeted clinical behavior and format), and CPD-REACTION use. We used the Mixed Methods Appraisal Tool to evaluate the methodological quality of the studies. Data extracted were analyzed using descriptive statistics and the Student t test (2-tailed) for bivariate analysis. The results are presented as a narrative synthesis reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Overall, 65 citations were eligible and referred to 52 primary studies. The number of primary studies reporting the use of CPD-REACTION has increased continuously since 2014 from 1 to 16 publications per year (2021). It is available in English, French, Spanish, and Dutch. Most of the studies were conducted in Canada (30/52, 58%). Furthermore, 40 different clinical behaviors were identified. The most common CPD format was e-learning (34/52, 65%). The original version of the CPD-REACTION questionnaire was used in 31 of 52 studies, and an adapted version in 18 of 52 studies. In addition, 31% (16/52) of the studies measured both the pre- and postintervention scores. In 22 studies, CPD providers were university-based. Most studies targeted interprofessional groups of health professionals (31/52, 60%).

CONCLUSIONS

The use of CPD-REACTION has increased rapidly and across a wide range of clinical behaviors and formats, including a web-based format. Further research should investigate the most effective way to adapt the CPD-REACTION questionnaire to a variety of clinical behaviors and contexts.

TRIAL REGISTRATION

PROSPERO CRD42018116492; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=116492.

RIME 60 Years: Celebration and Future Horizons

This year marks the 60th anniversary (1961-2021) of Research in Medical Education (RIME). Over the past 6 decades, RIME has selected medical education research to be presented each year at the Association of American Medical Colleges Annual Meeting: Learn Serve Lead and published in a supplement of Academic Medicine. In this article, the authors surveyed RIME chairs from the past 20 years to identify ways that RIME has advanced medical education research and to generate ideas for future directions. RIME chairs described advancements in the rigor and impact of RIME research and the timeliness of the topics, often serving as a driver for cutting-edge research. They highlighted RIME's role in promoting qualitative research, introducing new epistemologies, and encouraging networking as a means of career advancement. Going forward, RIME chairs suggested (1) strengthening collaborations with formal advanced MEd and PhD degree programs, (2) creating formal mentorship channels for junior and minority faculty, and (3) promoting research related to knowledge translation.