Rebooting Kirkpatrick: Integrating Information System Theory Into the Evaluation of Web-based Continuing Professional Development Interventions for Interprofessional Education

Using Administrative Data in Primary Care to Evaluate the Effectiveness of a Continuing Professional Development Program Focused on the Management of Patients Living With Obesity

INTRODUCTION

There are guidelines for referral to medical and/or surgical weight loss interventions (MSWLI) in Ontario; however, only about one-third of eligible patients in our region are being referred for consideration of MSWLI.

METHODS

A planning committee, including a registered dietician, psychiatrist, endocrinologist, bariatric surgeon, family physician, and educationalists, developed an interdisciplinary continuing professional development (CPD) program focused on practical approaches to the management of patients living with obesity. The Kirkpatrick model was used to evaluate the educational outcomes of the CPD program specifically focusing on Level-2, -3, and -4 outcomes based on self-reported questionnaire and health administrative data.

RESULTS

Eighteen primary care providers from the CPD program agreed to participate in this study, and 16 primary care providers (89%) completed the postintervention questionnaire and granted us access to their MSWLI referral data; 94% of study participants reported changes to their knowledge, comfort, and confidence (Level 2), as well as expected change in their future behaviour (Level 3) following the CPD program. However, there was no change in Kirkpatrick Level-4 outcomes, despite more than 90% of participants indicating that they will be making changes to their practice after the program.

DISCUSSION

The CPD program in our study was overwhelmingly well received and participants reported knowledge (Level 2) and behavioural (Level 3) changes following participation; however, there was no detectable change in their clinical practice (Level 4). The methodology described in our proof-of-concept study can be modified and adopted to evaluate Level-4 outcomes in other studies of effectiveness of CPD interventions.

How do dietetics students learn about sustainability? A scoping review

AIM

Globally, sustainability and planetary health are emerging as areas of critical importance. In 2015, the 2030 Agenda for Sustainable Development was adopted by the United Nations member states. Since then, the United Nations Educational, Scientific and Cultural Organization and the Commonwealth Secretariat have published guidelines for educators to embed sustainability content into curricula. This scoping review aims to identify how student dietitians learn about sustainability, how learning opportunities are evaluated, their outcomes, and whether these guidelines have translated into teaching activities contained in dietetic degrees.

METHODS

A scoping review was used to address the aims. Eight electronic databases and Google Scholar were searched from inception to March 2022 for articles describing dietetics students' participation in learning activities focused on sustainability. Data that addressed the research aims were charted independently by two researchers, then narratively synthesised.

RESULTS

Twelve articles met the inclusion criteria. A range of teaching approaches and evaluation methods were used, from passive learning in lectures to experiential learning activities. A change in knowledge or behaviour was found for experiential learning activities (n = 5). For articles published after 2015 (n = 9), two mentioned the Sustainable Development Goals and no articles referenced the published guidelines.

CONCLUSIONS

A paucity of evidence exists describing how dietetics students learn about sustainability and their learning outcomes. Of the 12 articles published, varied teaching approaches and evaluation methods have resulted in inconsistencies in the reporting of outcomes. The minimal reference to the Sustainable Development Goals and published guidelines suggests a slow translation of knowledge to practice.

Teaching staff in interprofessional education: A proposed terminology

Different terms (e.g., interprofessional, multiprofessional, interdisciplinary) are used in interprofessional education and collaboration without sufficient differentiation or precision in regard to meaning. In recent years academic publications in English and German have contributed to clarifying this issue. However, there are no definitions internationally or in the German-speaking countries (Germany, Austria, Switzerland) specifically referring to the people engaged in teaching interprofessional education. Teaching in interprofessional education has evolved from the traditional role of expert to one of mentor or facilitator. It is also evident that those who teach play a central role in the success of interprofessional courses. While many different designations are used to refer to interprofessional teachers in the relevant literature and in the language of daily use, a uniform and adequate terminology should be used to refer to such teaching staff. Based on literature reviews, this commentary seeks to propose terms for teaching staff active in the area of interprofessional education and thus provide a basis for discussion in the German-speaking countries. Taking the results of the literature analysis and the roles of teachers in interprofessional settings into consideration, we propose that the English term "IP facilitator" (IP for interprofessional) should also be used in the German-speaking world and "facilitateur IP" in the French-speaking world. A French translation is included in attachment 1 to enable broader discussion in Switzerland.

Position paper of the GMA Committee Interprofessional Education in the Health Professions - current status and outlook

In the wake of local initiatives and developmental funding programs, interprofessionality is now included in national curricula in the German-speaking countries. Based on the 3P model (presage, process, product), this position paper presents the development of interprofessional education in recent years in Germany, Austria and Switzerland and places it in an international context. Core aspects as legal frameworks, including amendments to occupational regulations as well as the formation of networks and faculty development are basic requirements for interprofessional education. New topics and educational settings take shape in the process of interprofessional education: patient perspectives and teaching formats, such as online courses, become more important or are newly established. The influence of the COVID-19 pandemic on interprofessional education is explored as well. Among many new interprofessional courses, particularly the implementation of interprofessional training wards in Germany and Switzerland are positive examples of successful interprofessional education. The objective of interprofessional education continues to be the acquisition of interprofessional competencies. The main focus is now centered on evaluating this educational format and testing for the corresponding competencies. In the future, more capacities will be required for interprofessional continuing education and post-graduate education. Structured research programs are essential to ascertain the effects of interprofessional education in the German-speaking countries. In this position paper the GMA committee on interprofessional education encourages further advancement of this topic and expresses the aim to continue cooperating with other networks to strengthen and intensify interprofessional education and collaboration in healthcare.

Optimizing the integration of family caregivers in the delivery of person-centered care: evaluation of an educational program for the healthcare workforce

BACKGROUND

While family caregivers provide 70-90% of care for people living in the community and assist with 10-30% of the care in congregate living, most healthcare providers do not meaningfully involve family caregivers as partners in care. Recent research recommends that the healthcare workforce receive competency-based education to identify, assess, support, and partner with family caregivers across the care trajectory.

OBJECTIVE

This paper reports a mixed-methods evaluation of a person-centered competency-based education program on Caregiver-Centered Care for the healthcare workforce.

METHODS

This foundational education was designed for all healthcare providers and trainees who work with family caregivers and is offered free online (caregivercare.ca). Healthcare providers from five healthcare settings (primary, acute, home, supportive living, long-term care) and trainees in medicine, nursing, and allied health were recruited via email and social media. We used the Kirkpatrick-Barr health workforce training evaluation framework to evaluate the education program, measuring various healthcare providers' learner satisfaction with the content (Level 1), pre-post changes in knowledge and confidence when working with family caregivers (Level 2), and changes in behaviors in practice (Level 3).

RESULTS

Participants were primarily healthcare employees (68.9%) and trainees (21.7%) and represented 5 healthcare settings. Evaluation of the first 161 learners completing the program indicated that on a 5-point Likert scale, the majority were satisfied with the overall quality of the education (Mean(M) = 4.69; SD = .60). Paired T-tests indicated that out of a score of 50, post-education changes in knowledge and confidence to work with family caregivers was significantly higher than pre-education scores (pre M = 38.90, SD = 6.90; post M = 46.60, SD = 4.10; t(150) = - 16.75, p < .0001). Qualitative results derived from open responses echoed the quantitative findings in satisfaction with the education delivery as well as improvements in learners' knowledge and confidence.

CONCLUSION

Health workforce education to provide person-centered care to all family caregivers is an innovative approach to addressing the current inconsistent system of supports for family caregivers. The education program evaluated here was effective at increasing self-reported knowledge and confidence to work with family caregivers.

Artificial Intelligence Education Programs for Health Care Professionals: Scoping Review

BACKGROUND

As the adoption of artificial intelligence (AI) in health care increases, it will become increasingly crucial to involve health care professionals (HCPs) in developing, validating, and implementing AI-enabled technologies. However, because of a lack of AI literacy, most HCPs are not adequately prepared for this revolution. This is a significant barrier to adopting and implementing AI that will affect patients. In addition, the limited existing AI education programs face barriers to development and implementation at various levels of medical education.

OBJECTIVE

With a view to informing future AI education programs for HCPs, this scoping review aims to provide an overview of the types of current or past AI education programs that pertains to the programs' curricular content, modes of delivery, critical implementation factors for education delivery, and outcomes used to assess the programs' effectiveness.

METHODS

After the creation of a search strategy and keyword searches, a 2-stage screening process was conducted by 2 independent reviewers to determine study eligibility. When consensus was not reached, the conflict was resolved by consulting a third reviewer. This process consisted of a title and abstract scan and a full-text review. The articles were included if they discussed an actual training program or educational intervention, or a potential training program or educational intervention and the desired content to be covered, focused on AI, and were designed or intended for HCPs (at any stage of their career).

RESULTS

Of the 10,094 unique citations scanned, 41 (0.41%) studies relevant to our eligibility criteria were identified. Among the 41 included studies, 10 (24%) described 13 unique programs and 31 (76%) discussed recommended curricular content. The curricular content of the unique programs ranged from AI use, AI interpretation, and cultivating skills to explain results derived from AI algorithms. The curricular topics were categorized into three main domains: cognitive, psychomotor, and affective.

CONCLUSIONS

This review provides an overview of the current landscape of AI in medical education and highlights the skills and competencies required by HCPs to effectively use AI in enhancing the quality of care and optimizing patient outcomes. Future education efforts should focus on the development of regulatory strategies, a multidisciplinary approach to curriculum redesign, a competency-based curriculum, and patient-clinician interaction.

Development of a digital learning program for physiotherapists to enhance clinical implementation of aerobic exercise in stroke rehabilitation

BACKGROUND

This paper describes the initial development process of an eLearning continuing professional education program primarily for post-licensure physiotherapists -"Electronic Aerobic Exercise Recommendations to Optimize Best Practices in Care after Stroke" (eAEROBICS). Our objective was to develop an evidence-based, clinically relevant, user-friendly eLearning program for online delivery tailored to facilitate prescription of aerobic exercise post-stroke by physiotherapists. The Demand Driven Learning Model guided curriculum design, delivery, and evaluation. Based on previously identified gaps in physiotherapists' knowledge of aerobic exercise, four learning modules were developed and delivered using an eLearning platform to maximize cost-effectiveness and flexibility. Five physiotherapists volunteered to pilot eAEROBICS, providing preliminary feedback on strengths and suggestions for improvement.

RESULTS

Theoretical information and clinical applications addressed the learning objectives of each module in a logical manner. All technical or administrative issues encountered during program delivery were addressed. The feedback from the pilot end-users informed modifications to the eAEROBICS program.

CONCLUSIONS

Processes used in developing eAEROBICS have the potential to serve as a model of electronic continuing professional education for other areas of physiotherapy practice. Further investigation of end-user perspectives and clinical impact of the program is warranted to determine the overall effectiveness of the program.

An Education Framework for Effective Implementation of a Health Information System: Scoping Review

Background

To optimize their use of a new Health Information System (HIS), supporting health care providers require effective HIS education. Failure to provide this education can significantly hinder an organization’s HIS implementation and sustainability efforts.

Objective

The aim of this review is to understand the most effective educational strategies and approaches to enable health care providers to optimally use an HIS.

Methods

Ovid MEDLINE, Ovid Embase, EBSCO Cumulative Index to Nursing and Allied Health Literature, and EBSCO Education Resources Information Center were searched to identify relevant papers. Relevant studies were systematically reviewed and analyzed using a qualitative thematic analysis approach.

Results

Of the 3539 studies screened, 17 were included for data extraction. The literature on the most effective approaches to enable health care providers to optimally use an HIS emphasized the importance of investing in engaging and understanding learners in the clinical context, maximizing the transfer of learning to care, and designing continuous and agile evaluation to meet the emerging demands of the clinical environment.

Conclusions

This review supports the advancement of a new HIS learning framework that organizational leaders and educators can use to guide HIS education design and development. Future research should examine how this framework can be translated into practice.

Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews

Background

E-learning is rapidly growing as an alternative way of delivering education in nursing. Two contexts regarding the use of e-learning in nursing are discussed in the literature: (1) education among nursing students and (2) nurses’ continuing education within a life-long learning perspective. A systematic review of systematic reviews on e-learning for nursing and health professional students in an academic context has been published previously; however, no such review exists regarding e-learning for registered nurses in a continuing education context.

Objective

We aimed to systematically summarize the qualitative and quantitative evidence regarding the effects of e-learning on nursing care among nurses in a continuing education context.

Methods

We conducted a systematic review of systematic qualitative, quantitative, and mixed-studies reviews, searching within four bibliographic databases. The eligibility criteria were formulated using the population, interventions, comparisons, outcomes, and study design (PICOS) format. The included population was registered nurses. E-learning interventions were included and compared with face-to-face and any other e-learning interventions, as well as blended learning. The outcomes of interest were derived from two models: nursing-sensitive indicators from the Nursing Care Performance Framework (eg, teaching and collaboration) and the levels of evaluation from the Kirkpatrick model (ie, reaction, learning, behavior, and results).

Results

We identified a total of 12,906 records. We retrieved 222 full-text papers for detailed evaluation, from which 22 systematic reviews published between 2008 and 2018 met the eligibility criteria. The effects of e-learning on nursing care were grouped under Kirkpatrick’s levels of evaluation: (1) nurse reactions to e-learning, (2) nurse learning, (3) behavior, and (4) results. Level 2, nurse learning, was divided into three subthemes: knowledge, skills, attitude and self-efficacy. Level 4, results, was divided into patient outcomes and costs. Most of the outcomes were reported in a positive way. For instance, nurses were satisfied with the use of e-learning and they improved their knowledge. The most common topics covered by the e-learning interventions were medication calculation, preparation, and administration.

Conclusions

The effects of e-learning are mainly reported in terms of nurse reactions, knowledge, and skills (ie, the first two levels of the Kirkpatrick model). The effectiveness of e-learning interventions for nurses in a continuing education context remains unknown regarding how the learning can be transferred to change practice and affect patient outcomes. Further scientific, methodological, theoretical, and practice-based breakthroughs are needed in the fast-growing field of e-learning in nursing education, especially in a life-learning perspective.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42016050714; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=50714