Evaluation of an e-Learning Program for Community Pharmacists for Dispensing Emicizumab (Hemlibra) in France: Nationwide Cross-Sectional Study

BACKGROUND

Since June 2021, patients with hemophilia A with antifactor VIII inhibitors and those with severe hemophilia A without antifactor VIII inhibitors treated with Hemlibra have had to choose between a community or hospital pharmacy. The French reference center for hemophilia developed the HEMOPHAR e-learning program for community pharmacists for dispensing emicizumab.

OBJECTIVE

This study aims to evaluate the efficiency and safety of this new care pathway by assessing the HEMOPHAR e-learning program.

METHODS

The methodology is based on Kirkpatrick's model for evaluating the immediate reaction of trained community pharmacists (level 1), their level of acquired knowledge (level 2), and their professional practice after 3 months of dispensation (level 3).

RESULTS

The HEMOPHAR e-learning program reached a large audience, with 67% (337/502) of the eligible community pharmacists following it. The immediate reaction was overall satisfying. High rates of engagement were reported with 63.5% (214/337) to 73.3% (247/337) of completed training modules, along with high rates of success with quizzes of 61.5% (174/337) to 95.7% (244/337). We observed that 83.9% (193/230) of the community pharmacists needed less than 2 attempts to pass the quiz of the module related to professional practice, while the other quizzes required more attempts. Advice on compliance and drug interactions were most frequently provided to patients by the community pharmacists.

CONCLUSIONS

This study suggests ways to improve the training of community pharmacists and to optimize coordination with treatment centers. This study also reports on the feasibility of switching to a community pharmacy in a secure pharmaceutical circuit, including in the context of a rare bleeding disease.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05449197; https://clinicaltrials.gov/study/NCT05449197.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/43091.

Impact of women's feedback on midwifery student learning: A thematic analysis of students' reflections

PROBLEM

Little is known about the educational impact of providing routine, online feedback from women on midwifery student learning and clinical practice.

BACKGROUND

Feedback on students' clinical performance has historically been provided by lecturers and clinical supervisors. Women's feedback is not routinely collected or evaluated for impact on student learning.

AIM

To evaluate the impact of women's feedback about continuity of care experiences with a midwifery student on learning and practice.

DESIGN

Descriptive, exploratory qualitative study.

METHODS

All second-and third-year Bachelor of Midwifery students undertaking clinical placement between February and June 2022 at one Australian university, submitted formative, guided written reflections on de-identified women's feedback they received through their ePortfolio. Data were analysed using reflexive thematic analysis.

FINDINGS

Forty-four of the 69 eligible students (64%) submitted reflections on feedback received. Three themes emerged: 1) Confidence boosting, 2) Deeply integrating Midwifery Metavalues, and 3) Enhancing commitment to continuity. Three subthemes: connection, future practice and advocacy were identified. Women's feedback positively impacts student learning and places the woman in the educational feedback loop.

CONCLUSION

This study is an international first evaluating the impact of feedback from women on midwifery students' learning. Students reported greater confidence in their clinical practice, a deeper understanding of their midwifery philosophy, and an intention to advocate for, and work in, midwifery continuity models after graduation. Routine feedback about women's experiences should be embedded into midwifery education programs.

The effect of the Educational Scholar Program as a longitudinal faculty development program on the capability of educators as scholars

INTRODUCTION

The Educational Scholar Program (ESP) is designed and implemented as a longitudinal and institution-based faculty development program. The present study aimed to assess the effect of the ESP on educators' capabilities to undertake SoTL activities associated with their scholar role.

METHODS

This study was conducted from 2017 to 2022. The participants (n = 64) were educators in six schools of Shahid Sadoughi University of Medical Sciences. The ESP was a faculty development program that consisted of training and project-based stages. The educators experienced small-group learning, self-directed learning, and reflective assignments in the training stage. In the second stage, the educators completed a SoTL (Scholarship of Teaching and Learning) project. Learner-related outcomes based on Kirkpatrick model was assessed. The reaction of educators (satisfaction, active participation in the ESP, and the perception of mentoring sessions) was assessed by three questionnaires (Reaction level). The educators' learning was evaluated by modified essay questions and their project reports (Learning and Behavior levels). Outputs of the ESP including journal publications, abstracts presented at meetings or congresses, grant funding, awards in educational festivals, promotions, projects with ongoing implementation following the ESP, and conducting further SoTL projects after ESP were assessed quantitatively over two years after participating in the ESP (Results level). Data were summarized by descriptive statistics (mean, percentage, SD, 95% Confidence Interval (CI)). Cut-off scores of the instruments was calculated with a standard setting method which introduced by Cohen-Schotanus and Van DerVleuten. Data analyzed by One-sample t-test.

RESULTS

Sixty-four of 72 (89%) educators completed the ESP. The mean (CI) satisfaction score of educators was 42 (CI: 26.92-58.28), the active participation was 92 (CI: 80.24-103.76). The scores of the mentoring assessment from the perspective of the educators were reported at 90 (CI: 78.24- 101.76). The mean (95%CI) learning scores in the essay examination were 88 (CI: 70.36- 105.64), and project assessment were 90 (CI: 78.24- 101.76). The results showed the educators' scores in reaction and learning significantly higher than the cut-off scores. (P < 0.05). Most projects were conducted in curriculum development and assessment/evaluation domains. The number of projects with ongoing implementation over the two years following the ESP and the acquisition of grants was higher than other outputs in the results level.

CONCLUSION

The ESP, as an institute-based longitudinal program, enhanced the learner-related outcomes (in four levels of reaction, learning, behavior, and results). The creation of practical learning and supportive mechanisms influenced on the results. The outcomes of ESP indicated that the educators prepared to conduct SoTL activities in their educational community.

A Taxonomy for Research Intergrity Training: Design, Conduct, and Improvements in Research Integrity Courses

Trainers often use information from previous learning sessions to design or redesign a course. Although universities conducted numerous research integrity training in the past decades, information on what works and what does not work in research integrity training are still scattered. The latest meta-reviews offer trainers some information about effective teaching and learning activities. Yet they lack information to determine which activities are plausible for specific target groups and learning outcomes and thus do not support course design decisions in the best possible manner. This article wants to change this status quo and outlines an easy-to-use taxonomy for research integrity training based on Kirkpatrick's four levels of evaluation to foster mutual exchange and improve research integrity course design. By describing the taxonomy for research integrity training (TRIT) in detail and outlining three European projects, their intended training effects before the project started, their learning outcomes, teaching and learning activities, and their assessment instruments, this article introduces a unified approach. This article gives practitioners references to identify didactical interrelations and impacts and (knowledge) gaps in how to (re-)design an RI course. The suggested taxonomy is easy to use and enables an increase in tailored and evidence-based (re-)designs of research integrity training.

Evaluating a dental sleep apnea mini-residency program using the Kirkpatrick model

OBJECTIVE

The aim of this retrospective study was to evaluate a Dental Sleep Medicine Mini-Residency (DSMMR) continuing education (CE) program using the Kirkpatrick model.

METHODS

After receiving ethical approval, data from participants in the 2019-2020 DSMMR CE course were included for the Kirkpatrick evaluation. The analysis was stratified and all the Kirkpatrick levels were integrated: level 1 (satisfaction) was assessed via Likert scale and open-ended questions; level 2 (learning) was evaluated using pretest and posttest knowledge data following Module 1 (M1) and an assessment of multiple-choice questions (MCQs) developed by participants; level 3 (behavior) was evaluated using Likert scale questions; and level 4 (results) was assessed via the percentage of participants who passed the American Board of Dental Sleep Medicine (ABDSM) examination on their first attempt.

RESULTS

A total of 90 participants were included in the study. At least 83.1% of participants agreed/strongly agreed with positively worded statements about satisfaction. Knowledge scores significantly increased from pre-M1 to post-M1 (p < 0.001); however, only 15.2% of MCQs were evaluated as well-formulated. At least 88.6% of participants agreed/strongly agreed with positively worded statements about transfer of knowledge/skills to their practice. 91.1% passed the ABDSM examination on their first attempt.

CONCLUSION

The evaluation of the 2019-2020 DSMMR using the Kirkpatrick model suggests its overall positive impact as a training program. The Kirkpatrick model provided information that can be used to improve the quality of a program. Future studies should assess other dental CE programs using the Kirkpatrick model or another evaluation model.

Evaluation of the Care Pathway in the Context of the Dispensing of Emicizumab (Hemlibra) in Community Pharmacies in France: Protocol for a Cross-sectional Study Based on the Kirkpatrick Model

BACKGROUND

Commercialized since 2019, emicizumab (Hemlibra) was available only in French hospital pharmacies for prophylaxis of hemophilia A with or without inhibitors. Since June 15, 2021, patients can choose between a hospital and community pharmacy. These changes in the care pathway have important organizational consequences for patients, their relatives, and health professionals. Two training programs are available for community pharmacists: the "HEMOPHAR" training program proposed by the national reference center for hemophilia and the Roche training program proposed by the laboratory that markets the product.

OBJECTIVE

The PASODOBLEDEMI study aims to evaluate the direct impact of the training programs provided to community pharmacists in the context of the dispensing of emicizumab, and to evaluate patients' satisfaction with their treatment whether they choose dispensation from a community pharmacy or retained dispensation from the hospital pharmacy.

METHODS

We designed a cross-sectional study based on the 4-level Kirkpatrick evaluation model: the immediate reaction of community pharmacists following training (Reaction), the knowledge acquired during the training (Learning), the professional practice of community pharmacists during dispensing of the product (Behavior), and patients' satisfaction related to the treatment whether it is dispensed from a hospital or from a community pharmacy (Results).

RESULTS

Considering that single outcome measures cannot adequately reflect the complexity of this new organization, the Kirkpatrick evaluation model provides 4 distinct outcomes: the immediate reaction after the HEMOPHAR training program, the level of knowledge acquired after the HEMOPHAR training program, the impact of training on professional practice, and patient satisfaction with access to emicizumab. We developed specialized questionnaires for each of the 4 levels of the Kirkpatrick evaluation model. All community pharmacists involved in dispensing emicizumab, whether they have followed the HEMOPHAR or the Roche training program or neither, were eligible for inclusion. All patients with severe hemophilia A were eligible, irrespective of inhibitor use, age, treatment with emicizumab, and whether they chose dispensation from a community pharmacy or retained dispensation from a hospital pharmacy.

CONCLUSIONS

The new organization for dispensing emicizumab to patients with hemophilia A in French community pharmacies must be accompanied by optimal safety and quality conditions due to the risk of serious and urgent bleeding situations in the management of rare bleeding diseases. The elaboration of the PASODOBLEDEMI protocol has already a positive impact with the commitment of all health professionals, physicians, hospital and community pharmacists, and the patient community. The results will be disseminated among the French authorities and will enable, if necessary, proposing this access model to other rare diseases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05449197, https://clinicaltrials.gov/ct2/show/NCT05449197?term=NCT05449197; ClinicalTrials.gov NCT05450640, https://clinicaltrials.gov/ct2/show/NCT05450640?term=NCT05450640.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/43091.

Applying the Kirkpatrick model to evaluate dental students' experience of learning about antibiotics use and resistance

INTRODUCTION

"Antibiotic resistance" is of main concern in global health and that it could hinder the achievement of sustainable development goals. One of the reported contributing factors is the irrational prescribing behaviour of healthcare professionals including dentists. Efforts to design and evaluate effective educational programmes for undergraduate dental students about appropriate prescribing behaviour during their early educational years could mitigate the risk of antibiotic resistance.

MATERIALS AND METHODS

A total of 322 students participated in the study. Their knowledge of antibiotics and antibiotic resistance was assessed. Students received a specially designed, interactive educational programme. The success of the programme was assessed after the educational sessions and 2 years later using Kirkpatrick's four-level evaluation model.

RESULTS

This study is the first study to use Kirkpatrick model to evaluate an educational programme for dental students. The results of the first level of evaluation showed an overall satisfaction score of 82.1%. The second level revealed an increase in the percentage of correct answers after the educational sessions from 68.3% to 80%, and significant agreement with responsible antibiotics usage (p-value = .020, Effect size = 0.121). The third level showed that the percentage of correct answers 2 years later was 87.5%. The fourth level confirmed the success of the programme as 88.9% of participants reported using the knowledge gained from the programme when prescribing antibiotics.

CONCLUSIONS

Our study emphasises the success of the used educational programme and highlights the need for educational interventions in the under graduate dental curriculum.

Evaluating the Effects of Training to Improve Teaching Skills of Health Sciences Educators in Sudan

Purpose

This evaluation draws evidence on the outcome of learning and teaching courses that were offered to a number of teaching staff (referred to as trainers) of the Academies of Health Sciences, Midwifery Schools and Centre for Continuous Professional Development in Sudan.

Methods

The evaluation was a cross-sectional, institution-based study conducted from October 2017 to January 2018 involving qualitative design. It consisted of direct observations of teaching, focus group discussions with students and semi-structured interviews with managers of teaching institutions.

Results

The findings of direct observations revealed that the learning and teaching course has positive effects on the trainers' ability to have clear, well-stated learning objectives; their presentation skills; and their use of different teaching methods. Moreover, the observations showed that trainers who attended the learning and teaching course now encourage the students to ask questions and are providing them with timely feedback about their learning. As students were the main beneficiaries for improved teaching, focus group discussions have shown that students were generally satisfied with trainers' performance regarding the stated learning outcomes, curricula design, use of a range of teaching methods and assessment methods. During in-depth interviews, managers of training institutions expressed satisfaction with the trainers' performance regarding the development of training materials, learners' assessment, supervision, and evaluation of training.

Conclusion

Short courses on teaching and learning were a valuable investment for trainers, students, and the overall performance of health sciences training institutions. The evaluation revealed that trainers were able to apply what they have learnt. Moreover, students and managers noticed improvement in the performance of trainers in teaching, course delivery and assessment. It is recommended to expand, cascade and institutionalize the short courses on learning and teaching to all states to improve the capabilities of trainers, which would eventually contribute to the production of competent allied health professions in Sudan.

Academic Half-Day Education Experience in Post-graduate Medical Training: A Scoping Review of Characteristics and Learner Outcomes

Background

The academic half-day (AHD) has grown in popularity for medical education because it intends to provide learners with uninterrupted, immersive learning time that may promote participant attendance, engagement, and knowledge. Little is known about the extent of use, forms, or effectiveness of AHD in Post-graduate medical education. This scoping review summarizes existing literature and describes the learning outcomes, according to the Kirkpatrick model of learning evaluation, of AHD experiences on Post-graduate medical trainees.

Methods

Authors used Arksey and O'Malley's methodological framework, searching electronic scientific literature databases from the years of 1977-2019 with relevant key terms and identifying 735 papers. Two independent raters completed title/abstract screening and then extracted pertinent data from papers meeting specified criteria.

Results

Authors identified 38 relevant papers published in English, originating from programs in US (n = 19) and Canada (n = 19), spanning 4 disciplines: Medicine (n = 17, 45%), Pediatrics (n = 10, 26%), Critical Care/Surgery (n = 9, 24%), Radiology (n = 2, 5%). A majority (n = 33, 87%) described specific educational experiences; most focused on residents only (n = 27). The educational experiences included various teaching strategies; few were didactics only (n = 4) and most were multi-modal including simulation, case-based learning, problem-based learning, and/or self-directed online study. AHD size ranged from 5 to 364 participants (median 39). AHD length was 1.5-6 h (median 3). Required resources were inconsistently described. When evaluations of the specific educational experience were reported (n = 35 studies), the majority of studies used weak research designs (e.g., one group, pre/post-test, n = 19); few studies used strong research designs (e.g., randomized controlled trial, n = 2). Positive effects of AHD ranged across Kirkpatrick levels 1-3 learner outcomes.

Conclusions

The composition and content of AHD in Post-graduate medical education vary. Few studies of AHD use stringent research designs, and none include learner outcome measures at the highest Kirkpatrick level (i.e., level 4 results/patient outcomes). A consensus definition and further high-quality research on AHD in Post-graduate medical education is needed.

Teaching evidence-based medicine with electronic databases for preclinical education

To evaluate the efficacy of an educational module on evidence-based medicine (EBM) assisted with electronic medical databases (EMDs) for preclinical education, medical students (n = 111) were matriculated in a program consisted of 16 2-h sessions on EBM plus hands-on experience on EMDs in a problem-based learning-type format. Students were required to make an oral presentation on designated clinical scenarios before and after the sessions, without prior notice, as an indicator of performance. In addition, questionnaires focusing on behavioral changes, awareness, and confidence of mastering EBM were administered before and after the sessions to assess the attitudinal and behavioral impact of the intervention on the participants. We found evidence of better postprogram performance in utilizing EBM-relevant concepts and resources when the enrolled medical students were giving oral presentations. Moreover, the participants reported increased awareness of EBM and, behaviorally, increased utilization of EBM-relevant resources provided by libraries. Also, they reported improvement on appropriately using EBM-relevant resources, and 99% of the participants reported strong confidence in practicing EBM. In conclusion, modules on EBM implemented with EMDs benefitted medical students in scenario-oriented PBL tutorials. Improvements in awareness, behavior, confidence, and performance in mastering EBM were noted.

Evaluating the effect of an in-service training workshop on ICD-10 coding instructions of pregnancy, childbirth and the puerperium for clinical coders

The quality of the data coded based on the 10^th revision of the International Classification of Diseases (ICD-10) can be improved by providing continuous education and promoting the clinical coders’ knowledge and skills. Due to the significance of maternal health in promoting the health of society, the present study evaluated the effects of an in-service training workshop on ICD-10 coding instructions of pregnancy, childbirth, and the puerperium for clinical coders. This applied evaluation study was conducted to evaluate the effects of a coding instructions training course focusing on the 15^th chapter of the ICD-10. The statistical population comprised 45 clinical coders working in the hospitals. The data were collected by a researcher-made questionnaire scored on a five-point Likert scale at the reaction level and by pretest and posttest questionnaires at the learning level. The data were then analyzed by descriptive statistics at the reaction level and by a paired-samples t-test at the learning level. The participants’ satisfaction with the training course was 94.7% on average at the reaction level. At the learning level, the results of the paired-samples t-test showed a significant difference between the means of scores before and after the training course (p=0.000). The training course led to satisfaction and enhanced the capabilities of the clinical coders with regard to coding the 15^th chapter of ICD-10. Clinical Coders must receive training on the new changes and guidelines in the other chapters of ICD-10 based on its most recent revision and employ them in the workplace.

Implementation research training for learners in low- and middle-income countries: evaluating behaviour change after participating in a massive open online course

BACKGROUND

Implementation research (IR) can play a critical role in the delivery of disease control interventions, particularly in low- and middle-income countries (LMICs). The growing demand for IR training has led to the development of a range of training programmes and university courses, the majority of which can not be accessed by learners in LMICs. This article reports on the evaluation of the massive open online course (MOOC) developed by the Special Programme for Research and Training in Tropical Diseases hosted by WHO on the topic of IR with a focus on infectious diseases of poverty. This study followed the Kirkpatrick Model to evaluate training programmes with a specific focus on post-training changes in behaviour.

METHODS

MOOC participants were invited to take part in an anonymous online survey examining their knowledge of IR and how they applied it in their professional practice approximately 1-1.5 years after completing their course. The survey contained 43 open-ended, multiple choice and Likert-type questions. Descriptive statistics were calculated for the quantitative data and responses to the open-ended questions were thematically coded.

RESULTS

A total of 748 MOOC participants responded to the survey. The demographic profile of the survey respondents aligned with that of the MOOC participants, with nearly 70% of respondents originating from Africa. Responses to the quantitative and open-ended survey questions revealed that respondents' knowledge of IR had improved to a large extent as a result of the MOOC, and that they used the knowledge and skills gained in their professional lives frequently and had consequently changed their professional behaviour. Respondents most often cited the problem-solving aspect of IR as a substantial area of behavioral change influenced by participating in the MOOC.

CONCLUSIONS

These findings indicate that the MOOC was successful in targeting learners from LMICs, in strengthening their IR knowledge and contributing to their ability to apply it in their professional practice. The utility of MOOCs for providing IR training to learners in LMICs, where implementation challenges are encountered often, makes this platform an ideal standalone learning tool or one that could be combined with other training formats.

Leveraging gains from African Center for Integrated Laboratory Training to combat HIV epidemic in sub-Saharan Africa

Background

In sub-Saharan Africa, there is dearth of trained laboratorians and strengthened laboratory systems to provide adequate and quality laboratory services for enhanced HIV control. In response to this challenge, in 2007, the African Centre for Integrated Laboratory Training (ACILT) was established in South Africa with a mission to train staffs from countries with high burdens of diseases in skills needed to strengthen sustainable laboratory systems. This study was undertaken to assess the transference of newly gained knowledge and skills to other laboratory staff, and to identify enabling and obstructive factors to their implementation.

Methods

We used Kirkpatrick model to determine training effectiveness by assessing the transference of newly gained knowledge and skills to participant’s work environment, along with measuring enabling and obstructive factors. In addition to regular course evaluations at ACILT (pre and post training), in 2015 we sent e-questionnaires to 867 participants in 43 countries for course participation between 2008 and 2014. Diagnostics courses included Viral Load, and systems strengthening included strategic planning and Biosafety and Biosecurity. SAS v9.44 and Excel were used to analyze retrospective de-identified data collected at six months pre and post-training.

Results

Of the 867 participants, 203 (23.4%) responded and reported average improvements in accuracy and timeliness in Viral Load programs and to systems strengthening. For Viral Load testing, frequency of corrective action for unsatisfactory proficiency scores improved from 57 to 91%, testing error rates reduced from 12.9% to 4.9%; 88% responders contributed to the first national strategic plan development and 91% developed strategies to mitigate biosafety risks in their institutions. Key enabling factors were team and management support, and key obstructive factors included insufficient resources and staff’s resistance to change.

Conclusions

Training at ACILT had a documented positive impact on strengthening the laboratory capacity and laboratory workforce and substantial cost savings. ACILT’s investment produced a multiplier effect whereby national laboratory systems, personnel and leadership reaped training benefits. This laboratory training centre with a global clientele contributed to improve existing laboratory services, systems and networks for the HIV epidemic and is now being leveraged for COVID-19 testing that has infected 41,332,899 people globally.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-020-06005-8.

[Examination of the effects of a role-playing training program for the improvement of planning and management skills of public health nurses]

Objectives　The Japan Health Insurance Association (JHIA) conducts training in each branch facility to improve the ability of public health nurses (PHNs). The headquarters of the PHNs of JHIA and the researchers conducted a training program for leaders of PHNs at each JHIA branch. The goal of the program was to create a training plan using role-play to acquire facilitation skills. This study aimed to examine the effects of training.Methods　The study was designed in accordance with the Instructional Designs. The training goals were as follows: (1) understanding the role of facilitation and the facilitator in the debriefing session after the role-play, (2) understanding facilitation techniques, (3) being confident in performing as facilitators in the debriefing session; and (4) conducting the debriefing sessions using facilitation skills. The evaluation of the training was based on the Kirkpatrick model from the viewpoint of confidence in using facilitation, knowledge of facilitation, and conduct of training and utilization of facilitation technology. Questionnaire evaluations were conducted three times before the training, immediately after the training, and three months after the training. In August 2016, 4.5 hours of training were conducted in one day.Results　There were 79 participants in the training group. The mean points of knowledge and confidence were 2.6-3.6 before training, 6.3-7.9 after training, and 6.0-6.9 at 3 months after training. The participants rated their interest in three questions of the training as high as 8.1-8.6. In addition, 64.6% of participants held a role-play session at each branch within three months of the initial training. In the role-play session planned by the participants, the practitioners implemented the explanation of the purpose and the rules at each branch 96.1% and 98.0%, respectively. Participants who had attended facilitation training prior to our program scored higher points of knowledge and confidence before and after three months. Three months after the training, 79 participants responded to the question of the role of the facilitator in role-playing. The descriptions were categorized into "opinions on the basics and planning of role-play training" and "opinions on the roles in retrospectives."Conclusion　The participants evaluated the training contents and the materials used as appropriate, and their knowledge and confidence in facilitation improved after the training. To maintain and improve facilitation skills, the educational system needs to implement training using role-playing at each branch consistently.

Supporting migrants and refugees with posttraumatic stress disorder: development, pilot implementation, and pilot evaluation of a continuing interprofessional education for healthcare providers

BACKGROUND

Refugees and migrants face an increased risk of developing posttraumatic stress disorder (PTSD). Adequate care can be insufficient due to language barriers, cultural differences, and knowledge deficits of health service providers. Therefore, professional associations requested that healthcare providers to be educated to provide culturally sensitive care. An evidence-based educational intervention in the form of a continuing interprofessional education (CIPE) for healthcare providers on the topic of PTSD in migrants and refugees was developed, pilot-implemented, and evaluated according to the first two levels of the Kirkpatrick evaluation model (reaction and learning).

METHODS

The development of a curriculum for the CIPE intervention was based on a narrative literature review. Its content was validated by experts (N = 17) in an online survey and analyzed using both the Content Validity Index and a thematic analysis. The evaluation of the CIPE intervention was performed by conducting a pilot study with a quasi-experimental single group, using a pre-posttest design. In total, there were 39 participants distributed among three pilot courses. We collected and analyzed data on satisfaction, knowledge, and feasibility.

RESULTS

The curriculum for a half-day course, consisting of 8 modules, showed almost excellent content validity (S-CVI = 0.92). In the pilot-implementation phase, participants were "very satisfied" with the pilot courses and a positive effect on their knowledge was detected. No correlation between satisfaction and knowledge gain was found.

CONCLUSIONS

The CIPE intervention can be considered feasible and seems promising in its effects on satisfaction and knowledge. The insights gained in this study can be used to adapt and optimize the educational intervention, whereby the feedback from course attendees is particularly useful. Future studies need to further examine the effects in larger samples and more robust study designs.

Study design and protocol for a comprehensive evaluation of a UK massive open online course (MOOC) on quality improvement in healthcare

INTRODUCTION

Massive open online courses (MOOCs) offer a flexible approach to online and distance learning, and are growing in popularity. Several MOOCs are now available, to help learners build on their knowledge in a number of healthcare topics. More research is needed to determine the effectiveness of MOOCs as an online education tool, and explore their long-term impact on learners' professional practice. We present a protocol describing the design of comprehensive, mixed-methods evaluation of a MOOC, 'QualityImprovement (QI) inHealthcare: the Case for Change', which aims to improve learner's knowledge and understanding of QI approaches in healthcare, and to increase their confidence in participating, and possibly leading QI projects.

METHODS AND ANALYSIS

A pre-post study design using quantitative and qualitative methods will be used to evaluate the QI MOOC. Different elements of the RE-AIM (reach, effectiveness and maintenance) and Kirkpatrick (reaction, learning and behaviour) models will be used to guide the evaluation. All learners who register for the course will be invited to participate in the QI MOOC evaluation study. Those who consent will be asked to complete a presurvey to assess baseline QI knowledge (self-report and objective) and perceived confidence in engaging in QI activities. On completion of the course, participants will complete a postsurvey measuring again knowledge and perceived confidence. Feedback on the course content and how it can be improved. A subset of participants will be invited to take part in a follow-up qualitative interview, 3 months after taking the course, to explore in depth how the MOOC impacted their behaviour in practice.

ETHICS AND DISSEMINATION

The study has been approved by the University of Bath Human Research Ethics Committee (reference: 2958). Study findings will be published in peer-reviewed journals, and disseminated at conference and departmental presentations, and more widely using social media, microblogging sites and periodicals aimed at healthcare professionals.

Evaluation of a drug and alcohol safety education program in aviation using interrupted time series and the Kirkpatrick framework

Compliance with drug and alcohol regulations are required by 14 CFR Part 120/ and 49 CFR Part 40. These regulations affect approximately 7200 aviation-related companies and their associated services. Consequences for noncompliance can include loss of revenue from imposition of civil penalties, suspension, or revocation of the company's certificate to conduct business. Front End Analysis (FEA) was conducted to determine specific performance problems and provide five tailored educational interventions to address them. Program evaluation was conducted using Interrupted Time Series (ITS) modeling. Results showed significant (p < .05) decreases in nonconformities across all five models with small to moderate effect sizes. Based on the relative effects, values for reductions in civil penalty costs between 16% and 47%, were predicted. Actual sanction reductions from the pre-to-post-intervention periods were confirmed to be 24.21%. The study supported the efficacy of the ITS approach for implementation of level four Kirkpatrick evaluation.

A pilot study on the simulation-based training for Ethiopia skilled birth attendants

This simulation-based Ethiopia Midwife Train-the-Trainer program for skilled birth attendants is one part of the China-UK Global Health Support Program (GHSP) activities and an approach to improving the Ethiopia trainers' midwifery teaching skills. This pilot study aimed to evaluate the effectiveness of a simulation-based training for Ethiopia skilled birth attendants. This program was divided into three stages. In Stage 1, Fudan team trained 5 trainers from Ethiopia. In Stage 2, Fudan team supervised and evaluated the 1st stage Ethiopia trainers to train their peers becoming the trainers in Fudan University. In Stage 3, Fudan team evaluated the training program implementation in established Advanced Midwifery Practice Center in Tirunesh Beijing General Hospital. The participatory observational method was used for process evaluation and the Kirkpatrick model was used for effectiveness evaluation. The Ethiopia staff's teaching skills for simulation-based midwifery training were strengthened and their abilities to deal with various complicated obstetric situations were improved by group explanation, questions, rehearsal, and step-by-step interactive teaching (p < .01). The mean score of total mastery degree was 3.48 ± 0.16 by self-assessment and 3.63 ± 0.16 by peer-assessment in stage 2, and 3.27 ± 0.24 by self-assessment and 3.38 ± 0.34 by peer-assessment in stage 3.The GHSP-OP4-V01 Ethiopia Midwife Train-the-Trainer Program has received positive response in Ethiopia by in-depth interview and questionnaire survey. This study demonstrated that the simulation-based training is an effective approach for midwifery Train-the-Trainer program in Ethiopia.

The evaluation of a home-based paediatric nursing service: concept and design development using the Kirkpatrick model

Background

A new paediatric hospital-in-the-home nursing service required evaluation.

Aims

To determine whether the education and training provided for nursing staff employed in the service was effective.

Methods

This paper presents the way in which a training evaluation model supported the design and evaluation of a training programme for registered nurses working in an out-of-hospital, home-based nursing service for paediatric patients.

Results

The Kirkpatrick model provides a framework for evaluating the effectiveness of workforce training for any industry including healthcare (Kirkpatrick, 2009).

Conclusions

That the Kirkpatrick model is an appropriate framework to evaluate a nursing training programme, but it is imperative to evaluate all levels of the model to be able to ascertain the success of the training and the impact on clinical practice.

Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model

PURPOSE

Medical professionals from Korea and Laos have been working together to develop a continuing professional development training program covering the major clinical fields of primary care. This study aimed to evaluate the effectiveness of the program from 2013 to 2014 using the Kirkpatrick model.

METHODS

A questionnaire was used to evaluate the reaction of the trainees, and the trainers assessed the level of trainees' performance at the beginning and the end of each clinical section. The transfer (behavioral change) of the trainees was evaluated through the review of medical records written by the trainees before and after the training program.

RESULTS

The trainees were satisfied with the training program, for which the average score was 4.48 out of 5.0. The average score of the trainees' performance at the beginning was 2.39 out of 5.0, and rose to 3.88 at the end of each section. The average score of the medical records written before the training was 2.92 out of 5.0, and it rose to 3.34 after the training. The number of patient visits to the district hospitals increased.

CONCLUSION

The continuing professional development training program, which was planned and implemented with the full engagement and responsibility of Lao health professionals, proved to be effective.

Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation

Background:

There are several evaluation models that can be used to evaluate the effect of in-service training; one of them is the Kirkpatrick model. The aim of the present study is to assess the in-service training of cardiopulmonary resuscitation (CPR) for nurses based on the Kirkpatrick's model.

Materials and Methods:

This study is a cross-sectional study based on the Kirkpatrick's model in which the efficacy of in-service training of CPR to nurses was assessed in the Shahadaye Lenjan Hospital in Isfahan province in 2014. 80 nurses and Nurse's aides participated in the study after providing informed consent. The in-service training course was evaluated in reaction, learning, behavior, and results level of the Kirkpatrick model. Data were collected through a researcher-made questionnaire.

Results:

The mean age of the participants was 35 ± 8.5 years. The effectiveness score obtained in the reaction level (first level in the Kirkpatrick model) was 4.2 ± 0.32. The effectiveness score in the second level of model or the learning level was 4.70 ± 0.09, which is statistically significant (P < 0.001). The effectiveness score at the third and fourth level were 4.1 ± 0.34 and 4.3 ± 0.12, respectively. Total effectiveness score was 4.35.

Conclusions:

The results of this study showed that CPR in-service training has a favorable effect on all four levels of the Kirkpatrick model for nurses and nurse's aides.