Evaluating the effect of interventions for strengthening non-physician anesthetists' education in Ethiopia: a pre- and post-evaluation study

Does higher performance in a national licensing examination predict better quality of care? A longitudinal observational study of Ethiopian anesthetists

BACKGROUND

Ethiopia made a national licensing examination (NLE) for associate clinician anesthetists a requirement for entry into the practice workforce. However, there is limited empirical evidence on whether the NLE scores of associate clinicians predict the quality of health care they provide in low-income countries. This study aimed to assess the association between anesthetists' NLE scores and three selected quality of patient care indicators.

METHODS

A multicenter longitudinal observational study was conducted between January 8 and February 7, 2023, to collect quality of care (QoC) data on surgical patients attended by anesthetists (n = 56) who had taken the Ethiopian anesthetist NLE since 2019. The three QoC indicators were standards for safe anesthesia practice, critical incidents, and patient satisfaction. The medical records of 991 patients were reviewed to determine the standards for safe anesthesia practice and critical incidents. A total of 400 patients responded to the patient satisfaction survey. Multivariable regressions were employed to determine whether the anesthetist NLE score predicted QoC indicators.

RESULTS

The mean percentage of safe anesthesia practice standards met was 69.14%, and the mean satisfaction score was 85.22%. There were 1,120 critical incidents among 911 patients, with three out of five experiencing at least one. After controlling for patient, anesthetist, facility, and clinical care-related confounding variables, the NLE score predicted the occurrence of critical incidents. For every 1% point increase in the total NLE score, the odds of developing one or more critical incidents decreased by 18% (aOR = 0.82; 95% CI = 0.70 = 0.96; p = 0.016). No statistically significant associations existed between the other two QoC indicators and NLE scores.

CONCLUSION

The NLE score had an inverse relationship with the occurrence of critical incidents, supporting the validity of the examination in assessing graduates' ability to provide safe and effective care. The lack of an association with the other two QoC indicators requires further investigation. Our findings may help improve education quality and the impact of NLEs in Ethiopia and beyond.

Assessment of clinical competence of graduating medical students and associated factors in Ethiopia

BACKGROUND

Ethiopia has scaled up medical education to improve access to healthcare which presented challenges to maintaining training quality. We conducted a study to assess the clinical competence of graduating medical students and the associated factors.

METHODS AND MATERIALS

A pretest assessment of a quasi-experimental study was conducted in 10 medical schools with a sample size of 240 students. We randomly selected 24 students per school. Clinical competence was assessed in a 12-station objective structured clinical examination. The clinical learning environment (CLE), simulation training, and practice exposure were self-rated. Mean scores for clinical competence, and satisfaction in the CLE and simulation training were calculated. Proportions of students with practice exposure, and who agreed on CLE and simulation items were done. Independent t-tests were used to look at competence differences among subgroups. Bivariate and multiple linear regression models were fitted for the outcome variable: competence score. A 95% statistical confidence interval and p-value < 0.05 were used for making statistical decisions. A 75% cut-off score was used to compare competence scores.

RESULTS

Graduating medical students had a mean competence score of 72%. Low scores were reported in performing manual vacuum aspiration (62%), lumbar puncture (64%), and managing childbirth (66%). Female students (73%) had a significantly higher competence score than males (70%). Higher cumulative grade point average (CGPA), positive appraisal of the CLE, and conducting more clinical procedures were associated with greater competence scores. Nearly half of the students were not satisfied with the clinical practice particularly due to the large student number and issues affecting the performance assessment. About two-thirds of the students were not satisfied with the sufficiency of models and equipment, and the quality of feedback during simulation training. Nearly one-third of the students never performed lumbar puncture, manual vacuum aspiration, and venipuncture.

CONCLUSIONS

Medical students had suboptimal clinical competence. A better clinical learning environment, higher cumulative GPA, and more practice exposure are associated with higher scores. There is a need to improve student clinical practice and simulation training. Strengthening school accreditation and graduates' licensing examinations is also a way forward.

Ethiopian Anesthetist Licensing Examination: A Qualitative Study of Concerns and Unintended Consequences

Background

Ethiopia increased its anesthesia workforce drastically by expanding the training of associate clinician anesthetists. Following this expansion, the Ministry of Health established an entry-level anesthesia licensing examination to ensure patient safety. However, there is limited empirical evidence on the impacts of licensing exams in low- and middle-income countries. This study aimed to explore the concerns and undesirable consequences of the anesthetist licensing examination in Ethiopia.

Methods

A qualitative design using a grounded theory approach was employed by collecting data from 10 anesthesia teaching institutions. We conducted 15 in-depth interviews with instructors and six focus groups with students and graduates who took the exam recently. Interviews and focus groups were audio-recorded, transcribed verbatim, and analyzed using Atlas.ti 23. We also extracted secondary data from the academic committee meeting minutes, curricula, faculty appraisal reports, and program quality self-review reports.

Results

Qualitative analysis revealed three central categories of concerns and untoward consequences of the anesthetist NLE: exam management, educational management, and student behavior. Exam management concerns were related to exam validity, fairness, and consistent enforcement of pass/fail decisions. The unintended consequences of the exam on education management were perceived as promoting teaching and learning for the exam, increasing faculty workload, and resulting in superficial and patchy educational reforms. Study participants also reported adverse psychosocial effects and increased cheating behaviors among students as undesirable consequences of the exam on student behavior.

Conclusion

Our study identified some concerns and unintended consequences of the Ethiopian anesthetist licensing examination. These lessons learned may contribute to improving the quality of licensing examinations in Ethiopia and beyond.

The Ethiopian Anesthetist Licensing Examination and Associated Improvement in in-School Student Performance: A Retrospective Study

Background

Ethiopia introduced a national licensing examination (NLE) in response to growing concerns about the competence of graduates and the quality of education. This study aimed to assess the associated in-school student performance changes in anesthetist training programs following NLE implementation.

Methods

Academic records of 1493 graduate anesthetists were retrospectively obtained from eight universities before (n=932) and after (n=561) NLE implementation. Four universities were first-generation (oldest), three were second-generation, and one was third-generation (newest). We compared the yearly (Y1 to Y4) and cumulative grade point averages (GPA) to assess if there were in-school student performance differences between the two periods. The Kruskal-Wallis and Mann-Whitney U-tests were used to compare groups. Results are presented as a median, interquartile range, a 95% confidence interval (CI) for median differences, and Cohen's r effect size.

Results

Overall, there was a small to moderate improvement in student academic performance following NLE implementation. However, the statistically significant differences were limited to first-generation university students and those entering directly from high school. We found considerable positive differences in all five performance measures in first-generation university students, with Year-1 GPA and cumulative GPA measurements exhibiting large effect sizes (Cohen's r = 0.96 and 0.79, respectively, p <0.005). Those entering from high school demonstrated significant differences in four of five performance measures, with the largest positive gains on the year-1 GPA (median before [n=765] and after [n=480]: 3.11-3.30, 95% CI (0.09, 0.22), r=0.46, p <0.005)). Second- and third-generation university students showed no significant differences, while nurse entrants exhibited a significant difference in their Y2GPA scores only with an actual drop in performance.

Conclusion

The Ethiopian anesthetist NLE is associated with an overall modest in-school academic performance improvement, supporting its use. The stagnant or declining performance among nurse entrants and the newest (second and third) generation university students deserve further scrutiny.

Impact of anesthetist licensing examination on quality of education in Ethiopia: a qualitative study of faculty and student perceptions

BACKGROUND

Ethiopia drastically increased the anesthesia workforce density by training 'associate clinician anesthetists' as a task-shifting and sharing strategy. However, there were growing concerns about educational quality and patient safety. Accordingly, the Ministry of Health introduced the anesthetist national licensing examination (NLE) to assure the quality of education. However, empirical evidence is scarce to support or refute the overall impact of NLEs, which are relatively costly for low- and middle-income settings. Therefore, this study aimed to explore the impact of introducing NLE on anesthetists' education in Ethiopia.

METHODS

We conducted a qualitative study using a constructivist grounded theory approach. Data were prospectively collected from ten anesthetist teaching institutions. Fifteen in-depth interviews were conducted with instructors and academic leaders, and six focus groups were held with students and recently tested anesthetists. Additional data were gathered by analyzing relevant documents, including versions of curricula, academic committee minutes, program quality review reports, and faculty appraisal reports. Interviews and group discussions were audiotaped, transcribed verbatim and analyzed using Atlas.ti 9 software.

RESULTS

Both faculty and students demonstrated positive attitudes toward the NLE. Student motivation, faculty performance, and curriculum strengthening were the three primary changes that emerged, resulting in three subsequent spin-offs on assessment, learning, and quality management practices. Academic leaders' dedication to evaluating examination data and turning these into action led to changes that improved education quality. Increased accountability, engagement, and collaboration were the predominant factors facilitating change.

CONCLUSION

Our study indicates that the Ethiopian NLE has prompted anesthesia teaching institutions to improve their teaching, learning, and assessment practices. However, more work is required to improve exam acceptability among stakeholders and drive broader changes.

Organizational peer support to enable rehabilitating surgical services in Northern Ethiopia

The ongoing violent conflict in Northern Ethiopia has caused displacement, death, and destruction. Health services infrastructure became one of the primary victims of the war, leaving millions unable to access essential surgical health services at a time when demand for surgical interventions is on the rise. Rehabilitating surgical services was identified as a priority by the federal government, regional health bureaus, and humanitarian organizations, forming an integral part in rebuilding communities after war. Under the auspices of the Federal Ministry of Health of Ethiopia, a hospital twinning program between providers in non-conflict and conflict affected areas was first introduced in December 2021, now including 13 active partnerships. The program builds on a previous best practice gained from the Ethiopian Hospital Alliance for Quality to strengthen local health care providers in regaining capabilities to serve local populations. Field experience of two hospital twinning projects have shown significant scope of organizational peer support at times of crisis, successfully enabling conflict-afflicted hospitals to regain the capacity necessary to re-introduce surgical services. While overcoming challenges such as lack of basic supplies including electricity and blood may be required to further increase the scope of this program in Northern Ethiopia, relative success highlights important lessons for similar approaches in areas affected by conflict, or natural disasters.

Establishing a sustainable training programme in anaesthesia in Ethiopia

Background

Lack of qualified staff is a major hindrance for quality and safety improvements in anaesthesia and critical care in many low‐income countries. Support in specialist training may enhance perioperative treatment and have a positive downstream impact on other hospital services, which may improve the overall standard of care.

Methods

Between 2011 and 2019, consultant anaesthetists from Haukeland University Hospital in Norway supported a postgraduate anaesthesia‐training programme at Addis Ababa University/Tikur Anbessa Specialised Hospital in Ethiopia. The aim of the programme was to build a self‐sustainable work force of anaesthetists across the country who could perform high quality anaesthesia within the confinement of limited local resources. Over the course of 10 years, an almost continuous rotation of experienced anaesthetists and intensivists assisted training of Ethiopian residents in anaesthesia and critical care. Local specialists organised the programme; however, external support was necessary during this period to establish a sustainable training programme.

Results

Since the programme's commencement at Addis Ababa University in 2011, 159 residents have entered the programme and 71 have graduated. As the number of qualified anaesthetists increased, Ethiopian specialists gradually obtained responsibility for the programme. Candidates are recruited from various regions and from neighbouring countries. Five other Ethiopian training sites have been established. To date (May 2022), 112 residents have completed their training in Ethiopia, and 195 residents expect to graduate within 3 years.

Conclusion

Nearly 11 years after establishment of the programme, locally trained highly qualified anaesthetists work in Ethiopia's major hospitals throughout the country.