How changing quality management influenced PGME accreditation: a focus on decentralization and quality improvement

Enabling Implementation of Competency Based Medical Education through an Outcomes-Focused Accreditation System

Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.

Implementing quality management strategies improves clinical quality as a voluntary medical male circumcision program in Namibia matures: a process analysis

BACKGROUND

Surgical voluntary medical male circumcision (VMMC) is a safe procedure; however, maintaining quality standards at scale, particularly during scale-up, is a challenge making ongoing quality management (QM) efforts essential. This study describes program quality measured by rates of adverse events (AEs) over four years of VMMC implementation in Namibia, compares AE rates over time, and discusses QM processes that contextualize AE trends and illustrate improvements in quality as the program matured. The International Training and Education Center for Health (I-TECH) assisted the Namibian Ministry of Health and Social Services (MoHSS) in expanding VMMC in three regions among boys and men over 10 years of age between January 2015 and September 2019.

METHODS

A comprehensive package of QM strategies was implemented by multi-disciplinary onsite teams with support from national and international technical advisors. Retrospective routine MoHSS data from the VMMC register, client forms, and monthly AE reports were collected during implementation in the three regions to assess the impact of QM interventions on AEs and to calculate the proportion of clients who experienced AEs over time. The proportion of clients who experienced an AE over time was compared using a Cochran-Armitage test for trend.

RESULTS

Between January 2015 and September 2019, 40,336 clients underwent VMMC and 593 (1.5%) clients experienced a post-operative AE in the three supported regions. The AE rate was highest in the first quarter of clinical service delivery in each region (January-March 2015 in Oshana and Zambezi, October-December 2017 in //Kharas) but declined over the implementation period as the program matured. This observed trend between program maturity and declining AE rates over time was significant (p < 0.001) when compared using a Cochran-Armitage test for trend.

CONCLUSIONS

As the I-TECH-supported VMMC program matured, QM measures were introduced and routinized, and clinical quality improved over time with the rate of AEs decreasing significantly over the implementation period. Applying systematic and continuous QM processes and approaches across the continuum of VMMC services and considering local context can contribute to increased clinical safety. QM measures that are established in more mature program sites can be quickly adopted to respond to quality issues in program expansion sites.

Postgraduate medical education quality metrics panels can be enhanced by including learner outcomes

Postgraduate medical education (PME) quality assurance at Health Education England (HEE) currently relies upon survey data. As no one metric can reflect all aspects of training, and each has its limitations, additional metrics should be explored. At HEE (West Midlands), we explored the use of learner outcomes, speciality examination pass rates and Annual Review of Competence Progression (ARCP) outcomes, as quality metrics. Feedback received from our local Quality Forum of 40 senior educators frames the discussion through this paper. Overall, learner outcomes are useful quality metrics that add to survey data to provide a more comprehensive picture of PME quality. However, the utility of ARCP outcomes as quality metrics is currently limited by concerns regarding variations in ARCP practice between regions. To address these concerns, ARCPs need the same processes, rigour, scrutiny and investment as other high-stakes assessments. This will improve the reliability and validity of the ARCP as an assessment and improve the usefulness of ARCP outcomes as quality metrics. Research is required to determine the optimal combination of metrics to use in PME quality assurance and to appraise the validity and reliability of the ARCP as an assessment.

A "fit for purpose" framework for medical education accreditation system design

BACKGROUND

Accreditation is a key feature of many medical education systems, helping to ensure that programs teach and assess learners according to applicable standards, provide optimal learning environments, and produce professionals who are competent to practise in challenging and evolving health care systems. Although most medical education accreditation systems apply similar standards domains and process elements, there can be substantial variation among accreditation systems at the level of design and implementation. A discussion group at the 2013 World Summit on Outcomes-Based Accreditation examined best practices in health professional education accreditation systems and identified that the literature examining the effectiveness of different approaches to accreditation is scant. Although some frameworks for accreditation design do exist, they are often specific to one phase of the medical education continuum.

MAIN TEXT

This paper attempts to define a framework for the operational design of medical education accreditation that articulates design options as well as their contextual and practical implications. It assumes there is no single set of best practices in accreditation system development but, rather, an underlying set of design decisions. A "fit for purpose" approach aims to ensure that a system, policy, or program is designed and operationalized in a manner best suited to local needs and contexts. This approach is aligned with emerging models for education and international development that espouse decentralization.

CONCLUSION

The framework highlights that, rather than a single best practice, variation among accreditation systems is appropriate provided that is it tailored to the needs of local contexts. Our framework is intended to provide guidance to administrators, policy-makers, and educators regarding different approaches to medical education accreditation and their applicability and appropriateness in local contexts.

The role of accreditation in 21st century health professions education: report of an International Consensus Group

BACKGROUND

Accreditation is considered an essential ingredient for an effective system of health professions education (HPE) globally. While accreditation systems exist in various forms worldwide, there has been little written about the contemporary enterprise of accreditation and even less about its role in improving health care outcomes. We set out to 1) identify a global, contemporary definition of accreditation in the health professions, 2) describe the relationship of educational accreditation to health care outcomes, 3) identify important questions and recurring issues in twenty-first century HPE accreditation, and 4) propose a framework of essential ingredients in present-day HPE accreditation.

METHODS

We identified health professions accreditation leaders via a literature search and a Google search of HPE institutions, as well as by accessing the networks of other leaders. These leaders were invited to join an international consensus consortium to advance the scholarship and thinking about HPE accreditation. We describe the consensus findings from the International Health Professions Accreditation Outcomes Consortium (IHPAOC).

RESULTS

We define accreditation as the process of formal evaluation of an educational program, institution, or system against defined standards by an external body for the purposes of quality assurance and enhancement. In the context of HPE, accreditation is distinct from other forms of program evaluation or research. Accreditation can enhance health care outcomes because of its ability to influence and standardize the quality of training programs, continuously enhance curriculum to align with population needs, and improve learning environments. We describe ten fundamental and recurring elements of accreditation systems commonly found in HPE and provide an overview of five emerging developments in accreditation in the health professions based on the consensus findings.

CONCLUSIONS

Accreditation has taken on greater importance in contemporary HPE. These consensus findings provide frameworks of core elements of accreditation systems and both recurring and emerging design issues. HPE scholars, educators, and leaders can build on these frameworks to advance research, development, and operation of high-quality accreditation systems worldwide.

Evaluation of continuous quality improvement in accreditation for medical education

BACKGROUND

Accreditation systems are based on a number of principles and purposes that vary across jurisdictions. Decision making about accreditation governance suffers from a paucity of evidence. This paper evaluates the pros and cons of continuous quality improvement (CQI) within educational institutions that have traditionally been accredited based on episodic evaluation by external reviewers.

METHODS

A naturalistic utility-focused evaluation was performed. Seven criteria, each relevant to government oversight, were used to evaluate the pros and cons of the use of CQI in three medical school accreditation systems across the continuum of medical education. The authors, all involved in the governance of accreditation, iteratively discussed CQI in their medical education contexts in light of the seven criteria until consensus was reached about general patterns.

RESULTS

Because institutional CQI makes use of early warning systems, it may enhance the reflective function of accreditation. In the three medical accreditation systems examined, external accreditors lacked the ability to respond quickly to local events or societal developments. There is a potential role for CQI in safeguarding the public interest. Moreover, the central governance structure of accreditation may benefit from decentralized CQI. However, CQI has weaknesses with respect to impartiality, independence, and public accountability, as well as with the ability to balance expectations with capacity.

CONCLUSION

CQI, as evaluated with the seven criteria of oversight, has pros and cons. Its use still depends on the balance between the expected positive effects-especially increased reflection and faster response to important issues-versus the potential impediments. A toxic culture that affects impartiality and independence, as well as the need to invest in bureaucratic systems may make in impractical for some institutions to undertake CQI.

From prescription to guidance: a European framework for generic competencies

In postgraduate medical education, required competencies are described in detail in existing competency frameworks. This study proposes an alternative strategy for competency-based medical education design, which is supported by change management theories. We demonstrate the value of allowing room for re-invention and creative adaptation of innovations. This new strategy was explored for the development of a new generic competency framework for a harmonised European curriculum in Obstetrics and Gynaecology. The generic competency framework was developed through action research. Data were collected by four European stakeholder groups (patients, nurses, midwives and hospital boards), using a variety of methods. Subsequently, the data were analysed further in consensus discussions with European specialists and trainees in Obstetrics and Gynaecology. These discussions ensured that the framework provides guidance, is specialty-specific, and that implementation in all European countries could be feasible. The presented generic competency framework identifies four domains: 'Patient-centred care', 'Teamwork', 'System-based practice' and 'Personal and professional development'. For each of these four domains, guiding competencies were defined. The new generic competency framework is supported by European specialists and trainees in Obstetrics and Gynaecology, as well as by their European stakeholders. According to change management theories, it seems vital to allow room for re-invention and creative adaptation of the competency framework by medical professionals. Therefore, the generic competency framework offers guidance rather than prescription. The presented strategy for competency framework development offers leads for implementation of competency-based medical education as well as for development of innovations in postgraduate medical education in general.

Accreditation systems for Postgraduate Medical Education: a comparison of five countries

There is a widespread consensus about the need for accreditation systems for evaluating post-graduate medical education programs, but accreditation systems differ substantially across countries. A cross-country comparison of accreditation systems could provide valuable input into policy development processes. We reviewed the accreditation systems of five countries: The United States, Canada, The United Kingdom, Germany and Israel. We used three information sources: a literature review, an online search for published information and applications to some accreditation authorities. We used template analysis for coding and identification of major themes. All five systems accredit according to standards, and basically apply the same accreditation tools: site-visits, annual data collection and self-evaluations. Differences were found in format of standards and specifications, the application of tools and accreditation consequences. Over a 20-year period, the review identified a three-phased process of evolution-from a process-based accreditation system, through an adaptation phase, until the employment of an outcome-based accreditation system. Based on the five-system comparison, we recommend that accrediting authorities: broaden the consequences scale; reconsider the site-visit policy; use multiple data sources; learn from other countries' experiences with the move to an outcome-based system and take the division of roles into account.