Shared medical regulation in a time of increasing calls for accountability and transparency: comparison of recertification in the United States, Canada, and the United Kingdom

Continuing professional development (CPD) for anesthetists: A systematic review

BACKGROUND

In accordance with the focus on patient safety and quality in healthcare, continuing professional development (CPD) has received increasing levels of attention as a means to ensure physicians maintain their clinical competencies and are fit to practice. There is some evidence of a beneficial effect of CPD, though few studies have evaluated its effect within anesthesia. The primary aim of this systematic review was to establish which CPD activities anesthetists are engaged in and their effectiveness. The secondary aim was to explore which methods are employed to evaluate anesthetists' clinical performance.

METHODS

Databases searched: Medline, Embase and Web of Science, in May 2023. Additional papers were identified through searching the references of included studies. Eligible studies included anesthetists, either exclusively or combined with other healthcare professionals, who underwent a learning activity or assessment method as part of a formalized CPD program or a stand-alone activity. Non-English language studies, non-peer reviewed studies and studies published prior to 2000 were excluded. Eligible studies were quality assessed and narratively synthesized, with results presented as descriptive summaries.

RESULTS

A total of 2112 studies were identified, of which 63 were eligible for inclusion, encompassing more than 137,518 participants. Studies were primarily of quantitative design and medium quality. Forty-one studies reported outcomes of single learning activities, whilst 12 studies investigated different roles of assessment methods in CPD and ten studies evaluated CPD programs or combined CPD activities. A 36 of the 41 studies reported positive effects of single learning activities. Investigations of assessment methods revealed evidence of inadequate performance amongst anesthetists and a mixed effect of feedback. Positive attitudes and high levels of engagement were identified for CPD programs, with some evidence of a positive impact on patient/organizational outcomes.

DISCUSSION

Anesthetists are engaged in a variety of CPD activities, with evidence of high levels of satisfaction and a positive learning effect. However, the impact on clinical practice and patient outcomes remains unclear and the role of assessment is less well-defined. There is a need for further, high-quality studies, evaluating a broader range of outcomes, in order to identify which methods are most effective to train and assess specialists in anesthesia.

An exploration into physician and surgeon data sensemaking: a qualitative systematic review using thematic synthesis

Providing electronic health data to medical practitioners to reflect on their performance can lead to improved clinical performance and quality of care. Understanding the sensemaking process that is enacted when practitioners are presented with such data is vital to ensure an improvement in performance. Thus, the primary objective of this research was to explore physician and surgeon sensemaking when presented with electronic health data associated with their clinical performance. A systematic literature review was conducted to analyse qualitative research that explored physicians and surgeons experiences with electronic health data associated with their clinical performance published between January 2010 and March 2022. Included articles were assessed for quality, thematically synthesised, and discussed from the perspective of sensemaking. The initial search strategy for this review returned 8,829 articles that were screened at title and abstract level. Subsequent screening found 11 articles that met the eligibility criteria and were retained for analyses. Two articles met all of the standards within the chosen quality assessment (Standards for Reporting Qualitative Research, SRQR). Thematic synthesis generated five overarching themes: data communication, performance reflection, infrastructure, data quality, and risks. The confidence of such findings is reported using CERQual (Confidence in the Evidence from Reviews of Qualitative research). The way the data is communicated can impact sensemaking which has implications on what is learned and has impact on future performance. Many factors including data accuracy, validity, infrastructure, culture can also impact sensemaking and have ramifications on future practice. Providing data in order to support performance reflection is not without risks, both behavioural and affective. The latter of which can impact the practitioner's ability to effectively make sense of the data. An important consideration when data is presented with the intent to improve performance.Registration This systematic review was registered with Prospero, registration number: CRD42020197392.

Doctors' attitudes to maintenance of professional competence: A scoping review

CONTEXT

Recent decades have seen the international implementation of programmes aimed at assuring the continuing competence of doctors. Maintenance of Professional Competence (MPC) programmes aim to encourage doctors' lifelong learning and ensure high-quality, safe patient care; however, programme requirements can be perceived as bureaucratic and irrelevant to practice, leading to disengagement. Doctors' attitudes and beliefs about MPC are critical to translating regulatory requirements into committed and effective lifelong learning. We aimed to summarise knowledge about doctors' attitudes to MPC to inform the development of MPC programmes and identify under-researched areas.

METHODS

We undertook a scoping review following Arksey and O'Malley, including sources of evidence about doctors' attitudes to MPC in the United States, the United Kingdom, Canada, Australia, New Zealand and Ireland, and using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) as a guide.

RESULTS

One hundred and twenty-five sources of evidence were included in the review. One hundred and two were peer-reviewed publications, and 23 were reports. Most were from the United Kingdom or the United States and used quantitative or mixed methods. There was agreement across jurisdictions that MPC is a good idea in theory but doubt that it achieves its objectives in practice. Attitudes to the processes of MPC, and their impact on learning and practice were mixed. The lack of connection between MPC and practice was a recurrent theme. Barriers to participation were lack of time and resources, complexity of the requirements and a lack of flexibility in addressing doctors' personal and professional circumstances.

CONCLUSIONS

Overall, the picture that emerged is that doctors are supportive of the concept of MPC but have mixed views on its processes. We highlight implications for research and practice arising from these findings.

Continuing professional development for anesthesiologists: A systematic review protocol

BACKGROUND

Regulators increasingly use formalized programs that are based on continuing professional development (CPD) activities to ensure that physicians are fit to practice. There is convincing evidence regarding the positive effects of CPD activities on performance and patient outcomes. However, there is limited available studies, investigating its effect in anesthesia, specifically. Moreover, although there exists considerable evidence linking specific CPD activities to improved performance, only few studies have investigated the effect of combinations of activities, or formalized systems, as a whole. Consequently, to address this uncertainty regarding the impact of CPD activities, within anesthesiology, this systematic review aims to establish which activities anesthesiologists are engaged in and their impact on clinical competence and subsequent patient outcomes.

METHODS

A systematic review of the current literature regarding CPD for fully qualified anesthesiologists will be undertaken. Characteristics of the included studies will be summarized descriptively, and the screening process will be outlined using the preferred reporting items for systematic reviews and meta-analysis flow diagram. Given the diverse methods adopted within medical education research, it is anticipated that there will be significant heterogeneity between the included studies and therefore, a meta-analysis will not be possible and a narrative synthesis approach will be usd. The outcomes of interest include type of CPD learning activity and/or assessment method anesthesiologists are engaged in; and their effectiveness, either as standalone activities or as part of formalized systems.

CONCLUSION

The aim of the study was to give an overview of the breadth and nature of CPD activities, and their effects on fully qualified anesthesiologists' clinical competences and patient outcomes.

Maintenance of professional competence in Ireland: a national survey of doctors' attitudes and experiences

OBJECTIVES

Programmes to ensure doctors' maintenance of professional competence (MPC) have been established in many countries. Since 2011, doctors in Ireland have been legally required to participate in MPC. A significant minority has been slow to engage with MPC, mirroring the contested nature of such programmes internationally. This study aimed to describe doctors' attitudes and experiences of MPC in Ireland with a view to enhancing engagement.

PARTICIPANTS

All registered medical practitioners in Ireland required to undertake MPC in 2018 were surveyed using a 33-item cross-sectional mixed-methods survey designed to elicit attitudes, experiences and suggestions for improvement.

RESULTS

There were 5368 responses (response rate 42%). Attitudes to MPC were generally positive, but the time, effort and expense involved outweighed the benefit for half of doctors. Thirty-eight per cent agreed that MPC is a tick-box exercise. Heavy workload, travel, requirement to record continuing professional development activities and demands placed on personal time were difficulties cited. Additional support, as well as higher quality, more varied educational activities, were among suggested improvements. Thirteen per cent lacked confidence that they could meet requirements, citing employment status as the primary issue. MPC was particularly challenging for those working less than full-time, in locum or non-clinical roles, and taking maternity or sick leave. Seventy-seven per cent stated a definite intention to comply with MPC requirements. Being male, or having a basic medical qualification from outside Ireland, was associated with less firm intention to comply.

CONCLUSIONS

Doctors need to be convinced of the benefits of MPC to them and their patients. A combination of clear communication and improved relevance to practice would help. Addition of a facilitated element, for example, appraisal, and varied ways to meet requirements, would support participation. MPC should be adequately resourced, including provision of high-quality free educational activities. Systems should be established to continually evaluate doctors' perspectives.

Definitions of Physician Certification Used in the North American Literature: A Scoping Review

INTRODUCTION

The authors sought to identify how physician specialty certification is defined in the North American literature.

METHODS

A rigorous, established six-stage scoping review framework was used to identify the North American certification literature published between January 2006 and May 2016 relating to physician specialty certification. Data were abstracted using a charting form developed by the study team. Quantitative summary data and qualitative thematic analysis of the purpose of certification were derived from the extracted data.

RESULTS

A two stage screening process identified 88 articles that met predefined criteria. Only 14 of the 88 articles (16%) contained a referenced purpose of certification. Eighteen definitions were identified from these articles. Definitional concepts included lifelong learning and continuous professional development, assessment of competence and performance, performance improvement, public accountability, and professional standing.

DISCUSSION

Most articles identified in this scoping review did not define certification or describe its purpose or intent. Future studies should provide a definition of certification to further scholarly examination of its intent and effects and inform its further evolution.

Is Collecting Patient Feedback "a Futile Exercise" in the Context of Recertification?

OBJECTIVE

Patient feedback is considered integral to maintaining excellence, patient safety, and professional development. However, the collection of and reflection on patient feedback may pose unique challenges for psychiatrists. This research uniquely explores the value, relevance, and acceptability of patient feedback in the context of recertification.

METHODS

The authors conducted statistical and inductive thematic analyses of psychiatrist responses (n = 1761) to a national census survey of all doctors (n = 26,171) licensed to practice in the UK. Activity theory was also used to develop a theoretical understanding of the issues identified.

RESULTS

Psychiatrists rate patient feedback as more useful than some other specialties. However, despite asking a comparable number of patients, psychiatrists receive a significantly lower response rate than most other specialties. Inductive thematic analysis identified six key themes: (1) job role, setting, and environment; (2) reporting issues; (3) administrative barriers; (4) limitations of existing patient feedback tools; (5) attitudes towards patient feedback; and (6) suggested solutions.

CONCLUSIONS

The value, relevance, and acceptability of patient feedback are undermined by systemic tensions between division of labor, community understanding, tool complexity, and restrictive rule application. This is not to suggest that patient feedback is "a futile exercise." Rather, existing feedback processes should be refined. In particular, the value and acceptability of patient feedback tools should be explored both from a patient and professional perspective. If issues identified remain unresolved, patient feedback is at risk of becoming a "futile exercise" that is denied the opportunity to enhance patient safety, quality of care, and professional development.

Doctors' attitudes to, beliefs about, and experiences of the regulation of professional competence: a scoping review protocol

BACKGROUND

Historically, individual doctors were responsible for maintaining their own professional competence. More recently, changing patient expectations, debate about the appropriateness of professional self-regulation, and high-profile cases of malpractice have led to a move towards formal regulation of professional competence (RPC). Such programmes require doctors to demonstrate that they are fit to practice, through a variety of means. Participation in RPC is now part of many doctors' professional lives, yet it remains a highly contested area. Cost, limited evidence of impact, and lack of relevance to practice are amongst the criticisms cited. Doctors' attitudes towards RPC, their beliefs about its objectives and effectiveness, and their experiences of trying to meet its requirements can impact engagement with the process. We aim to conduct a scoping review to map the empirical literature in this area, to summarise the key findings, and to identify gaps for future research.

METHODS

We will conduct our review following the six phases outlined by Arksey and O'Malley, and Levac. We will search seven electronic databases: Academic Search Complete, Business Source Complete, CINAHL, PsycINFO, PubMed, Social Sciences Full Text, and SocINDEX for relevant publications, and the websites of medical regulatory and educational organisations for documents. We will undertake backward and forward citation tracking of selected studies and will consult with international experts regarding key publications. Two researchers will independently screen papers for inclusion and extract data using a piloted data extraction tool. Data will be collated to provide a descriptive summary of the literature. A thematic analysis of the key findings will be presented as a narrative summary of the literature.

DISCUSSION

We believe that this review will be of value to those tasked with the design and implementation of RPC programmes, helping them to maximise doctors' commitment and engagement, and to researchers, pointing to areas that would benefit from further enquiry. This research is timely; internationally existing programmes are evolving, new programmes are being initiated, and many jurisdictions do not yet have programmes in place. There is an opportunity for learning across different programmes and from the experiences of established programmes. Our review will support that learning.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO does not register scoping reviews.

Primary care performance of alternatively licenced physicians in Ontario, Canada: a cross-sectional study using administrative data

OBJECTIVES

Medical Regulatory Authorities (MRAs) provide licences to physicians and monitor those physicians once in practice to support their continued competence. In response to physician shortages, many Canadian MRAs developed alternative licensure routes to allow physicians who do not meet traditional licensure criteria to obtain licences to practice. Many physicians have gained licensure through alternative routes, but the performance of these physicians in practice has not been previously examined. This study compared the performance of traditionally and alternatively licenced physicians in Ontario using quality indicators of primary care. The purpose of this study was to examine the practice performance of alternatively licenced physicians and provide evaluative evidence for alternative licensure policies.

DESIGN

A cross-sectional retrospective examination of Ontario health administrative data was conducted using Poisson regression analyses to compare the performance of traditionally and alternatively licenced physicians.

SETTING

Primary care in Ontario, Canada.

PARTICIPANTS

All family physicians who were licenced in Ontario between 2000 and 2012 and who had complete medical billing data in 2014 were included (n=11 419).

OUTCOME MEASURES

Primary care quality indicators were calculated for chronic disease management, preventive paediatric care, cancer screening and hospital readmission rates using Ontario health administrative data.

RESULTS

Alternatively licenced physicians performed similarly to traditionally licenced physicians in many primary care performance measures. Minimal differences were seen across groups in indicators of diabetic care, congestive heart failure care, asthma care and cancer screening rates. Larger differences were found in preventive care for children less than 2 years of age, particularly for alternatively licenced physicians who entered Ontario from another Canadian province.

CONCLUSIONS

Our findings demonstrate that alternatively licenced physicians perform similarly to traditionally licenced physicians across many indicators of primary care. Our study also demonstrates the utility of administrative data for examining physician performance and evaluating medical regulatory policies and programmes.

A case for revisiting peer review: Implications for professional self-regulation and quality improvement

Background

Quality improvement in healthcare has often been promoted as different from and more valuable than peer review and other professional self-regulation processes. In spite of attempts to harmonize these two approaches, the perception of dichotomous opposition has persisted. A sequence of events in the troubled California prison system fortuitously isolated workforce interventions from more typical quality improvement interventions. Our objectives were to (1) evaluate the relative contributions of professional accountability and quality improvement interventions to an observed decrease in population mortality and (2) explore the organizational dynamics that potentiated positive outcomes.

Methods

Our retrospective mixed-methods case study correlated time-series analysis of mortality with the timing of reform interventions. Quantitative and qualitative evidence was drawn from court documents, public use files, internal databases, and other archival documents.

Results

Change point analysis reveals with 98% confidence that a significant improvement in age-adjusted natural mortality occurred in 2007, decreasing from 138.7 per 100,000 in the 1998–2006 period to 106.4 in the 2007–2009 period. The improvement in mortality occurred after implementation of accountability processes, prior to implementation of quality improvement interventions. Archival evidence supports the positive impact of physician competency assessments, robust peer review, and replacement of problem physicians.

Conclusions

Our analysis suggests that workforce accountability provides a critical quality safeguard, and its neglect in scholarship and practice is unjustified. As with quality improvement, effective professional self-regulation requires systemic implementation of enabling policies, processes, and staff resources. The study adds to evidence that the distribution of physician performance contains a heterogeneous left skew of dyscompetence that is associated with significant harm and suggests that professional self-regulation processes such as peer review can reduce that harm. Beyond their responsibility for direct harm, dyscompetent professionals can have negative impacts on group performance. The optimal integration of professional accountability and quality improvement systems merits further investigation.

Doctors on the move: a European case study on the key characteristics of national recertification systems

OBJECTIVES

With increased cross-border movement, ensuring safe and high-quality healthcare has gained primacy. The purpose of recertification is to ensure quality of care through periodically attesting doctors' professional proficiency in their field. Professional migration and facilitated cross-border recognition of qualifications, however, make us question the fitness of national policies for safeguarding patient care and the international accountability of doctors.

DESIGN AND SETTING

We performed document analyses and conducted 19 semistructured interviews to identify and describe key characteristics and effective components of 10 different European recertification systems, each representing one case (collective case study). We subsequently compared these systems to explore similarities and differences in terms of assessment criteria used to determine process quality.

RESULTS

Great variety existed between countries in terms and assessment formats used, targeting cognition, competence and performance (Miller's assessment pyramid). Recertification procedures and requirements also varied significantly, ranging from voluntary participation in professional development modules to the mandatory collection of multiple performance data in a competency-based portfolio. Knowledge assessment was fundamental to recertification in most countries. Another difference concerned the stakeholders involved in the recertification process: while some systems exclusively relied on doctors' self-assessment, others involved multiple stakeholders but rarely included patients in assessment of doctors' professional competence. Differences between systems partly reflected different goals and primary purposes of recertification.

CONCLUSION

Recertification systems differ substantially internationally with regard to the criteria they apply to assess doctors' competence, their aims, requirements, assessment formats and patient involvement. In the light of professional mobility and associated demands for accountability, we recommend that competence assessment includes patients' perspectives, and recertification practices be shared internationally to enhance transparency. This can help facilitate cross-border movement, while guaranteeing high-quality patient care.

"Certified … now what?" On the Challenges of Lifelong Learning: Report from an AMEE 2017 Symposium

The increasing mobility of patients and healthcare professionals across the countries of Europe has highlighted the wide variations in both medical training, and provision of medical competency and skills. The maintenance of the standards defining competency and skills have national and international implications and have proved challenging for national regulatory bodies. Thus each nation has introduced different types of Continuing Professional Development (CPD), recertification and relicensing systems. At the Symposium entitled: " 'Certified … now what?' On the Challenges of Lifelong Learning" in August 2017 at the Association for Medical Education in Europe (AMEE) annual conference, we reviewed differing European national relicensing systems were reviewed. The review highlighted various lifelong learning and competence assessment approaches using examples from different medical specialties across several European countries.

The implementation of medical revalidation: an assessment using normalisation process theory

Background

Medical revalidation is the process by which all licensed doctors are legally required to demonstrate that they are up to date and fit to practise in order to maintain their licence. Revalidation was introduced in the United Kingdom (UK) in 2012, constituting significant change in the regulation of doctors. The governing body, the General Medical Council (GMC), envisages that revalidation will improve patient care and safety. This potential however is, in part, dependent upon how successfully revalidation is embedded into routine practice. The aim of this study was to use Normalisation Process Theory (NPT) to explore issues contributing to or impeding the implementation of revalidation in practice.

Methods

We conducted seventy-one interviews with sixty UK policymakers and senior leaders at different points during the development and implementation of revalidation: in 2011 (n = 31), 2013 (n = 26) and 2015 (n = 14). We selected interviewees using purposeful sampling. NPT was used as a framework to enable systematic analysis across the interview sets.

Results

Initial lack of consensus over revalidation’s purpose, and scepticism about its value, decreased over time as participants recognised the benefits it brought to their practice (coherence category of NPT). Though acceptance increased across time, revalidation was not seen as a legitimate part of their role by all doctors. Key individuals, notably the Responsible Officer (RO), were vital for the successful implementation of revalidation in organisations (cognitive participation category). The ease with which revalidation could be integrated into working practices varied greatly depending on the type of role a doctor held and the organisation they work for and the provision of resources was a significant variable in this (collective action category). Formal evaluation of revalidation in organisations was lacking but informal evaluation was taking place. Revalidation had not yet reached the stage where feedback was being used for improvement (reflexive monitoring category).

Conclusions

Requiring all organisations to use the same revalidation model made revalidation easy to integrate into existing work for some but problematic for others. In order for revalidation to be fully embedded and successful, impeding factors, such as a lack of resources, need to be addressed.

Workshop to address gaps in regulation of minimally manipulated autologous cell therapies for homologous use in Canada

In Canada, minimally manipulated autologous cell therapies for homologous use (MMAC-H) are either regulated under the practice of medicine, or as drugs or devices under the Food and Drugs Act, Food and Drug Regulations (F&DR) or Medical Device Regulations (MDR). Cells, Tissues and Organs (CTO) Regulations in Canada are restricted to minimally manipulated allogeneic products for homologous use. This leaves an important gap in the interpretation of existing regulations. The purposes of this workshop co-organized by the Stem Cell Network and the Centre for Commercialization of Regenerative Medicine (CCRM) were to discuss the current state of regulation of MMAC-H therapies in Canada and compare it with other regulatory jurisdictions, with the intent of providing specific policy recommendations to Health Canada. Participants came to a consensus on the need for well-defined common terminology between regulators and stakeholders, a common source of confusion and misinformation. A need for a harmonized national approach to oversight of facilities providing MMAC-H therapies based on existing standards, such as Canadian Standards Association (CSA), was also voiced. Facilities providing MMAC-H therapies should also participate in collection of long-term data to ensure patient safety and efficacy of therapies. Harmonization across provinces of the procedures and practices involving administration of MMAC-H would be preferred. Participants felt that devices used to process MMAC-H are adequately regulated under existing MDR. Overly prescriptive regulation will stifle innovation, whereas insufficient regulation might allow unsafe or ineffective therapies to be offered. Until a clear, balanced and explicit approach is articulated, regulatory uncertainty remains a barrier.

Competency-based medical education and continuing professional development: A conceptualization for change

Competency-based medical education (CBME) is as important in continuing professional development (CPD) as at any other stage of a physician's career. Principles of CBME have the potential to revolutionize CPD. Transitioning to CBME-based CPD will require a cultural change to gain commitment from physicians, their employers and institutions, CPD providers, professional organizations, and medical regulators. It will require learning to be aligned with professional and workplace standards. Practitioners will need to develop the expertise to systematically examine their own clinical performance data, identify performance improvement opportunities and possibilities, and develop a plan to address areas of concern. Health care facilities and systems will need to produce data on a regular basis and to develop and train CPD educators who can work with physician groups. Stakeholders, such as medical regulatory authorities who are responsible for licensing physicians and other standard-setting bodies that credential and develop maintenance-of-certification systems, will need to change their paradigm of competency enhancement through CPD.

Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians' Professional Performance

INTRODUCTION

Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT.

METHODS

The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's α and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated.

RESULTS

For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's α ≥ 0.84 and item-total correlations ≥ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three to four coworkers were sufficient.

DISCUSSION

The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor.

National programmes for validating physician competence and fitness for practice: a scoping review

OBJECTIVE

To explore and categorise the state of existing literature for national programmes designed to affirm or establish the continuing competence of physicians.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, ERIC, Sociological Abstracts, web/grey literature (2000-2014).

SELECTION

Included when a record described a (1) national-level physician validation system, (2) recognised as a system for affirming competence and (3) reported relevant data.

DATA EXTRACTION

Using bibliographic software, title and abstracts were reviewed using an assessment matrix to ensure duplicate, paired screening. Dyads included both a methodologist and content expert on each assessment, reflective of evidence-informed best practices to decrease errors.

RESULTS

45 reports were included. Publication dates ranged from 2002 to 2014 with the majority of publications occurring in the previous six years (n=35). Country of origin--defined as that of the primary author--included the USA (N=32), the UK (N=8), Canada (N=3), Kuwait (N=1) and Australia (N=1). Three broad themes emerged from this heterogeneous data set: contemporary national programmes, contextual factors and terminological consistency. Four national physician validation systems emerged from the data: the American Board of Medical Specialties Maintenance of Certification Program, the Federation of State Medical Boards Maintenance of Licensure Program, the Canadian Revalidation Program and the UK Revalidation Program. Three contextual factors emerged as stimuli for the implementation of national validation systems: medical regulation, quality of care and professional competence. Finally, great variation among the definitions of key terms was identified.

CONCLUSIONS

There is an emerging literature focusing on national physician validation systems. Four major systems have been implemented in recent years and it is anticipated that more will follow. Much of this work is descriptive, and gaps exist for the extent to which systems build on current evidence or theory. Terminology is highly variable across programmes for validating physician competence and fitness for practice.

Short-term mortality following surgical procedures for the diagnosis of pediatric brain tumors: outcome analysis in 5533 children from SEER, 2004-2011

OBJECT Thirty-day mortality is increasingly a reference metric regarding surgical outcomes. Recent data estimate a 30-day mortality rate of 1.4-2.7% after craniotomy for tumors in children. No detailed analysis of short-term mortality following a diagnostic neurosurgical procedure (e.g., resection or tissue biopsy) for tumor in the US pediatric population has been conducted. METHODS The Surveillance, Epidemiology and End Results (SEER) data sets identified patients ≤ 21 years who underwent a diagnostic neurosurgical procedure for primary intracranial tumor from 2004 to 2011. One- and two-month mortality was estimated. Standard statistical methods estimated associations between independent variables and mortality. RESULTS A total of 5533 patients met criteria for inclusion. Death occurred within the calendar month of surgery in 64 patients (1.16%) and by the conclusion of the calendar month following surgery in 95 patients (1.72%). Within the first calendar month, patients < 1 year of age (n = 318) had a risk of death of 5.66%, while those from 1 to 21 years (n = 5215) had a risk of 0.88% (p < 0.0001). By the end of the calendar month following surgery, patients < 1 year (n = 318) had a risk of death of 7.23%, while those from 1 to 21 years (n = 5215) had a risk of 1.38% (p < 0.0001). Children < 1 year at diagnosis were more likely to harbor a high-grade lesion than older children (OR 1.9, 95% CI 1.5-2.4). CONCLUSIONS In the SEER data sets, the risk of death within 30 days of a diagnostic neurosurgical procedure for a primary pediatric brain tumor is between 1.16% and 1.72%, consistent with contemporary data from European populations. The risk of mortality in infants is considerably higher, between 5.66% and 7.23%, and they harbor more aggressive lesions.

Approaches to management of complaints and notifications about health practitioners in Australia

In 2005, the Australian Productivity Commission made a recommendation that a national health registration regimen and a consolidated national accreditation regimen be established. On 1 July 2010, the National Registration and Accreditation Scheme (NRAS) for health practitioners came into effect and the Australian Health Practitioner Regulation Agency (AHPRA) became the single national oversight agency for health professional regulation. It is governed by the Health Practitioner Regulation National Law Act (the National Law). While all states and territories joined NRAS for registration and accreditation, NSW did not join the scheme for the handling of complaints, but retained its existing co-regulatory complaint-handling system. All other states and territories joined the national notification (complaints) scheme prescribed in the National Law. Because the introduction of NRAS brings with it new processes and governance around the management of complaints that apply to all regulated health professionals in all states and territories except NSW, where complaints management remains largely unchanged, there is a need for comparative analysis of these differing national and NSW approaches to the management of complaints/notifications about health professionals, not only to allow transparency for consumers, but also to assess consistency of decision making around complaints/notifications across jurisdictions. This paper describes the similarities and differences for complaints/notifications handling between the NRAS and NSW schemes and briefly discusses subsequent and potential changes in other jurisdictions.

Patient Safety Risks of Basic Urological Procedures Performed by Junior and Senior Residents

OBJECTIVE

To investigate the current performance of urological residents regarding basic urological procedures in relation to patient safety issues and the identification of specific training needs.

DESIGN

Observational data of 146 urethrocystoscopies (UCSs), 27 transrectal ultrasounds of the prostate (TRUSs), 38 transrectal ultrasound-guided prostatic biopsies (TRUSPs), and 30 transurethral resections of bladder tumor (TURBTs) were collected. Performance was evaluated using scoring lists including details on completeness of procedural steps, level of independence, time, and the incidence of unintended events. The causal factors contributing to the unintended events were identified by 2 expert urologists and classified according to the recognized PRISMA method.

SETTING

This study was performed in 5 teaching hospitals in the Netherlands.

PARTICIPANTS

We included 11 junior residents and 5 senior residents in urology in the final study cohort.

RESULTS

Senior residents showed a lower degree of completeness in material usage than junior residents did during UCS (p < 0.01) and in preparation, material usage, and procedure during TRUSP (all p < 0.05). In UCS and TURBT, senior residents received significantly less feedback than junior residents did (both p < 0.01). Incidence of unintended events for junior vs senior residents was 11% and 4% in UCS, 0% and 7% in transrectal ultrasound of the prostate, 36% and 62% in TRUSP, and 41% and 23% in TURBT, respectively. Overall, unintended events were mainly caused by human factors, in particular, verification and skills-based issues.

CONCLUSION

Present performance of basic urological procedures involves a high percentage of unintended events, especially in TRUSP and TURBT, which are mainly caused by human factors and are a potential threat for patient safety. Junior residents are less independent but more thorough in the performance of UCS and TRUSP than senior residents are. Targeted skills training including assessment should be implemented before privileges for independent practice are granted to reduce the incidence of unintended events and optimize patient safety.

Role for assessment in maintenance of certification: physician perceptions of assessment

INTRODUCTION

The Royal College of Physicians and Surgeons of Canada modified its Maintenance of Certification (MOC) framework in 2011 to further incentivize assessment activities compared to group and self-learning. The purpose of this study was to explore physician's perceptions of their access to assessment activities, barriers to participation in assessment, and the need for the Royal College to further support its fellows in gaining access to assessment activities.

METHODS

A questionnaire-based survey was sent to all participants of the MOC program as part of a program evaluation examining recent changes to the MOC program.

RESULTS

5259 respondents contributed responses. Most physicians were comfortable with the revised framework for assessment while approximately 40% were neutral regarding whether lack of access to self-assessment activities was a problem. Respondents expressed a need for more self-assessment programs particularly those developed outside of Canada. Neither a lack of feedback about performance or discomfort with recording performance gaps was perceived as a barrier to participation in assessment activities. Physician comments were consistent with the quantitative data and elaborated on the need to develop and recognize more assessment activities.

DISCUSSION

Physicians accepted the revised MOC program framework but perceived difficulty in accessing assessment programs, activities, and tools. As the framework changed again January 2014, requiring all fellows and MOC program participants to completion of at least 25 credits in each section of the MOC program (including assessment) during their new 5-year MOC cycle, additional resources will be needed to support opportunities for physicians to engage in assessment.

Managing patient complaints in China: a qualitative study in Shanghai

OBJECTIVES

To examine the handling system for patient complaints and to identify existing barriers that are associated with effective management of patient complaints in China.

SETTING

Key stakeholders of the handling system for patient complaints at the national, Shanghai municipal and hospital levels in China.

PARTICIPANTS

35 key informants including policymakers, hospital managers, healthcare providers, users and other stakeholders in Shanghai.

PRIMARY AND SECONDARY OUTCOME MEASURES

Semistructured interviews were conducted to understand the process of handling patient complaints and factors affecting the process and outcomes of patient complaint management.

RESULTS

The Chinese handling system for patient complaints was established in the past decade. Hospitals shoulder the most responsibility of patient complaint handling. Barriers to effective management of patient complaints included service users' low awareness of the systems in the initial stage of the process; poor capacity and skills of healthcare providers, incompetence and powerlessness of complaint handlers and non-transparent exchange of information during the process of complaint handling; conflicts between relevant actors and regulations and unjustifiable complaints by patients during solution settlements; and weak enforcement of regulations, deficient information for managing patient complaints and unwillingness of the hospitals to effectively handle complaints in the postcomplaint stage.

CONCLUSIONS

Barriers to the effective management of patient complaints vary at the different stages of complaint handling and perspectives on these barriers differ between the service users and providers. Information, procedure design, human resources, system arrangement, unified legal system and regulations and factors shaping the social context all play important roles in effective patient complaint management.

Regulating healthcare complaints: a literature review

Purpose

– The purpose of this paper is to explore approaches to the regulation of healthcare complaints and disciplinary processes.

Design/methodology/approach

– A literature review was conducted across Medline, Sociological Abstracts, Web of Science, Google Scholar and the health, law and social sciences collections of Informit, using terms tapping both the complaints process and regulation generally.

Findings

– A total of 118 papers dealing with regulation of health complaints or disciplinary proceedings were located. The review reveals a shift away from self-regulation towards greater external oversight, including innovative regulatory approaches including “networked governance” and flexible or “responsive” regulation. It reports growing interest in adoption of strategic and responsive approaches to health complaints governance, by rejecting traditional legal forms in favor of more strategic and responsive forms, taking account of the complexity of adverse health events by tailoring responses to individual circumstances of complainants and their local environments.

Originality/value

– The challenge of how to collect and harness complaints data to improve the quality of healthcare at a systemic level warrants further research. Scope also exists for researching health complaints commissions and other “meta-regulatory” bodies to explore how to make these processes fairer and better able to meet the complex needs of complainants, health professionals, health services and society.

The relationship between physician participation in continuing professional development programs and physician in-practice peer assessments

PURPOSE

To investigate the relationship between physicians' performance, as evaluated through in-practice peer assessments, and their participation in continuing professional development (CPD).

METHOD

The authors examined the predictive effects of participating in the CPD programs of the Royal College of Physicians and Surgeons of Canada and the College of Family Physicians of Canada one year before in-practice peer assessments conducted by the medical regulatory authority in Ontario, Canada, in 2008-2009. Two multivariate logistic regression models were used to determine whether physicians who reported participating in any CPD and group-based, assessment-based, and/or self-directed CPD activities were more or less likely to receive satisfactory assessments than physicians who had not. All models were adjusted for the effects of sex, age, specialty certification, practice location, number of patient visits per week, hours worked per week, and international medical graduate status.

RESULTS

A total of 617 physicians were included in the study. Analysis revealed that physicians who reported participating in any CPD activities were significantly more likely (odds ratio [OR] = 2.5; P = .021) to have satisfactory assessments than those who had not. In addition, physicians participating in group-based CPD activities were more likely to have satisfactory assessments than those who did not (OR = 2.4; P = .016).

CONCLUSIONS

There is encouraging evidence supporting a positive predictive association between participating in CPD and performance on in-practice peer assessments. The findings have potential implications for policies which require physicians to participate in programs of lifelong learning.

Comparison of survival between the UK and US after surgery for most common pediatric CNS tumors

BACKGROUND

We report a population-based study examining long-term outcomes for common pediatric CNS tumors comparing results from the UK with the National Cancer Institute's Surveillance, Epidemiology, and End Results (SEER) data set and with the literature. No such international study has previously been reported.

METHODS

Data between 1996 and 2005 from the UK National Registry of Childhood Tumours (NRCT) and the SEER registry were analyzed. We calculated actuarial survival at each time point from histological diagnosis, with death from any cause as the endpoint. Kaplan-Meier estimation and log-rank testing (Cox proportional hazards regression analysis) were used to calculate survival differences among tumor subtypes, adjusting for age at diagnosis.

RESULTS

Population-based outcomes for each tumor type are presented. Overall age-adjusted survival, stratifying for histology (combining pilocytic astrocytoma, anaplastic astrocytoma, glioblastoma, primitive neuroectodermal tumor, medulloblastoma, and ependymoma), is significantly lower for NRCT than SEER (hazard ratio 0.71, P < .001) and at 1, 5, and 10 years. Both NRCT and SEER outcomes are worse than those reported from trials.

CONCLUSION

Analyzing data from comprehensive registries minimizes bias associated with trials and institutional studies. The reasons for the poorer outcomes in children treated in the UK are unclear. Likewise, the differences in outcomes between patients in trials and those not in trials need further investigation. We recommend that all children with CNS tumors be recruited into studies-even if these are observational studies. We also suggest that registries be suitably funded to publish independent outcome data (including morbidity) at both a national and an institutional level.

The 'special obligations' of the modern Hippocratic Oath for 21st century medicine

CONTEXT

Profound advances and discoveries in medicine have markedly improved the lives of many over the 50 years since the modern Hippocratic Oath was written. Regrettably, these advances were and continue to be implemented suboptimally and inequitably across the globe. 'Special obligations to all my fellow humans' is an important theme of the modern Oath. From this perspective, we reflect on the special obligations of the medical profession, and examine how these obligations have changed over the past 50 years.

METHODS

We draw from perspectives of the social contract, professionalism, quality improvement, patient safety and a group of 31 international colleagues involved in medical education as we examine these obligations for individual doctors, health care institutions and medical education systems. The perspectives of the 31 clinician-educators helped us to situate the meaning of the theme of 'special obligations' in the context of challenges facing medical education and health care in the 21st century.

OBSERVATIONS

Improving the quality of care and patient safety, and reducing health care disparities are now paramount as 'special obligations' for doctors, health care systems and medical education organisations, and require us to work collectively and collaboratively in an increasingly interconnected world. In our view, traditions such as the Hippocratic Oath will be worthy of public support only when the medical profession demonstrates in meaningful and transparent ways that it is meeting its social and civic obligations to make the world, not just health care, a better place.

Ensuring high-quality patient care: the role of accreditation, licensure, specialty certification and revalidation in medicine

CONTEXT

The accreditation of medical school programmes and the licensing and revalidation (or recertification) of doctors are thought to be important for ensuring the quality of health care. Whereas regulation of the medical profession is mandated in most jurisdictions around the world, the processes by which doctors become licensed, and maintain their licences, are quite varied. With respect to educational programmes, there has been a recent push to expand accreditation activities. Here too, the quality standards on which medical schools are judged can vary from one region to another.

OBJECTIVES

Given the perceived importance placed by the public and other stakeholders on oversight in medicine, both at the medical school and individual practitioner levels, it is important to document and discuss the regulatory practices employed throughout the world.

METHODS

This paper describes current issues in regulation, provides a brief summary of research in the field, and discusses the need for further investigations to better quantify relationships among regulatory activities and improved patient outcomes.

DISCUSSION

Although there is some evidence to support the value of medical school accreditation, the direct impact of this quality assurance initiative on patient care is not yet known. For both licensure and revalidation, some investigations have linked specific processes to quality indicators; however, additional evaluations should be conducted across the medical education and practice continuum to better elucidate the relationships among regulatory activities and patient outcomes. More importantly, the value of accreditation, licensure and revalidation programmes around the world, including the effectiveness of specific protocols employed in these diverse systems, needs to be better quantified and disseminated.

United Kingdom 30-day mortality rates after surgery for pediatric central nervous system tumors

OBJECT

In an increasing culture of medical accountability, 30-day operative mortality rates remain one of the most objective measurements reported for the surgical field. The authors report population-based 30-day postoperative mortality rates among children who had undergone CNS tumor surgery in the United Kingdom.

METHODS

To determine overall 30-day operative mortality rates, the authors analyzed the National Registry of Childhood Tumors for CNS tumors for the period 2004-2007. The operative mortality rate for each tumor category was derived. In addition, comparison was made with the 30-day operative mortality rates after CNS tumor surgery reported in the contemporary literature. Finally, by use of a funnel plot, institutional performance for 30-day operative mortality was compared for all units across the United Kingdom.

RESULTS

The overall 30-day operative mortality rate for children undergoing CNS tumor surgery in the United Kingdom during the study period was 2.7%. When only malignant CNS tumors were analyzed, the rate increased to 3.5%. One third of the deaths occurred after discharge from the hospital in which the surgery had been performed. The highest 30-day operative mortality rate (19%) was for patients with choroid plexus carcinomas. A total of 20 institutions performed CNS tumor surgery during the study period. Rates for all institutions fell within 2 SDs. No trend associating operative mortality rates and institutional volume was found. In comparison, review of the contemporary literature suggests that the postoperative mortality rate should be approximately 1%.

CONCLUSIONS

The authors believe this to be the first report of national 30-day surgical mortality rates specifically for children with CNS tumors. The study raises questions about the 30-day mortality rate among children undergoing surgery for CNS tumors. International consensus should be reached on a minimum data set for outcomes and should include 30-day operative mortality rates.

Identifying and managing performance concerns in community pharmacists in the UK

OBJECTIVES

To explore current arrangements for identifying and managing performance concerns in community pharmacists in the UK.

METHODS

Semi-structured qualitative telephone interviews were conducted with 20 senior managers from community pharmacies and locum agencies.

RESULTS

A strong emphasis was placed on business performance alongside other aspects of professional performance in the identification of performance concerns in pharmacists. The majority of concerns were identified reactively, through customer complaints, peer- or self-referral, or following a dispensing error. Community pharmacies sought to manage performance concerns internally where possible, but only the larger organizations had the infrastructure to provide their own training or other remedial support. Several challenges to identifying and managing performance concerns were identified. There were few mechanisms for identifying and supporting locum pharmacists with performance issues.

CONCLUSIONS

Being 'for-profit' organizations, community pharmacies may prioritize business performance over ensuring the professional performance of pharmacists, the responsibility for which would be left to the individual pharmacist. This may be detrimental to the quality of care provided. With the growth of independent sector providers more widely, these findings may have implications for the regulation of other health care professionals' performance.

Specialty board certification in the United States: issues and evidence

BACKGROUND

The American Board of Medical Specialties (ABMS) certification and maintenance of certification (MOC) programs strive to provide the public with guidance about a physician's competence. This study summarizes the literature on the effectiveness of these programs.

METHOD

A literature search was conducted for studies published between 1986 and April 2013 and limited to ABMS certification. A modified version of Kirkpatrick's 4 levels of program evaluation included the reaction of stakeholders to certification, the extent to which physicians are encouraged to improve, the relationship between performance in the programs and nonclinical external measures of physician competence, and the relationship of performance in the programs with clinical quality measures.

RESULTS

Patients' and hospitals' value of board certification and physician participation in MOC are high. Physicians are conflicted as to whether the effort involved is worth its value. Self-reported evidence shows improvement in knowledge, practice infrastructure, communication with patients and peers, and clinical care. Certification performance is generally related to nonclinical external measures such as types of training, practice characteristics, demographics, and disciplinary actions. In general, physicians who are board certified provide better patient care, albeit the results have modest effect sizes and are not unequivocal.

CONCLUSIONS

Certification boards should continuously try to improve their programs in response to feedback from stakeholders, changes in the way physicians practice, as well as the growth in the fields of measurement and technology. Keeping pace with these changes in a responsible and evidence-based way is important.

Maintenance of skills, competencies, and performance in allergy and clinical immunology: time to lay the foundation for a universal approach

Allergist/clinical immunologist maintenance of certification and training program reaccreditation are mandatory in some countries. The World Allergy Organization conducted surveys in 2009 and 2011 to assess where such programs were available and to promote the establishment of such programs on a global level. This was done with the presumption that after such an "inventory," World Allergy Organization could offer guidance to its Member Societies on the promotion of such programs to assure the highest standards of practice in the field of allergy and clinical immunology. This review draws on the experience of countries where successful programs are in place and makes recommendations for those wishing to implement such programs for the specialty.

Monitoring community pharmacist's quality of care: a feasibility study of using pharmacy claims data to assess performance

BACKGROUND

Public pressure has increasingly emphasized the need to ensure the continuing quality of care provided by health professionals over their careers. Health profession's regulatory authorities, mandated to be publicly accountable for safe and effective care, are revising their quality assurance programs to focus on regular evaluations of practitioner performance. New methods for routine screening of performance are required and the use of administrative data for measuring performance on quality of care indicators has been suggested as one attractive option. Preliminary studies have shown that community pharmacy claims databases contain the information required to operationalize quality of care indicators. The purpose of this project was to determine the feasibility of routine use of information from these databases by regulatory authorities to screen the quality of care provided at community pharmacies.

METHODS

Information from the Canadian province of Quebec's medication insurance program provided data on prescriptions dispensed in 2002 by more than 5000 pharmacists in 1799 community pharmacies. Pharmacy-specific performance rates were calculated on four quality of care indicators: two safety indicators (dispensing of contra-indicated benzodiazepines to seniors and dispensing of nonselective beta-blockers to patients with respiratory disease) and two effectiveness indicators (dispensing asthma or hypertension medications to non-compliant patients). Descriptive statistics were used to summarize performance.

RESULTS

Reliable estimates of performance could be obtained for more than 90% of pharmacies. The average rate of dispensing was 4.3% (range 0 - 42.5%) for contra-indicated benzodiazepines, 15.2% (range 0 - 100%) for nonselective beta-blockers to respiratory patients, 10.7% (range 0 - 70%) for hypertension medications to noncompliant patients, and 43.3% (0 - 91.6%) for short-acting beta-agonists in over-use situations. There were modest correlations in performance across the four indicators. Nine pharmacies (0.5%) performed in the lowest quartile in all four of the indicators, and 5.3% (n = 95) performed in the lowest quartile on three of four indicators.

CONCLUSIONS

Routinely collected pharmacy claims data can be used to monitor indicators of the quality of care provided in community pharmacies, and may be useful in future to identify underperforming pharmacists, measure the impact of policy changes and determine predictors of best practices.

The role of assessment in competency-based medical education

Competency-based medical education (CBME), by definition, necessitates a robust and multifaceted assessment system. Assessment and the judgments or evaluations that arise from it are important at the level of the trainee, the program, and the public. When designing an assessment system for CBME, medical education leaders must attend to the context of the multiple settings where clinical training occurs. CBME further requires assessment processes that are more continuous and frequent, criterion-based, developmental, work-based where possible, use assessment methods and tools that meet minimum requirements for quality, use both quantitative and qualitative measures and methods, and involve the wisdom of group process in making judgments about trainee progress. Like all changes in medical education, CBME is a work in progress. Given the importance of assessment and evaluation for CBME, the medical education community will need more collaborative research to address several major challenges in assessment, including "best practices" in the context of systems and institutional culture and how to best to train faculty to be better evaluators. Finally, we must remember that expertise, not competence, is the ultimate goal. CBME does not end with graduation from a training program, but should represent a career that includes ongoing assessment.