Effectiveness of educational interventions on the improvement of drug prescription in primary care: a critical literature review

Validation of the QualiPresc instrument for assessing the quality of drug prescription writing in primary health care

INTRODUCTION

Adverse events related to drug prescriptions are the main patient safety issue in primary care; however there is a lack of validated instruments for assessing the quality of prescription writing, which covers the prescriber, the patient and the drug information.

OBJECTIVE

To develop and validate the QualiPresc instrument to assess and monitor the quality of drug prescriptions in primary care, accompanied by a self-instruction direction, with the goal of filling the gap in validated instruments to assess the quality of prescription writing.

METHODOLOGY

A validation study conducted in a municipality in Northeastern Brazil, based on prescriptions prepared in January 2021 by physicians assigned to 18 Basic Health Units and filed in 6 distribution/dispensing units. Four steps were covered: 1) Analysis of content validity of each indicator (relevance and adequacy); 2) Analysis of reliability via intra and inter-rater agreement of each indicator; 3) Analysis of the utility of each indicator; 4) Construction and analysis of the reliability of a weighted composite indicator based on effectiveness and safety scores for each indicator.

RESULTS

Twenty-nine potential indicators were listed, but only 13 were approved for validity, reliability and usefulness. Twelve indicators were excluded because of validity (<90% validity index) and four because they were not useful in the context of the study. Three weighted composite indicators were tested, but only one was approved for reliability and usefulness. The validated instrument therefore contains 13 indicators and 1 weighted composite indicator.

CONCLUSION

This study demonstrates the validity, reliability and usefulness of QualiPresc for the evaluation of prescription writing in the context of primary care. Application to contexts such as secondary care and tertiary care requires cross-cultural adaptation and new content validity. Educators, managers and health care professionals can access QualiPresc online, free of charge, to assess performance and provide feedback involving drug prescribers.

Changing prescribing behaviours with educational outreach: an overview of evidence and practice

BACKGROUND

General practitioners (GPs), or family practitioners, are tasked with prescribing medications that can be harmful to the community if they are inappropriately prescribed or used (e.g. opioids). Educational programs, such as educational outreach (EO), are designed to change the behaviour of health professionals. The purpose of this study was to identify the efficacy of EO programs at changing the prescribing behaviour of GPs.

METHODS

This study included an evidence and practice review, comprising a rapid review supplemented by interviews with people who are familiar with EO implementation for regulation purposes. Seven databases were searched using terms related to health professionals and prescribing. Systematic and narrative reviews published in English after 2007 were included. Non-statistical analysis was used to report intervention efficacy. Three government representatives participated in semi-structured interviews to aid in understanding the relevance of review findings to the Victorian context. Interviews were transcribed verbatim and thematically analysed for emerging themes.

RESULTS

Fourteen reviews were identified for the evidence review. Isolated (e.g. EO program delivered by itself) and multifaceted (e.g. EO program supplemented by other interventions) programs were found to change prescribing behaviours. However, limited evidence suggests that EO can successfully change prescribing behaviours specific to GPs. Isolated EO can successfully change health professional prescribing behaviours, although cheaper alternatives such as letters might be just as effective. Multifaceted EO can also successfully change health professional prescribing behaviours, especially in older adults, but it remains unclear as to what combination of interventions works best. Success factors for EO reported by government representatives included programs having practical rather than didactic foci; making EO compulsory; focussing EO on preventing adverse events; using monetary or professional development incentives; and in-person delivery.

CONCLUSIONS

Educational outreach can successfully change prescribing behaviours but evidence specific to GPs is lacking. Key characteristics of EO that could optimise success include ensuring the EO program is tailored, involves practical learning and uses incentives that are meaningful to clinicians.

Effectiveness of a peer-mediated educational intervention in improving general practitioner diagnostic assessment and management of dementia: a cluster randomised controlled trial

OBJECTIVE

Test effectiveness of an educational intervention for general practitioners (GPs) on quality of life and depression outcomes for patients.

DESIGN

Double-blind, cluster randomised controlled trial.

SETTING

General practices in Australia between 2007 and 2010.

PARTICIPANTS

General practices were randomly allocated to the waitlist (n=37) or intervention (n=66) group, in a ratio of 1:2. A total of 2030 (1478 intervention; 552 waitlist) community-dwelling participants aged 75 years or older were recruited via 168 GPs (113 intervention; 55 waitlist).

INTERVENTIONS

A practice-based academic detailing intervention led by a peer educator that included: (1) training in use of the GP assessment of cognition dementia screening instrument; (2) training in diagnosis and management based on Royal Australian College of General Practitioners Dementia Guidelines; (3) addressing GPs' barriers to dementia diagnosis; and (4) a business case outlining a cost-effective dementia assessment approach.

OUTCOME MEASURES

Primary outcome measures were patient quality of life and depression; secondary outcome measures were: (1) sensitivity and specificity of GP identification of dementia; (2) referral to medical specialists and/or support services; (3) patient satisfaction with care; and (4) carer quality of life, depression and satisfaction with care.

RESULTS

The educational intervention had no significant effect on patient quality of life or depression scores after 12 months. There were however improvements in secondary outcome measures including sensitivity of GP judgement of dementia (p=0.002; OR 6.0, 95% CI 1.92 to 18.73), satisfaction with GP communication for all patients (p=0.024; mean difference 2.1, 95% CI 0.27 to 3.93) and for patients with dementia (p=0.007; mean difference 7.44, 95% CI 2.02 to 12.86) and enablement of carers (p=0.0185; mean difference 24.77, 95% CI 4.15 to 45.40).

CONCLUSION

Practice-based academic detailing did not improve patient quality of life or depression scores but did improve detection of dementia in primary care and patient satisfaction with GP communication.

TRIAL REGISTRATION NUMBER

ACTRN12607000117415; Pre-results.

Physicians report adopting safer opioid prescribing behaviors after academic detailing intervention

Background This study evaluated an educational intervention intended to increase physicians' use of patient prescription history information from the state prescription monitoring program (PMP) and their adoption of clinical behaviors consistent with opioid prescription guidelines to reduce patient risk. Methods Physician volunteers (n = 87) in community practices and Veterans Administration medical settings in South Carolina received an office-based, individualized, educational intervention (Academic Detailing) from a trained pharmacist who promoted three key messages about safer opioid prescribing. Physicians were registered for the state PMP, guided through retrieving patient information from the PMP, and given patient-centered materials. Physicians consented to completing web-surveys; 68 (78%) completed follow-up surveys on average 12.2 weeks post-intervention. Results Of 43 respondents who did not use the PMP before the intervention, 83% adopted PMP use. Self-reports also revealed a significant increase in frequency of the following behaviors: 1) using patient report information from the PMP, 2) using a standardized scale to monitor pain intensity and interference with daily functioning, and 3) issuing orders for urine toxicology screens for patients maintained long-term on opioids. Conclusions The intervention was effective in promoting physician adoption of prescribing behaviors intended to reduce risks associated with prescription opioids. The self-report findings of this study should be confirmed by analysis using data on the number of queries submitted to the state's PMP. The present study suggests that a single academic detailing visit may be an effective tool for increasing physician voluntary registration and utilization of data on patients' prescription history contained in a state PMP.

Introducing a structured prescription form improves the quality of handwritten prescriptions in limited resource setting of developing countries

INTRODUCTION

Incomplete or illegible prescriptions can lead to serious errors in administration of the prescribed medication, which can become hazardous.

OBJECTIVE

Our aim is to determine if a structured prescription form can improve the quality of handwritten prescription in terms of completeness and legibility.

METHODS

We conducted a prospective, non-randomized, time series study of quality of written prescriptions of general practitioners at a tertiary teaching hospital in Peshawar, Pakistan. The study involved an intervention, composed of the introduction of a pre-printed structured prescription form. The data were collected within 4 weeks including a 2-week pre-intervention phase and 2-week post-intervention phase. Completeness, quality of prescriptions and legibility were compared before and after the intervention of the pre-printed structured prescription form.

RESULTS

A total of 463 prescriptions were obtained (260 in the pre-intervention phase and 203 in the post-intervention phase). Between pre-intervention phase and post-intervention phase, the Pakistan Medical and Dental Council registration number presence in prescriptions improved from 73.1% to 100% (P < 0.0005). The presence of prescriber's signature improved from 92.7% to 99% (P = 0.001). Drug duration was not missing in 99.5% in post-intervention phase as compared with 90.4% in pre-intervention phase (P < 0.0005). Prescriptions with no legibility problems improved from 76.2% to 94.1% (P < 0.0005). Although not statistically significant, prescriptions in which drug dosage was not missing improved from 85% to 90.6% (P = 0.07).

LIMITATIONS

We have a limited single-center study. A larger study in multiple settings is needed to develop adequate evidence for such interventions. Subjective nature of prescription legibility can also be considered as a limitation.

CONCLUSION

Structuring a prescription form alone may improve certain aspects of quality of written prescription in terms of completeness and legibility.

Clinically important drug-drug interactions in poly-treated elderly outpatients: a campaign to improve appropriateness in general practice

AIMS

The aim was to assess the impact of a campaign for general practitioners (GPs) to reduce clinically-important drug-drug interactions (DDIs) in poly-treated elderly patients.

METHODS

We compiled a list of 53 DDIs and analyzed reimbursed prescriptions dispensed to poly-treated (≥four drugs) elderly (>65 years) patients in the Emilia Romagna region during January 2011-June 2011 (first pre-intervention period), January 2012-June 2012 (second pre-intervention period) and January 2013-June 2013 (post-intervention period). Educational initiatives to GPs were completed in July 2012-December 2012. Pre-test/post-test analysis (2013 vs. 2012) was performed, also using predicted 2013 data (P < 0.01 for statistical significance).

RESULTS

Despite the slight increase in poly-therapy rate (16% in 2013, +1.5% from 2011), we found a stable or slightly declining number of potential DDIs for each elderly poly-treated patient (~1.5). In 2013, 11 DDIs exceeded 5% of prevalence rate: antidiabetics-β-adrenoceptor blockers ranked first (20.3%), followed by ACE Inhibitors (ACEIs)/sartans-non steroidal anti-inflammatory drugs (NSAIDs) (16.4%), diuretics-NSAIDs (13.6%), selective serotonin re-uptake inhibitors (SSRIs)-NSAIDs/acetyl salicylic acid (ASA) (12.7%) and corticosteroids-NSAIDs/ASA (9.7%). A remarkable reduction emerged for NSAID-related DDIs (diuretics-NSAIDs peaked -14.5%; P < 0.01), whereas prevalence of antidiabetics-β-adrenoceptor blockers increased (+7.9%; P < 0.01). When using predicted values, the statistical significance disappeared for antidiabetics-β-adrenoceptor blockers (+1.3%; P = 0.04), whereas it persisted for almost all NSAIDs-related DDIs: ACEIs/sartans-NSAIDs (-3.0%), diuretics-NSAIDs (-6.0%), SSRIs-NSAIDs/ASA (-5.9%).

CONCLUSIONS

This campaign contained the burden of DDIs in poly-treated elderly patients by 1) reducing most prevalent DDIs, especially NSAIDs-related DDIs and 2) balancing the observed rise in poly-therapy rate with stable rate in overall prescriptions of potentially interacting drugs per patient.

Educational interventions to improve prescription and dispensing of antibiotics: a systematic review

Background

Excessive and inappropriate antibiotic use contributes to growing antibiotic resistance, an important public-health problem. Strategies must be developed to improve antibiotic-prescribing. Our purpose is to review of educational programs aimed at improving antibiotic-prescribing by physicians and/or antibiotic-dispensing by pharmacists, in both primary-care and hospital settings.

Methods

We conducted a critical systematic search and review of the relevant literature on educational programs aimed at improving antibiotic prescribing and dispensing practice in primary-care and hospital settings, published in January 2001 through December 2011.

Results

We identified 78 studies for analysis, 47 in primary-care and 31 in hospital settings. The studies differed widely in design but mostly reported positive results. Outcomes measured in the reviewed studies were adherence to guidelines, total of antibiotics prescribed, or both, attitudes and behavior related to antibiotic prescribing and quality of pharmacy practice related to antibiotics. Twenty-nine studies (62%) in primary care and twenty-four (78%) in hospital setting reported positive results for all measured outcomes; fourteen studies (30%) in primary care and six (20%) in hospital setting reported positive results for some outcomes and results that were not statistically influenced by the intervention for others; only four studies in primary care and one study in hospital setting failed to report significant post-intervention improvements for all outcomes. Improvement in adherence to guidelines and decrease of total of antibiotics prescribed, after educational interventions, were observed, respectively, in 46% and 41% of all the reviewed studies. Changes in behaviour related to antibiotic-prescribing and improvement in quality of pharmacy practice was observed, respectively, in four studies and one study respectively.

Conclusion

The results show that antibiotic use could be improved by educational interventions, being mostly used multifaceted interventions.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2458-14-1276) contains supplementary material, which is available to authorized users.

Type 2 diabetes continuing medical education for general practitioners: what works? A systematic review

AIMS

To perform a systematic review of studies that have assessed the effectiveness of interventions designed to improve healthcare professionals' care of patients with diabetes and to assess the effects of educational interventions targeted at general practitioners' diabetes management.

METHODS

A computer search was conducted using the Cochrane Library, PubMed, Ovid MEDLINE, Scopus, EMBASE, Informit, Google scholar and ERIC from the earliest date of each database up until 2013. A supplementary review of reference lists from each article obtained was also carried out. Measured changes in general practitioners' satisfaction, knowledge, practice behaviours and patient outcomes were recorded.

RESULTS

Thirteen out of 1255 studies met the eligibility criteria, but none was specifically conducted in rural or remote areas. Ten studies were randomized trials. Fewer than half of the studies (5/13, 38.5%) reported a significant improvement in at least one of the following outcome categories: satisfaction with the programme, knowledge and practice behaviour. There was little evidence of the impact of general practitioner educational interventions on patient outcomes. Of the five studies that examined patient outcomes, only one reported a positive impact: a reduction in patient HbA1c levels.

CONCLUSIONS

Few studies examined the effectiveness of general practitioner Type 2 diabetes education in improving general practitioner satisfaction, knowledge, practices and/or patient outcomes. Evidence to support the effectiveness of education is partial and weak. To determine effective strategies for general practitioner education related to Type 2 diabetes, further well designed studies, accompanied by valid and reliable evaluation methods, are needed.

A cluster randomised controlled trial of a pharmacist-led collaborative intervention to improve statin prescribing and attainment of cholesterol targets in primary care

BACKGROUND

Small trials with short term follow up suggest pharmacists' interventions targeted at healthcare professionals can improve prescribing. In comparison with clinical guidance, contemporary statin prescribing is sub-optimal and achievement of cholesterol targets falls short of accepted standards, for patients with atherosclerotic vascular disease who are at highest absolute risk and who stand to obtain greatest benefit. We hypothesised that a pharmacist-led complex intervention delivered to doctors and nurses in primary care, would improve statin prescribing and achievement of cholesterol targets for incident and prevalent patients with vascular disease, beyond one year.

METHODS

We allocated general practices to a 12-month Statin Outreach Support (SOS) intervention or usual care. SOS was delivered by one of 11 pharmacists who had received additional training. SOS comprised academic detailing and practical support to identify patients with vascular disease who were not prescribed a statin at optimal dose or did not have cholesterol at target, followed by individualised recommendations for changes to management. The primary outcome was the proportion of patients achieving cholesterol targets. Secondary outcomes were: the proportion of patients prescribed simvastatin 40 mg with target cholesterol achieved; cholesterol levels; prescribing of simvastatin 40 mg; prescribing of any statin and the proportion of patients with cholesterol tested. Outcomes were assessed after an average of 1.7 years (range 1.4-2.2 years), and practice level simvastatin 40 mg prescribing was assessed after 10 years.

FINDINGS

We randomised 31 practices (72 General Practitioners (GPs), 40 nurses). Prior to randomisation a subset of eligible patients were identified to characterise practices; 40% had cholesterol levels below the target threshold. Improvements in data collection procedures allowed identification of all eligible patients (n = 7586) at follow up. Patients in practices allocated to SOS were significantly more likely to have cholesterol at target (69.5% vs 63.5%; OR 1.11, CI 1.00-1.23; p = 0.043) as a result of improved simvastatin prescribing. Subgroup analysis showed the primary outcome was achieved by prevalent but not incident patients. Statistically significant improvements occurred in all secondary outcomes for prevalent patients and all but one secondary outcome (the proportion of patients with cholesterol tested) for incident patients. SOS practices prescribed more simvastatin 40 mg than usual care practices, up to 10 years later.

INTERPRETATION

Through a combination of educational and organisational support, a general practice based pharmacist led collaborative intervention can improve statin prescribing and achievement of cholesterol targets in a high-risk primary care based population.

TRIAL REGISTRATION

International Standard Randomised Controlled Trials Register ISRCTN61233866.

Behaviour change interventions to promote prescribing of generic drugs: a rapid evidence synthesis and systematic review

OBJECTIVE

To summarise evidence on the effectiveness of behaviour change interventions to encourage prescribing of generic forms of prescription drugs where clinically appropriate in the UK National Health Service (NHS) and similar settings.

DESIGN

Systematic review.

SEARCH STRATEGY

We conducted a rapid evidence synthesis in two stages: First, we searched databases, such as the Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE), for systematic reviews of interventions that reported outcomes related to utilisation of generic drugs. In the second stage, we searched several databases including MEDLINE and EMBASE to identify primary studies of any interventions not covered by systematic reviews.

DATA EXTRACTION AND QUALITY ASSESSMENT

Data were extracted into a standardised data extraction form. Standardised quality assessment tools were used to assess study quality. Two reviewers were involved in data extraction and quality assessment.

RESULTS

10 reviews were included for the initial evidence synthesis, but most were of limited usefulness to our focused review question. One review evaluated the effect of prescribing policies using financial incentives and showed an increase in generic prescribing. Thirteen primary studies of other interventions were included for the rapid review. Two studies showed an increase in percentage of overall generic prescribing with an educational intervention; two studies showed an improvement in generic prescribing rates when physicians collaborated with pharmacists, though in one study this was not statistically significant; two US studies showed improvements in generic prescribing with electronic prescribing. Five out of seven studies showed positive results with multifaceted interventions.

CONCLUSIONS

The existing evidence remains insufficient to determine which behaviour change intervention or combination of interventions is most effective due to methodological weaknesses and conflicting results. Based on the evidence, financial incentives with educational intervention and audit/feedback look promising but decision-makers should take into account the practicality and costs of the interventions before implementation.

Does economic incentive matter for rational use of medicine? China's experience from the essential medicines program

BACKGROUND

Before the new round of healthcare reform in China, primary healthcare providers could obtain a fixed 15 % or greater mark-up of profits by prescribing and selling medicines. There were concerns that this perverse incentive was a key cause of irrational medicine use. China's new Essential Medicines Program (EMP) was launched in 2009 as part of the national health sector reform initiatives. One of its core policies was to eliminate primary care providers' economic incentives to overprescribe or prescribe unnecessarily expensive drugs, which were regarded as consequences of China's traditional financing system for health institutions.

OBJECTIVES

The objective of the study was to measure changes in prescribing patterns in primary healthcare facilities after the removal of the economic incentives for physicians to overprescribe as a result of the implementation of the EMP.

METHODS

A comparison design was applied to 8,258 prescriptions in 2007 and 8,278 prescriptions in 2010, from 83 primary healthcare facilities nationwide. Indicators were adopted to evaluate medicine utilization, which included overall number of medicines, average number of Western and traditional Chinese medicines, pharmaceutical expenditure per outpatient prescription, and proportion of prescriptions that contained two or more antibiotics. We further assessed the use of medicines (antibiotics, infusion, hormones, and intravenous injection) per disease-specific prescription for hypertension, diabetes, coronary artery heart disease, bronchitis, upper respiratory tract infection, and gastritis. A difference-in-difference analysis was employed to evaluate the net policy effect.

RESULTS

Overall changes in indicators were not found to be statistically significant between the 2 years. The results varied for different diseases. The number of Western drugs per outpatient prescription decreased while that of traditional Chinese medicines increased. Overuse of antibiotics remained an extensive problem in the treatment of many diseases, though there was some significant improvement in certain diseases, like diabetes in rural areas. Medicine expenditure per prescription also decreased.

CONCLUSIONS

It seems that the removal of a perverse economic incentive alone would not lead to improvement of healthcare providers' prescribing patterns. The rationality of the Essential Medicines List and the lack of payers' and providers' meaningful involvement in the development of the policy possibly contribute to the lack of significant changes in prescribing behaviors. It is suggested that China should adopt more comprehensive policies for healthcare facilities, physicians, patients, and payers, rather than just relying on economic incentives to improve rational use of medicines.

Prescription pattern and its influencing factors in Chinese county hospitals: a retrospective cross-sectional study

OBJECTIVE

This study aimed to investigate prescription patterns and influencing factors in Chinese county hospitals.

METHODS

Prescription quality was evaluated by five indicators proposed by WHO/INRUD. A questionnaire for doctors was designed by our research group. All internists, surgeons, obstetricians, gynecologists and pediatricians from 10 county hospitals in Anhui province were asked to fill the questionnaire. Their prescriptions from May 2011 to April 2012 were analyzed.

RESULTS

Three-hundred and thirty-seven doctors completed valid questionnaires, and 5099 prescriptions were analyzed. The average number of drugs per prescription was 3.52±2.31; the average percentage of generic drugs, antibiotic usage, injection drug usage, and drugs prescribed from the national essential drug list were 96.12%, 29.90%, 20.02% and 48.85%, respectively. Differences in final academic degree and specialty led to differences in all of the five prescription quality indicators. The older doctors tended to use more antibiotics. Doctors with more education, more training on rational drug use, and better acquisition of medicine knowledge prescribe a lower percentage of generic drugs. Moreover, the more supportive the doctor's attitude to national essential medicine policy, the higher the percentage of generic drugs were prescribed. A higher level of medical knowledge was associated with a higher percentage of drugs prescribed from the essential drugs list.

CONCLUSIONS

Promoting the education of medical knowledge on doctors, reinforcing the publicity of rational drug use to doctors, and initiating the performance evaluation for doctors are effective ways for improving prescription quality in Chinese county hospitals.

Change in antihypertensive drug prescribing after guideline implementation: a controlled before and after study

Background

Antihypertensive drug choices and treatment levels are not in accordance with the existing guidelines. We aimed to assess the impact of a guideline implementation intervention on antihypertensive drug prescribing.

Methods

In this controlled before and after study, the effects of a multifaceted (education, audit and feedback, local care pathway) quality programme was evaluated. The intervention was carried out in a health centre between 2002 and 2003. From each health care unit (n = 31), a doctor-nurse pair was trained to act as peer facilitators in the intervention.

All antihypertensive drugs prescribed by 25 facilitator general practitioners (intervention GPs) and 53 control GPs were retrieved from the nationwide Prescription Register for three-month periods in 2001 and 2003. The proportions of patients receiving specific antihypertensive drugs and multiple antihypertensive drugs were measured before and after the intervention for three subgroups of hypertension patients: hypertension only, with coronary heart disease, and with diabetes.

Results

In all subgroups, the use of multiple concurrent medications increased. For intervention patients with hypertension only, the odds ratio (OR) was 1.12 (95% CI 0.99, 1.25; p = 0.06) and for controls 1.13 (1.05, 1.21; p = 0.002). We observed no statistically significant differences in the change in the prescribing of specific antihypertensive agents between the intervention and control groups. The use of agents acting on the renin-angiotensin-aldosterone system increased in all subgroups (hypertension only intervention patients OR 1.19 (1.06, 1.34; p = 0.004) and controls OR 1.24 (1.15, 1.34; p < 0.0001).

Conclusions

A multifaceted guideline implementation intervention does not necessarily lead to significant changes in prescribing performance. Rigorous planning of the interventions and quality projects and their evaluation are essential.

Dementia diagnosis and management: a narrative review of changing practice

BACKGROUND

Early detection and management of dementia in primary care are difficult problems for practitioners. England's National Dementia Strategy 2009 seeks to improve these areas but there is limited evidence on how to achieve this most effectively.

AIM

This review aims to identify and appraise empirical studies of interventions designed to improve the performance of primary care practitioners in these areas.

DESIGN

A narrative review of primary-care based studies.

METHOD

Publications up to February 2010 were identified by searching the electronic databases MEDLINE, Embase, and PsycINFO, and bibliographies. The criterion for inclusion was that studies had to be of interventions aimed at improving detection or management of dementia in primary care. Exclusion criteria included studies in non-English publications, pharmacological interventions, and screening instrument studies. Quality was assessed using the PEDro (Physiotherapy Evidence Database) scale.

RESULTS

Fifteen studies were identified, of which 11 were randomised controlled trials. Eight reported educational interventions, and seven trialled service redesign, either by changing the service pathway or by introducing case management. Educationally, only facilitated sessions and decision-support software improved GPs' diagnosis of dementia, as did trials of service-pathway modification. Some of the case-management trials showed improved stakeholder satisfaction, decreased symptoms, and care that was more concordant with guidelines.

CONCLUSION

The quality of the studies varied considerably. Educational interventions are effective when learners are able to set their own educational agenda. Although modifying the service pathway and using case management can assist in several aspects of dementia care, these would require the provision of extra resources, and their value is yet to be tested in different health systems.

Quantitative evaluation of a clinical intervention aimed at changing prescriber behaviour in response to new guidelines

RATIONAL, AIMS AND OBJECTIVES

The objective of the study was to assess prescribers' compliance with guidelines for acute community-acquired pneumonia management in a paediatric university hospital centre before and after its dissemination.

METHOD

This quasi-experimental study without a control group was conducted before and after new community-acquired pneumonia management guidelines were disseminated in a tertiary care paediatric hospital. The pre-intervention (baseline) period was from October 2004 to March 2005. The intervention period was divided into two phases: (1) October 2005 to January 14, 2006 (consultation by peer leaders and networking) and (2) January 15, 2006, to March 2006 (dissemination of official guidelines and of a pre-printed prescription sheet, an educational session led by a peer leader for residents and further networking). We used a compliance score to assess prescriptions written by prescribers who practised in the units where the guidelines had to be followed.

RESULTS

The study included a total of 1151 prescriptions. The prescription compliance with the guidelines increased from 131/652 (20.1%) in the pre-intervention period to 264/499 (52.9%) in the post-intervention intervention period: a difference of 32.8% (CI 95% 27.4-38.0). Similar results were found if analysed according to affiliation (emergency department or wards). An inappropriate choice of antibiotic agent represented 347/521 (66.6%) of the causes of non-compliance in the pre-intervention period and 99/235 (42.1) in the intervention period: a difference of -24.5% (95% CI -31.8, -16.8).

CONCLUSION

Guideline dissemination for the management of acute community-acquired pneumonia significantly increased prescriber compliance in the emergency department and on wards.

Promoting free online CME for intimate partner violence: what works at what cost?

INTRODUCTION

There is a need to provide practicing physicians with training on the recognition and management of intimate partner violence (IPV). Online continuing medical education (CME) could help meet this need, but there is little information on the costs and effectiveness of promoting online CME to physicians. This lack of information may discourage IPV training efforts and the use of online CME in general.

METHODS

We promoted an interactive, multimedia, online IPV CME program, which offered free CME credit, to 92,000 California physicians for 24 months. We collected data on user satisfaction, the costs of different promotional strategies, and self-reported user referral source. We evaluated California physician awareness of the promotion via telephone surveys.

RESULTS

Over 2 years, the CME program was used by 1869 California physicians (2% of market), who rated the program's overall quality highly (4.52 on a 1-5 scale; 5 = excellent). The average promotional cost per physician user was $75. Direct mail was the most effective strategy, costing $143 each for 821 users. E-promotion via search engine advertising and e-mail solicitation had less reach, but was more cost efficient ($30-$80 per user). Strategies with no direct cost, such as notices in professional newsletters, accounted for 31% (578) of physician users. Phone surveys found that 24% of California physicians were aware of the online IPV CME program after 18 months of promotion.

DISCUSSION

Promoting online CME, even well-received free CME, to busy community physicians requires resources, in this case at least $75 per physician reached. The effective use of promotional resources needs to be considered when developing social marketing strategies to improve community physician practices. Organizations with an interest in promoting online training might consider the use of e-promotion techniques along with conventional promotion strategies.

Effect of educational outreach on general practice prescribing of antibiotics and antidepressants: a two-year randomised controlled trial

OBJECTIVE

Prescribing of broad spectrum antibiotics and antidepressants in general practice often does not accord with guidelines. The aim was to determine the effectiveness of educational outreach in improving the prescribing of selected antibiotics and antidepressants, and whether the effect is sustained for two years.

DESIGN

Single blind randomized trial.

SETTING

Twenty-eight general practices in Leicestershire, England.

INTERVENTION

Educational outreach visits were undertaken, tailored to barriers to change, 14 practices receiving visits for reducing selected antibiotics and 14 for improving antidepressant prescribing.

MAIN OUTCOME MEASURES

Number of items prescribed per 1000 registered patients for amoxicillin with clavulanic acid (co-amoxiclav) and quinolone antibiotics, and average daily quantities per 1000 patients for lofepramine and fluoxetine antidepressants, measured at the practice level for six-month periods over two years.

RESULTS

There was no effect on the prescribing of co-amoxiclav, quinolones, or fluoxetine, but prescribing of lofepramine increased in accordance with the guidelines. The increase persisted throughout two years of follow-up.

CONCLUSION

A simple, group-level educational outreach intervention, designed to take account of identified barriers to change, can have a modest but sustained effect on prescribing levels. However, outreach is not always effective. The context in which change in prescribing practice is being sought, the views of prescribers concerning the value of the drug, or other unrecognised barriers to change may influence the effectiveness of outreach.

CRISTOPH - a cluster-randomised intervention study to optimise the treatment of patients with hypertension in General Practice

Background

Recent guidelines for the management of hypertension focus on treating patients according to their global cardiovascular risk (CVR), rather than strictly keeping blood pressure, or other risk factors, below set limit values. The objective of this study is to compare the effect of a simple versus a complex educational intervention implementing this new concept among General Practitioners (GPs).

Methods/design

A prospective longitudinal cluster-randomised intervention trial with 94 German GPs consecutively enroling 40 patients each with known hypertension. All GPs then received a written manual specifically developed to transfer the global concept of CVR into daily General Practice. After cluster-randomisation, half of the GPs additionally received a clinical outreach visit, with a trained peer discussing with them the concept of global CVR referring to example study patients from the respective GP. Main outcome measure is the improvement of calculated CVR six months after intervention in the subgroup of patients with high CVR (but no history of cardiovascular disease), defined as 10-year-mortality ≥ 5% employing the European SCORE formula. Secondary outcome measures include the intervention's effect on single risk factors, and on prescription rates of drugs targeting CVR. All outcome measures are separately studied in the three subgroups of patients with 1. high CVR (defined as above), 2. low CVR (SCORE < 5%), and 3. a history of cardiovascular disease. The influence of age, sex, social status, and the perceived quality of the respective doctor-patient-relation on the effects will be examined.

Discussion

To our knowledge, no other published intervention study has yet evaluated the impact of educating GPs with the goal to treat patients with hypertension according to their global cardiovascular risk.

Trial registration

ISRCTN44478543

Disseminating findings from a drug class review: using best practices to inform prescription of antiepileptic drugs for bipolar disorder

Evidence from drug class reviews is often not accessible to practicing clinicians nor is it presented in a way that allows clinicians to use the information to guide treatment and prescribing decisions. Nevertheless, information from such reviews can be very helpful to clinicians as they evaluate the "evidence" provided to them through marketing strategies implemented, primarily, by the pharmaceutical industry and designed to influence their prescribing behavior. Unfortunately, these marketing strategies can be used to promote the off-label use of drugs that may not be efficacious. One example is the pharmaceutical marketing to promote off-label use of gabapentin (Neurontin) for the treatment of bipolar disorder, the legality of which was later addressed in a major lawsuit by the National Association of Attorneys General. We describe an effort to use counter-marketing strategies to compete with those implemented by the pharmaceutical industry and to help clinicians, principally psychiatrists, make use of available evidence to inform their prescription of antiepileptic drugs (AEDs) in the treatment of bipolar disorder. A growing body of literature describes industry marketing practices designed to influence prescriber behavior. This literature suggests that use of competing approaches involving the same underlying strategies to deliver highly credible information from trusted sources can inform prescriber knowledge and prescribing practice. We describe our use of existing evidence to develop accurate and convincing messages and materials to be disseminated nationally to counter industry misinformation and promote evidence-based prescription of AEDs.

Self-reported performance improvement strategies of highly successful Veterans Health Administration facilities

The Veterans Health Administration (VHA) has achieved considerable success in improving health care through the use of clinical performance measures. This report examines the self-reported strategies used by the most successful facilities in the VHA system. For fiscal year 2002, facilities that scored the highest on any of 24 clinical performance measures were queried as to which strategies they used to achieve their level of performance. The most commonly cited strategies across all performance categories were organizational change (55.6%), clinical reminders (41.4%), audit and feedback to providers (39.6%), and staff education (32.5%). Certain strategies were more likely to be cited for 1 or more specific performance categories (eg, clinical reminders for immunization [61.1%], screening [60.6%]). These findings suggest that successful facilities are generally using evidence-based strategies to achieve high clinical performance. However, some evidence-based implementation strategies were rarely cited (eg, use of clinical champions).

What is the role of quality circles in strategies to optimise antibiotic prescribing? A pragmatic cluster-randomised controlled trial in primary care

OBJECTIVE

To evaluate the effect on antibiotic prescribing of an intervention in existing local quality circles promoting an evidence-based guideline for acute rhinosinusitis.

DESIGN

A pragmatic cluster-randomised controlled trial comparing standard dissemination of the guideline by mail with an additional strategy using quality circles.

SETTING

General practice in Flanders, Belgium.

PARTICIPANTS

General practitioners (GPs) in 18 local quality circles were randomly allocated to two study arms. All GPs received the guideline by mail. GPs in the nine quality circles allocated to the intervention arm received an additional group intervention, which consisted of one self-led meeting using material introduced to the group moderator by a member of the research team.

MAIN OUTCOME MEASURES

Adherence to the guideline was measured as differences in the proportion of antibiotic prescriptions, including the choice of antibiotic, between the two study arms after the intervention period. GPs registered their encounters with patients presenting with signs and symptoms of acute rhinosinusitis in a booklet designed for the study.

RESULTS

A total of 75 doctors (29% of GPs in the participating quality circles) registered 408 consultations. In the intervention group, 56.9% of patients received an antibiotic compared with 58.3% in the control group. First-choice antibiotics were issued in 34.5% of antibiotic prescriptions in the intervention group compared with 29.4% in the control group. After adjusting for patient and GP characteristics, the ORadj for antibiotics prescribed in the intervention arm compared with the control arm was 0.63 (95% CI 0.29 to 1.37). There was no effect on the choice of antibiotic (ORadj 1.07, 95% CI 0.34 to 3.37).

CONCLUSION

A single intervention in quality circles of GPs integrated in the group's normal working procedure did not have a significant effect on the quality of antibiotic prescribing. More attention to the context and structure of primary care practice, and insight into the process of self-reflective learning may provide clues to optimise the effectiveness of quality circles.

Barriers to academic detailers: a qualitative study in general practice

BACKGROUND

International research has proved that academic detailing positively influences the prescribing behaviour of physicians. In the North-East Flanders region of Belgium, a quantitative study of the feasibility and acceptability of academic detailing in general practice has demonstrated that academic detailers are appreciated by physicians.

OBJECTIVE

In addition to the quantitative study, a qualitative study was carried out to determine physicians' attitudes to academic detailing and their objections against it.

METHODS

This study took place in general practice in the North-East Flanders region of Belgium, using semistructured interviews. Nineteen physicians who had previously had a visit from an academic detailer in the quantitative study, and 19 who had refused were invited for an interview.

RESULTS

Eighteen physicians who had been visited by an academic detailer, and nine who had refused a visit agreed to be interviewed. Although most physicians who had had a visit from an academic detailer appreciated the visit, some barriers were revealed. Physicians had doubts about the objectivity, stated that the information was not new or that they had other ways to obtain information. These barriers were also mentioned by the physicians who had refused a visit. Additional barriers were that the visits were time-consuming, meant to cut expenses, politically coloured, and patronising.

CONCLUSION

Doubts about independence and objectivity of the information were stated as important barriers both by physicians who had accepted a visit from an academic detailer and by those who had not. These issues need to be dealt with when designing future interventions with academic detailers.

Effects of a distance learning program on physicians' opioid- and benzodiazepine-prescribing skills

INTRODUCTION

Opioid misuse is common among patients with chronic nonmalignant pain. There is a pressing need for physicians to increase their confidence and competence in managing these patients.

METHODS

A randomized controlled trial of family physicians (N = 88) attending 1 of 4 continuing medical education events helped to determine the effectiveness of e-mail case discussions in changing physician behavior. Before random assignment, participants completed a pretest and attended a 3-hour didactic session on prescribing opioids and benzodiazepines. The intervention group participated in 10 weeks of e-mail case discussions, with designated participants responding to questions on cases. An addictions physician facilitated the discussion. Several months after the e-mail discussion, participants took part in a mock telephone consultation; a blinded researcher posing as a medical colleague asked for advice about 2 cases involving opioid and benzodiazepine prescribing. Using a checklist, the researcher recorded the questions asked and advice given by the physician.

RESULTS

On post-testing, both groups expressed greater optimism about treatment outcomes and were more likely to report using a treatment contract and providing advice about sleep hygiene. There were no significant differences between pretesting and post-testing between the groups on the survey. During the telephone consultation, the intervention group asked significantly more questions and offered more advice than the control group (odds ratio for question items, 1.27 [p = .03]; advice items, 1.33 [p = .01).

DISCUSSION

Facilitated by electronic mail and a medical expert, case discussion is an effective means of improving physician performance. Telephone consultation holds promise as a method for evaluating physicians' assessment and management skills.

[Cost-effectiveness and cost calculation in an intervention on medication-related problems in primary care]

OBJECTIVES

To assess the effectiveness of an intervention on medication-related problems (MRPs) detected in chronic patients with multiple medication at an urban primary care centre (PCC). Direct costs of the intervention were calculated.

DESIGN

Prospective and longitudinal intervention study.

SETTING

Primary care centre of Barceloneta. Barcelona, Spain.

PARTICIPANTS

155 patients receiving multiple medication over 8 months.

INTERVENTIONS

The PCC pharmacist informed the doctors in writing of the MRPs detected, the change in therapy recommended and the follow-up to the decision.

MAIN MEASUREMENTS

Number of MRPs, number of patients involved, age, number of drugs prescribed, and therapy groups involved.

RESULTS

161 MRPs were detected, with 92 patients involved of whom 57% were women. Average age was 72 and the average number of drugs prescribed was 8. 60% of the MRPs detected were within the safety category. There was an intervention in 100% of the MRPs detected and 32.3% were sorted out. The main limitation on doctors accepting the pharmacist's recommendation was that the MRPs detected were potential rather than real. The programme s saving was calculated at 2,588 euros (for direct costs only). The direct costs saved were 53.92 euros per MRP avoided.

CONCLUSIONS

The intervention on the MRPs detected in the PCC with the methodology used was effective. The direct costs saved per MRP sorted out exceeded the cost of investment in the programme.

Impact of a general practitioner educational intervention on osteoarthritis treatment in an elderly population

PURPOSE

We examined whether a continuing medical education intervention increased general practitioners' ability to select the proper pharmacological treatment for patients with osteoarthritis.

SUBJECTS AND METHODS

Eight towns in Quebec, Canada were randomly allocated to one of four intervention options, workshop and decision tree, workshop, decision tree, or no intervention. All general practitioners practicing in each town were eligible to participate. We evaluated all dispensed prescriptions for either a cyclooxygenase (COX)-2 inhibitor, nonselective nonsteroidal anti-inflammatory drug or acetaminophen written by eligible general practitioners between May 2000 and June 2001 to elderly patients suffering from osteoarthritis. We used a multi-level Bayesian hierarchical model to assess the impact of the interventions on prescription adequacy.

RESULTS

We analyzed 5318 dispensed prescriptions written by 249 general practitioners in the five-month preintervention period and 4610 dispensed prescriptions written by the same physicians in the five-month postintervention period. A score of zero or one was given to every prescription, with one indicating prescription adequacy according to guidelines provided during the interventions. Bayesian hierarchical models showed some improvement in scores in the post- versus preintervention periods in all four groups. The probability of an improvement in the towns allocated the workshop and decision tree over the control was 94%, compared with 74% in the workshop group and 55% in the decision tree group.

CONCLUSION

An interactive approach offered by peers and complemented by easy to use guidelines may enhance the general practitioner's ability to manage osteoarthritis patients.

Promoting a biopsychosocial orientation in family practice: effect of two teaching programs on the knowledge and attitudes of practising primary care physicians

The bio-psychosocial (BPS) approach to patient care has gained acceptance in medical education. However, reported teaching programs rarely describe the efficacy of alternative approaches to continuing medical education aimed at promoting a BPS approach. The objective was to describe and evaluate the effect of two teaching programs on learners' BPS knowledge, management intentions, patient-centered attitudes, professional self-esteem, burnout, work related strain and mental workload. The learners were Israeli general practitioners. The first ("didactic") program consisted of problem-based reading assignments, lectures and discussions. The second ("interactive") program consisted of reading assignments, lectures and discussions, in addition to role-playing exercises, Balint groups and one-to-one counseling by a facilitator. One month before and six months after the teaching interventions, we used structured questionnaires to test for knowledge, management intentions (responses to questions, such as "what would you tell a patient with ...") and attitudes. Both programs led to measurable improvement in knowledge, intentions, patient-centered attitudes and self-esteem. The interactive teaching approach improved significantly more the learners' professional self-esteem and intentions than the didactic approach. Self-reported burnout significantly increased after the program. It is concluded that teaching intervention enhanced a BPS orientation and led to changes in knowledge, intentions, self-esteem and attitudes. An interactive method of instruction was more effective in achieving some of these objectives than a didactic one. The observed increase in burnout was unexpected and requires further study and confirmation.

Prescription audit adjunct to rational pharmacotherapy education improves prescribing skills of medical students

OBJECTIVE

To assess the impact of a rational pharmacotherapy (RP) teaching program during clinical pharmacology clerkship by analyzing the results of prescription audits (PAs) of the medical students. Collectively, we intended to observe the possible improvement of the students in their prescribing, problem solving and self-directed learning skills.

DESIGN

At the beginning and end of the clerkship, the students were presented with cases of uncomplicated osteoarthritis to assess their prescribing skills; format and rationality were scored.

SETTING

A medical school in Turkey that teaches RP to the fourth-year students in clinical pharmacology clerkship.

PARTICIPANTS

There were 94 students of the 2002-2003 academic year in three groups and a single group of students belonging to the previous academic year tested. Of those students from the previous academic year, 26 were also analyzed a year later to demonstrate the long-term impact of the training.

MAIN OUTCOME MEASURES

Prescribing skills of medical students and their opinions about PA.

RESULTS

Direct assessment via PA demonstrated that the scores for post-clerkship prescriptions were far better than those for pre-clerkship prescriptions in terms of format and rationality. Long-term assessment showed that the scores declined within a year following clerkship, but they were still higher than those of their pre-clerkship scripts. Analysis of the questionnaires revealed that the students were satisfied with PA. The majority of the students stated they had learned the general principles of RP and gained better prescribing skills, and they intended to apply most of the principles learned to their future professional lives. The script format scores of a retrospectively created PA-exempted group were significantly lower than those of the students to whom an established PA education was given.

CONCLUSION

PA sessions were shown to be an easy and a useful method of both evaluating and reinforcing prescribing skills gained though problem-based RP education.

Factors that influence spontaneous reporting of adverse drug reactions: a model centralized in the medical professional

RATIONALE, AIMS AND OBJECTIVES

The spontaneous reporting of adverse drug reactions (ADRs) through the yellow card and made concrete by the knowledge and attitudes of doctors, has been rousing a great deal of bibliographical interest in recent years. However, there does not seem to be any actual revision in the theme on which the theoretical models that explain the process of decision in reporting are proposed. In this work an explanatory model of the factors that condition reporting is proposed and a revision of the literature on the subject has also been carried out.

METHODS

The proposed model is centralized in the medical professional and it considers the habit of reporting as the result of the doctor's formation and his interaction with the environment. The combination of knowledge-attitudes-practices and the theory of the satisfaction of needs seemed very adequate for ADR systematization.

RESULTS AND CONCLUSIONS

The results also indicate that, to improve the participation of health professionals in surveillance systems through spontaneous reporting, it might be necessary to design combined strategies that modify both intrinsic (knowledge, attitudes) and extrinsic (relationship between health professionals and their patients, the national health system and pharmaceutical companies) factors.

Efficacy of Educational Interventions Targeting Primary Care Providers' Practice Behaviors: an Overview of Published Systematic Reviews

OBJECTIVES

Primary care providers (e.g., family physicians, pediatricians, registered nurses, physician assistants, and nurse practitioners) could play a pivotal role in the provision of preventive services, especially for very young children (younger than 3 years old) and population groups with limited access to dental care. Given the current problems with access to dental care among low-income Americans, we contend there is a need to involve nondental primary health care providers in screening for and preventing oral health problems. The objective of this overview is to present findings from systematic reviews on the efficacy of continuing medical education, printed educational material, academic outreach, reminders, and local opinion leaders on the adoption of new knowledge and practices by primary care providers.

METHODS

A search was conducted using the Cochrane Library and MEDLINE. The search aimed to locate systematic reviews published between January 1988 and March 2003. Two researchers independently extracted data and assessed study quality using a modified version of the QUOROM statement.

RESULTS

Eleven systematic reviews were included in this overview. The evidence from the included systematic reviews showed that formal continuing medical education (CME) and distributing educational materials did not effectively change primary care providers' behaviors. There are effective interventions available to increase knowledge and change behaviors of primary care providers, such as small group discussion, interactive workshops, educational outreach visits, and reminders.

CONCLUSION

There is a limited knowledge base on the efficacy of the selected interventions on oral health screening by primary care providers. Considering the potential role of primary care providers in improving oral health of underserved populations, this research area should receive more attention.

Quality of prescribing for the elderly in Croatia-computerized pharmacy data can be used to screen for potentially inappropriate prescribing

OBJECTIVE

The quality of medication use by the elderly is a major safety concern because of an increasing number of people in this segment of the population and the fact that they are major drug consumers. The objective of this study was to test whether administrative computerized pharmacy prescription data could be used to assess the prevalence of potentially inappropriate medication use by the elderly in Rijeka, Croatia, using two sets of explicit criteria data.

METHODS

In 2002, the prevalence of potentially inappropriate medication use in the elderly aged 70 years or over was evaluated using Beers' criteria. Inappropriate co-prescribing was evaluated using a list of eight potentially harmful drug combinations. Information on the use of drugs was collected from a computerized pharmacy record of all prescriptions dispensed in the "Jadran Pharmacies" in Rijeka.

RESULTS

Five computerized pharmacies dispensed 306,000 (65% of all prescriptions in Rijeka); 78,000 were for those aged 70 years and over. Among these elderly, the prevalence of inappropriate choice of drugs was 2.2% and inadequate co-prescribing 1.9%. The most common inappropriate drug was a long-acting benzodiazepine, diazepam. Mostly used potentially harmful combinations were a non-steroidal anti-inflammatory drug (NSAID) co-prescribed with a diuretic, and two benzodiazepines co-prescribed.

CONCLUSION

We found that computerized pharmacy data can be used as a screening tool in efforts to improve the quality of drug prescribing in Rijeka. Explicit criteria for assessing inappropriate prescribing for the elderly can easily be applied to population based surveys. This study suggests a low rate of potentially inappropriate prescribing for the elderly in Rijeka.

Benefit coverage of novel antipsychotics in Medicaid programs and the Veterans Health Administration

When risperidone, olanzapine, and quetiapine were introduced, concerns were raised regarding the extent to which drug benefit policies might constrain their use. A national survey of eligible Medicaid programs (N = 47; 100%) and Veterans Health Administration facilities (N = 141; 94%) in 1998 found that within Medicaid, open formularies were common, preauthorization requirements were rare, and few barriers existed, whereas VHA facilities reported relatively more constraints in terms of formulary restrictions and preauthorization requirements. Although drug benefit policies have the potential to exert a major influence over prescribing practices, it is unlikely that these policies significantly restricted access to these antipsychotics.

Impact studies in continuing education for health professions: update

INTRODUCTION

This article critiques the questions asked and methods used in research syntheses in continuing education (CE) in the health professions, summarizes the findings of the syntheses, and makes recommendations for future CE research and practice.

METHODS

We identified 15 research syntheses published after 1993 in which primary CE studies were reviewed and the performance (behavior) of health professionals and/or patient health outcomes were examined.

RESULTS

The syntheses were categorized by the research questions they asked using a wave metaphor. Wave One (n = 3) syntheses sought to establish a general descriptive causal connection between CE and impact variables, asking, "Is CE effective, and for what outcomes?" Wave Two syntheses (n = 12) sought to explain the relationship between CE and impacts by identifying causal moderators, asking, "What kinds of CE are effective?"

DISCUSSION

Wave One findings confirm previous research that CE can improve knowledge, skills, attitudes, behavior, and patient health outcomes. Wave Two syntheses show that CE, which is ongoing, interactive, contextually relevant, and based on needs assessment, can improve knowledge, skills, attitudes, behavior, and health care outcomes. The most important implication of the present review is that there are differential impacts of CE programs, and the reasons for those impacts cannot be fully understood unless the context of the program is considered.