Effect of linking practice data to published evidence. A randomized controlled trial of clinical direct reports

Identifying behaviour change techniques in 287 randomized controlled trials of audit and feedback interventions targeting practice change among healthcare professionals

BACKGROUND

Audit and feedback (A&F) is among the most widely used implementation strategies, providing healthcare professionals with summaries of their practice performance to prompt behaviour change and optimize care. Wide variability in effectiveness of A&F has spurred efforts to explore why some A&F interventions are more effective than others. Unpacking the variability of the content of A&F interventions in terms of their component behaviours change techniques (BCTs) may help advance our understanding of how A&F works best. This study aimed to systematically specify BCTs in A&F interventions targeting healthcare professional practice change.

METHODS

We conducted a directed content analysis of intervention descriptions in 287 randomized trials included in an ongoing Cochrane systematic review update of A&F interventions (searched up to June 2020). Three trained researchers identified and categorized BCTs in all trial arms (treatment & control/comparator) using the 93-item BCT Taxonomy version 1. The original BCT definitions and examples in the taxonomy were adapted to include A&F-specific decision rules and examples. Two additional BCTs ('Education (unspecified)' and 'Feedback (unspecified)') were added, such that 95 BCTs were considered for coding.

RESULTS

In total, 47/95 BCTs (49%) were identified across 360 treatment arms at least once (median = 5.0, IQR = 2.3, range = 129 per arm). The most common BCTs were 'Feedback on behaviour' (present 89% of the time; e.g. feedback on drug prescribing), 'Instruction on how to perform the behaviour' (71%; e.g. issuing a clinical guideline), 'Social comparison' (52%; e.g. feedback on performance of peers), 'Credible source' (41%; e.g. endorsements from respected professional body), and 'Education (unspecified)' (31%; e.g. giving a lecture to staff). A total of 130/287 (45%) control/comparator arms contained at least one BCT (median = 2.0, IQR = 3.0, range = 0-15 per arm), of which the most common were identical to those identified in treatment arms.

CONCLUSIONS

A&F interventions to improve healthcare professional practice include a moderate range of BCTs, focusing predominantly on providing behavioural feedback, sharing guidelines, peer comparison data, education, and leveraging credible sources. We encourage the use of our A&F-specific list of BCTs to improve knowledge of what is being delivered in A&F interventions. Our study provides a basis for exploring which BCTs are associated with intervention effectiveness.

TRIAL REGISTRATIONS

N/A.

Peritoneal Dialysis: A Scoping Review of Strategies to Maximize pd Utilization

The percentage of end-stage renal disease (ESRD) patients treated with peritoneal dialysis (PD) has declined in many countries since the mid-1990s. Barriers to PD have been reviewed extensively in the literature, but evidence about strategies to address these barriers and maximize the safe and effective use of PD is lacking. We therefore decided to conduct a scoping review identifying strategies to maximize PD use in adults with ESRD. Our search strategy included the following online databases: MEDLINE (OVID), EMBASE, PubMed, Cochrane Controlled Trials Register, Current Controlled Trials, and Cochrane Database of Systematic Reviews for articles published from 1974 to November 2013. Experts in the field were contacted for information about other ongoing or unpublished studies. A complementary search was conducted in the gray literature. Websites of national, provincial or regional agencies were searched for documents regarding policies surrounding the use of PD. Individual dialysis centers need to identify barriers to increasing PD in their program and direct targeted strategies to maximize PD utilization. Our review highlights some effective strategies that may be used. Our review also highlights the need for further research into strategies to maximize PD utilization.

Determinants of evidence use in public health policy making: Results from a study across six EU countries

The knowledge-practice gap in public health is widely known. The importance of using different types of evidence for the development of effective health promotion has also been emphasized. Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking the most effective approaches, thus limiting the impact of health promotion strategies. This article focuses on facilitators and barriers in the use of evidence in developing health enhancing physical activity policies. Data was collected in 2012 by interviewing 86 key stakeholders from six EU countries (FI, DK, UK, NL, IT, RO) using a common topic guide. Content analysis and concept mapping was used to construct a map of facilitators and barriers. Barriers and facilitators experienced by most stakeholders and policy context in each country are analysed. A lack of locally useful and concrete evidence, evidence on costs, and a lack of joint understanding were specific hindrances. Also users' characteristics and the role media play were identified as factors of influence. Attention for individual and social factors within the policy context might provide the key to enhance more sustainable evidence use. Developing and evaluating tailored approaches impacting on networking, personal relationships, collaboration and evidence coproduction is recommended.

Measuring prevalence, reliability and variation in high-risk prescribing in general practice using multilevel modelling of observational data in a population database

Background

High-risk primary care prescribing is common and is known to vary considerably between practices, but the extent to which high-risk prescribing varies among individual general practitioners (GPs) is not known.

Objectives

To create prescribing safety indicators usable in existing electronic clinical data and to examine (1) variation in high-risk prescribing between patients, GPs and practices including reliability of measurement and (2) changes over time in high-risk prescribing prevalence and variation between practices.

Design

Descriptive analysis and multilevel logistic regression modelling of routine data.

Setting

UK general practice using routine electronic medical record data.

Participants

(1) For analysis of variation and reliability, 398 GPs and 26,539 patients in 38 Scottish practices. (2) For analysis of change in high-risk prescribing, ≈ 300,000 patients particularly vulnerable to adverse drug effects registered with 190 Scottish practices.

Main outcome measures

For the analysis of variation between practices and between GPs, five indicators of high-risk non-steroidal anti-inflammatory drug (NSAID) prescribing. For the analysis of change in high-risk prescribing, 19 previously validated indicators.

Results

Measurement of high-risk prescribing at GP level was feasible only for newly initiated drugs and for drugs similar to NSAIDs which are usually initiated by GPs. There was moderate variation between practices in total high-risk NSAID prescribing [intraclass correlation coefficient (ICC) 0.034], but this indicator was highly reliable (> 0.8 for all practices) at distinguishing between practices because of the large number of patients being measured. There was moderate variation in initiation of high-risk NSAID prescribing between practices (ICC 0.055) and larger variation between GPs (ICC 0.166), but measurement did not reliably distinguish between practices and had reliability > 0.7 for only half of the GPs in the study. Between quarter (Q)2 2004 and Q1 2009, the percentage of patients exposed to high-risk prescribing measured by 17 indicators that could be examined over the whole period fell from 8.5% to 5.2%, which was largely driven by reductions in high-risk NSAID and antiplatelet use. Variation between practices increased for five indicators and decreased for five, with no relationship between change in the rate of high-risk prescribing and change in variation between practices.

Conclusions

High-risk prescribing is common and varies moderately between practices. High-risk prescribing at GP level cannot be easily measured routinely because of the difficulties in accurately identifying which GP actually prescribed the drug and because drug initiation is often a shared responsibility with specialists. For NSAID initiation, there was approximately three times greater variation between GPs than between practices. Most GPs with above average high-risk prescribing worked in practices which were not themselves above average. The observed reductions in high-risk prescribing between 2004 and 2009 were largely driven by falls in NSAID and antiplatelet prescribing, and there was no relationship between change in rate and change in variation between practices. These results are consistent with improvement interventions in all practices being more appropriate than interventions targeted on practices or GPs with higher than average high-risk prescribing. There is a need for research to understand why high-risk prescribing varies and to design and evaluate interventions to reduce it.

Funding

Funding for this study was provided by the Health Services and Delivery Research programme of the National Institute for Health Research.

Audit and feedback: effects on professional practice and healthcare outcomes

BACKGROUND

Audit and feedback is widely used as a strategy to improve professional practice either on its own or as a component of multifaceted quality improvement interventions. This is based on the belief that healthcare professionals are prompted to modify their practice when given performance feedback showing that their clinical practice is inconsistent with a desirable target. Despite its prevalence as a quality improvement strategy, there remains uncertainty regarding both the effectiveness of audit and feedback in improving healthcare practice and the characteristics of audit and feedback that lead to greater impact.

OBJECTIVES

To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes and to examine factors that may explain variation in the effectiveness of audit and feedback.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2010, Issue 4, part of The Cochrane Library. www.thecochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 December 2010); MEDLINE, Ovid (1950 to November Week 3 2010) (searched 09 December 2010); EMBASE, Ovid (1980 to 2010 Week 48) (searched 09 December 2010); CINAHL, Ebsco (1981 to present) (searched 10 December 2010); Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present) (searched 12-15 September 2011).

SELECTION CRITERIA

Randomised trials of audit and feedback (defined as a summary of clinical performance over a specified period of time) that reported objectively measured health professional practice or patient outcomes. In the case of multifaceted interventions, only trials in which audit and feedback was considered the core, essential aspect of at least one intervention arm were included.

DATA COLLECTION AND ANALYSIS

All data were abstracted by two independent review authors. For the primary outcome(s) in each study, we calculated the median absolute risk difference (RD) (adjusted for baseline performance) of compliance with desired practice compliance for dichotomous outcomes and the median percent change relative to the control group for continuous outcomes. Across studies the median effect size was weighted by number of health professionals involved in each study. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient, and risk of bias within the trial itself. We also conducted exploratory analyses to assess the role of context and the targeted clinical behaviour. Quantitative (meta-regression), visual, and qualitative analyses were undertaken to examine variation in effect size related to these factors.

MAIN RESULTS

We included and analysed 140 studies for this review. In the main analyses, a total of 108 comparisons from 70 studies compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. After excluding studies at high risk of bias, there were 82 comparisons from 49 studies featuring dichotomous outcomes, and the weighted median adjusted RD was a 4.3% (interquartile range (IQR) 0.5% to 16%) absolute increase in healthcare professionals' compliance with desired practice. Across 26 comparisons from 21 studies with continuous outcomes, the weighted median adjusted percent change relative to control was 1.3% (IQR = 1.3% to 28.9%). For patient outcomes, the weighted median RD was -0.4% (IQR -1.3% to 1.6%) for 12 comparisons from six studies reporting dichotomous outcomes and the weighted median percentage change was 17% (IQR 1.5% to 17%) for eight comparisons from five studies reporting continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats, and when it includes both explicit targets and an action plan. In addition, the effect size varied based on the clinical behaviour targeted by the intervention.

AUTHORS' CONCLUSIONS

Audit and feedback generally leads to small but potentially important improvements in professional practice. The effectiveness of audit and feedback seems to depend on baseline performance and how the feedback is provided. Future studies of audit and feedback should directly compare different ways of providing feedback.

Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies

Recent studies showed wide variation in the extent to which guidelines and other types of best practice have been implemented as part of routine health care. This is also true for the delivery of renal replacement therapy (RRT) for ESRD patients. Increasing uptake of best practice within such complex care systems requires an understanding of implementation strategies and specific quality improvement (QI) techniques. Therefore, we systematically reviewed over 5000 titles published since 1990 and included papers describing planned attempts to accelerate uptake of best RRT practice into daily care. This resulted in a list of 93 QI initiatives, categorized in order to expedite shared learning. The majority of the initiatives were executed within the domains of vascular access, nutrition, and anemia management. Strategies oriented at patients were most common and many initiatives pre-defined an improvement target before starting implementation. Of the 93 initiatives, 22 were sufficiently robust methodologically to be analyzed in more detail. Our results tend to support previous findings that multifaceted strategies are more effective than single strategies. Improving our understanding of how to successfully implement best practice can inform system-level change and is the only way to close the gap between knowledge on what works and the actual care delivered to ESRD patients. Research into implementation, using specific QI techniques, should therefore be given priority in future.

Addressing the unit of analysis in medical care studies: a systematic review

OBJECTIVE

We assessed the frequency that patients are incorrectly used as the unit of analysis among studies of physicians' patient care behavior in articles published in high impact journals.

METHODS

We surveyed 30 high-impact journals across 6 medical fields for articles susceptible to unit of analysis errors published from 1994 to 2005. Three reviewers independently abstracted articles using previously published criteria to determine the presence of analytic errors.

RESULTS

One hundred fourteen susceptible articles were found published in 15 journals, 4 journals published the majority (71 of 114 or 62.3%) of studies, 40 were intervention studies, and 74 were noninterventional studies. The unit of analysis error was present in 19 (48%) of the intervention studies and 31 (42%) of the noninterventional studies (overall error rate 44%). The frequency of the error decreased between 1994-1999 (N = 38; 65% error) and 2000-2005 (N = 76; 33% error) (P = 0.001).

CONCLUSIONS

Although the frequency of the error in published studies is decreasing, further improvement remains desirable.

Impact of short evidence summaries in discharge letters on adherence of practitioners to discharge medication. A cluster-randomised controlled trial

BACKGROUND

International concern about quality of medical care has led to intensive study of interventions to ensure care is consistent with best evidence. Simple, inexpensive, feasible and effective interventions remain limited.

OBJECTIVE

We examined the impact of one-sentence evidence summaries appended to consultants' letters to primary care practitioners on adherence of the practitioners to recommendations made by the consultants regarding medication for patients with chronic medical problems.

DESIGN

Cluster-randomised trial.

SETTING

Secondary/primary care interface (urban district hospital/referral practices).

PARTICIPANTS

178 practices received one or more discharge letters with evidence summaries. The 66 practices in the intervention group provided feedback on 172 letters, and the 56 practices in the control group provided feedback on 96 letters.

RESULTS

Appending an evidence summary to discharge letters resulted in a decrease in non-adherence to discharge medication from 29.6% to 18.5% (difference adjusted for underlying medical condition 12.5%; p = 0.039). Among the five possible reasons for discontinuing discharge medication, the evidence summaries seemed to have the largest impact on budget-related reasons for discontinuation (2.6% in the intervention versus 10.7% in the control group (p = 0.052)). Most clinicians (72%) were enthusiastic about continuing receiving evidence summaries with discharge letters in routine care.

CONCLUSIONS

The one-sentence evidence summary is a simple, inexpensive, well-accepted intervention that may improve primary care practitioners' adherence to evidence-based consultant recommendations.

Effectiveness of an evidence-based chiropractic continuing education workshop on participant knowledge of evidence-based health care

BACKGROUND

Chiropractors must continue to learn, develop themselves professionally throughout their careers, and become self-directed and lifelong learners. Using an evidence-based approach increases the probability of optimal patient outcomes. But most chiropractors lack knowledge and interest in evidence-based approaches. The purpose of this study was to develop and measure the effectiveness of evidence-based training for chiropractic practitioners in a continuing education setting.

METHODS

We developed and evaluated a continuing education workshop on evidence-based principles and methods for chiropractic practitioners. Forty-seven chiropractors participated in the training and testing. The course consisted of 12.5 hours of training in which practitioners learned to develop focused questions, search electronic data bases, critically review articles and apply information from the literature to specific clinical questions. Following the workshop, we assessed the program performance through the use of knowledge testing and anonymous presentation quality surveys.

RESULTS

Eighty-five percent of the participants completed all of the test, survey and data collection items. Pretest knowledge scores (15-item test) were low (47%). Post intervention scores (15-item test) improved with an effect size of 2.0. A 59-item knowledge posttest yielded very good results (mean score 88%). The quality of presentation was rated very good, and most participants (90%) would "definitely recommend" or "recommend" the workshop to a colleague.

CONCLUSION

The results of the study suggest that the continuing education course was effective in enhancing knowledge in the evidence-based approach and that the presentation was well accepted.

Audit and feedback: effects on professional practice and health care outcomes

BACKGROUND

Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines. Yet, audit and feedback has not consistently been found to be effective.

OBJECTIVES

To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes.

SEARCH STRATEGY

We searched the Cochrane Effective Practice and Organisation of Care Group's register and pending file up to January 2004.

SELECTION CRITERIA

Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported objectively measured professional practice in a healthcare setting or healthcare outcomes.

DATA COLLECTION AND ANALYSIS

Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses were undertaken. For each comparison we calculated the risk difference (RD) and risk ratio (RR), adjusted for baseline compliance when possible, for dichotomous outcomes and the percentage and the percent change relative to the control group average after the intervention, adjusted for baseline performance when possible, for continuous outcomes. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: the type of intervention (audit and feedback alone, audit and feedback with educational meetings, or multifaceted interventions that included audit and feedback), the intensity of the audit and feedback, the complexity of the targeted behaviour, the seriousness of the outcome, baseline compliance and study quality.

MAIN RESULTS

Thirty new studies were added to this update, and a total of 118 studies are included. In the primary analysis 88 comparisons from 72 studies were included that compared any intervention in which audit and feedback is a component compared to no intervention. For dichotomous outcomes the adjusted risk difference of compliance with desired practice varied from - 0.16 (a 16 % absolute decrease in compliance) to 0.70 (a 70% increase in compliance) (median = 0.05, inter-quartile range = 0.03 to 0.11) and the adjusted risk ratio varied from 0.71 to 18.3 (median = 1.08, inter-quartile range = 0.99 to 1.30). For continuous outcomes the adjusted percent change relative to control varied from -0.10 (a 10 % absolute decrease in compliance) to 0.68 (a 68% increase in compliance) (median = 0.16, inter-quartile range = 0.05 to 0.37). Low baseline compliance with recommended practice and higher intensity of audit and feedback were associated with larger adjusted risk ratios (greater effectiveness) across studies.

AUTHORS' CONCLUSIONS

Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate. The relative effectiveness of audit and feedback is likely to be greater when baseline adherence to recommended practice is low and when feedback is delivered more intensively.

Audit and feedback: effects on professional practice and health care outcomes

BACKGROUND

Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines. Yet, audit and feedback has not been found to be consistently effective.

OBJECTIVES

To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes.

SEARCH STRATEGY

We searched the Cochrane Effective Practice and Organisation of Care Group's register up to January 2001. This was supplemented with searches of MEDLINE and reference lists, which did not yield additional relevant studies.

SELECTION CRITERIA

Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported objectively measured professional practice in a healthcare setting or healthcare outcomes.

DATA COLLECTION AND ANALYSIS

Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses were undertaken.

MAIN RESULTS

We included 85 studies, 48 of which have been added to the previous version of this review. There were 52 comparisons of dichotomous outcomes from 47 trials with over 3500 health professionals that compared audit and feedback to no intervention. The adjusted RDs of non-compliance with desired practice varied from 0.09 (a 9% absolute increase in non-compliance) to 0.71 (a 71% decrease in non-compliance) (median = 0.07, inter-quartile range = 0.02 to 0.11). The one factor that appeared to predict the effectiveness of audit and feedback across studies was baseline non-compliance with recommended practice.

REVIEWER'S CONCLUSIONS

Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate. The absolute effects of audit and feedback are more likely to be larger when baseline adherence to recommended practice is low.

Quality improvement interventions: what works?

Quality improvement efforts in healthcare focus on the use of interventions and tools to change identified processes. What interventions and tools are effective? A survey of the literature yields information on interventions and tools that have been used to successfully create quality improvement. Effective methods for changing clinical practice include face-to-face education outreach visits, involvement of local opinion leaders, reminder systems, repeated feedback from the senior medical staff, patient-mediated interventions, and a combination of interventions deployed simultaneously. Participation in an organized continuous quality improvement process is beneficial in conjunction with additional interventions.

From artisan to architect: the specialist and systems of provision of diabetes care in 2001

OBJECTIVE

To discuss the future role of the diabetes specialist and the endocrinologist in the care of patients with diabetes.

METHODS

We reviewed the literature on health care systems, integrated approaches to provision of health care, health care in the primary-care setting for patients with chronic illness, and population-based health care for patients with diabetes.

RESULTS

Specialists who care for patients with diabetes will assume a more significant role as caretakers of teams of health care professionals who are responsible for the primary care of patients with diabetes. The integration of specialist care at the primary-care level may lead to improved outcomes at the primary-care level, increased access to specialty care, and decreased costs for the health care system. Alternative systems, such as regulatory barriers to specialist care, are less likely to accomplish these outcomes.

CONCLUSION

In the near future, specialists will be expected to care for those who provide primary care for patients with diabetes. Endocrinologists and endocrinology training programs should recognize this emerging role and prepare for it.

Developing evidence-based clinicians

Evidence-based care has heralded an advancement for health with extensive rhetoric and incentives encouraging clinicians to become active participants in practice change. This paper explores two mechanisms-research utilization and systematic reviews--as processes to develop empowerment in clinicians, and develop an ethos of basing practice on research. The intent of these approaches is to create an environment that enables clinicians to seek solutions to practice problems, inform practice decisions and develop practice policy using evidence from research. The goal is to develop evidence-based clinicians who remain active and interested in their practice and who are committed to promoting the application of research to practice. The strengths and weaknesses of these methods are explored within two case studies where they are applied to the management of hypothermia and the efficacy of postoperative observations. Key points of application for each of the methods are outlined in the conclusions.

Integrated risk assessment and feedback reporting for clinical decision making in a Medicare Risk plan

The challenge of tapping into the rich resource of population-based, aggregated data to inform and guide clinical processes remains one of the largely unrealized potentials of managed care. This article describes a multifaceted approach of using health-related data to support providers in clinical decision making as an adjunct to case management and primary care delivery. The goal is to provide data that can be used for clinical decision making that is population based, yet individualized for specific patient care situations. Information reporting holds great potential in the clinical care of patients because it can be used to identify persons who could benefit from early detection, intervention, or treatment. It has been suggested that one of the keys to success in managed Medicare is the timely use of information that is detailed, comprehensive, and real-time describing key parameters of clinical encounters.

Evidence-based quality measurement

There is a strong need to provide scientific evidence supporting the effectiveness of specific health care interventions. Guidelines alone are unlikely to persuade physicians to modify their practice styles, unless research evidence having a motivational impact is also provided. Process-outcome parameters provide a solution for linking the process and outcome of health care and incorporate potentially motivating research evidence.