Improving Rescreening In Community Clinics: Does A System Approach Work?

Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes

BACKGROUND

Health professionals sometimes do not use the best evidence to treat their patients, in part due to unconscious acts of omission and information overload. Reminders help clinicians overcome these problems by prompting them to recall information that they already know, or by presenting information in a different and more accessible format. Manually-generated reminders delivered on paper are defined as information given to the health professional with each patient or encounter, provided on paper, in which no computer is involved in the production or delivery of the reminder. Manually-generated reminders delivered on paper are relatively cheap interventions, and are especially relevant in settings where electronic clinical records are not widely available and affordable. This review is one of three Cochrane Reviews focused on the effectiveness of reminders in health care.

OBJECTIVES

1. To determine the effectiveness of manually-generated reminders delivered on paper in changing professional practice and improving patient outcomes. 2. To explore whether a number of potential effect modifiers influence the effectiveness of manually-generated reminders delivered on paper.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, CINAHL and two trials registers on 5 December 2018. We searched grey literature, screened individual journals, conference proceedings and relevant systematic reviews, and reviewed reference lists and cited references of included studies.

SELECTION CRITERIA

We included randomised and non-randomised trials assessing the impact of manually-generated reminders delivered on paper as a single intervention (compared with usual care) or added to one or more co-interventions as a multicomponent intervention (compared with the co-intervention(s) without the reminder component) on professional practice or patients' outcomes. We also included randomised and non-randomised trials comparing manually-generated reminders with other quality improvement (QI) interventions.

DATA COLLECTION AND ANALYSIS

Two review authors screened studies for eligibility and abstracted data independently. We extracted the primary outcome as defined by the authors or calculated the median effect size across all reported outcomes in each study. We then calculated the median percentage improvement and interquartile range across the included studies that reported improvement related outcomes, and assessed the certainty of the evidence using the GRADE approach.

MAIN RESULTS

We identified 63 studies (41 cluster-randomised trials, 18 individual randomised trials, and four non-randomised trials) that met all inclusion criteria. Fifty-seven studies reported usable data (64 comparisons). The studies were mainly located in North America (42 studies) and the UK (eight studies). Fifty-four studies took place in outpatient/ambulatory settings. The clinical areas most commonly targeted were cardiovascular disease management (11 studies), cancer screening (10 studies) and preventive care (10 studies), and most studies had physicians as their target population (57 studies). General management of a clinical condition (17 studies), test-ordering (14 studies) and prescription (10 studies) were the behaviours more commonly targeted by the intervention. Forty-eight studies reported changes in professional practice measured as dichotomous process adherence outcomes (e.g. compliance with guidelines recommendations), 16 reported those changes measured as continuous process-of-care outcomes (e.g. number of days with catheters), eight reported dichotomous patient outcomes (e.g. mortality rates) and five reported continuous patient outcomes (e.g. mean systolic blood pressure). Manually-generated reminders delivered on paper probably improve professional practice measured as dichotomous process adherence outcomes) compared with usual care (median improvement 8.45% (IQR 2.54% to 20.58%); 39 comparisons, 40,346 participants; moderate certainty of evidence) and may make little or no difference to continuous process-of-care outcomes (8 comparisons, 3263 participants; low certainty of evidence). Adding manually-generated paper reminders to one or more QI co-interventions may slightly improve professional practice measured as dichotomous process adherence outcomes (median improvement 4.24% (IQR -1.09% to 5.50%); 12 comparisons, 25,359 participants; low certainty of evidence) and probably slightly improve professional practice measured as continuous outcomes (median improvement 0.28 (IQR 0.04 to 0.51); 2 comparisons, 12,372 participants; moderate certainty of evidence). Compared with other QI interventions, manually-generated reminders may slightly decrease professional practice measured as process adherence outcomes (median decrease 7.9% (IQR -0.7% to 11%); 14 comparisons, 21,274 participants; low certainty of evidence). We are uncertain whether manually-generated reminders delivered on paper, compared with usual care or with other QI intervention, lead to better or worse patient outcomes (dichotomous or continuous), as the certainty of the evidence is very low (10 studies, 13 comparisons). Reminders added to other QI interventions may make little or no difference to patient outcomes (dichotomous or continuous) compared with the QI alone (2 studies, 2 comparisons). Regarding resource use, studies reported additional costs per additional point of effectiveness gained, but because of the different currencies and years used the relevance of those figures is uncertain. None of the included studies reported outcomes related to harms or adverse effects.

AUTHORS' CONCLUSIONS

Manually-generated reminders delivered on paper as a single intervention probably lead to small to moderate increases in outcomes related to adherence to clinical recommendations, and they could be used as a single QI intervention. It is uncertain whether reminders should be added to other QI intervention already in place in the health system, although the effects may be positive. If other QI interventions, such as patient or computerised reminders, are available, they should be preferred over manually-generated reminders, but under close evaluation in order to decrease uncertainty about their potential effect.

Tailored interventions to address determinants of practice

BACKGROUND

Tailored intervention strategies are frequently recommended among approaches to the implementation of improvement in health professional performance. Attempts to change the behaviour of health professionals may be impeded by a variety of different barriers, obstacles, or factors (which we collectively refer to as determinants of practice). Change may be more likely if implementation strategies are specifically chosen to address these determinants.

OBJECTIVES

To determine whether tailored intervention strategies are effective in improving professional practice and healthcare outcomes. We compared interventions tailored to address the identified determinants of practice with either no intervention or interventions not tailored to the determinants.

SEARCH METHODS

We conducted searches of The Cochrane Library, MEDLINE, EMBASE, PubMed, CINAHL, and the British Nursing Index to May 2014. We conducted a final search in December 2014 (in MEDLINE only) for more recently published trials. We conducted searches of the metaRegister of Controlled Trials (mRCT) in March 2013. We also handsearched two journals.

SELECTION CRITERIA

Cluster-randomised controlled trials (RCTs) of interventions tailored to address prospectively identified determinants of practice, which reported objectively measured professional practice or healthcare outcomes, and where at least one group received an intervention designed to address prospectively identified determinants of practice.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed quality and extracted data. We undertook qualitative and quantitative analyses, the quantitative analysis including two elements: we carried out 1) meta-regression analyses to compare interventions tailored to address identified determinants with either no interventions or an intervention(s) not tailored to the determinants, and 2) heterogeneity analyses to investigate sources of differences in the effectiveness of interventions. These included the effects of: risk of bias, use of a theory when developing the intervention, whether adjustment was made for local factors, and number of domains addressed with the determinants identified.

MAIN RESULTS

We added nine studies to this review to bring the total number of included studies to 32 comparing an intervention tailored to address identified determinants of practice to no intervention or an intervention(s) not tailored to the determinants. The outcome was implementation of recommended practice, e.g. clinical practice guideline recommendations. Fifteen studies provided enough data to be included in the quantitative analysis. The pooled odds ratio was 1.56 (95% confidence interval (CI) 1.27 to 1.93, P value < 0.001). The 17 studies not included in the meta-analysis had findings showing variable effectiveness consistent with the findings of the meta-regression.

AUTHORS' CONCLUSIONS

Despite the increase in the number of new studies identified, our overall finding is similar to that of the previous review. Tailored implementation can be effective, but the effect is variable and tends to be small to moderate. The number of studies remains small and more research is needed, including trials comparing tailored interventions to no or other interventions, but also studies to develop and investigate the components of tailoring (identification of the most important determinants, selecting interventions to address the determinants). Currently available studies have used different methods to identify determinants of practice and different approaches to selecting interventions to address the determinants. It is not yet clear how best to tailor interventions and therefore not clear what the effect of an optimally tailored intervention would be.

Hierarchy of evidence and appraisal of limitations (HEAL) grading system

Despite more than 30 years of effort that has been dedicated to the improvement of grading systems for evaluating the quality of research study designs considerable shortcomings continue. These shortcomings include the failure to define key terms, provide a comprehensive list of design flaws, demonstrate the reliability of such grading systems, properly value non-randomized controlled trials, and develop theoretically-derived systems for penalizing and promoting the evidence generated by a study. Consequently, in light of the importance of grading guidelines in evidence-based medicine, steps must be taken to remedy these deficiencies. This article presents two methods--a grading system and a measure of methodological bias--for evaluating the quality of evidence produced by an efficacy study.

Effectiveness of interventions to increase screening for breast, cervical, and colorectal cancers: nine updated systematic reviews for the guide to community preventive services

CONTEXT

Screening reduces mortality from breast, cervical, and colorectal cancers. The Guide to Community Preventive Services previously conducted systematic reviews on the effectiveness of 11 interventions to increase screening for these cancers. This article presents results of updated systematic reviews for nine of these interventions.

EVIDENCE ACQUISITION

Five databases were searched for studies published during January 2004-October 2008. Studies had to (1) be a primary investigation of one or more intervention category; (2) be conducted in a country with a high-income economy; (3) provide information on at least one cancer screening outcome of interest; and (4) include screening use prior to intervention implementation or a concurrent group unexposed to the intervention category of interest. Forty-five studies were included in the reviews.

EVIDENCE SYNTHESIS

Recommendations were added for one-on-one education to increase screening with fecal occult blood testing (FOBT) and group education to increase mammography screening. Strength of evidence for client reminder interventions to increase FOBT screening was upgraded from sufficient to strong. Previous findings and recommendations for reducing out-of-pocket costs (breast cancer screening); provider assessment and feedback (breast, cervical, and FOBT screening); one-on-one education and client reminders (breast and cervical cancer screening); and reducing structural barriers (breast cancer and FOBT screening) were reaffirmed or unchanged. Evidence remains insufficient to determine effectiveness for the remaining screening tests and intervention categories.

CONCLUSIONS

Findings indicate new and reaffirmed interventions effective in promoting recommended cancer screening, including colorectal cancer screening. Findings can be used in community and healthcare settings to promote recommended care. Important research gaps also are described.

Multilevel interventions: study design and analysis issues

Multilevel interventions, implemented at the individual, physician, clinic, health-care organization, and/or community level, increasingly are proposed and used in the belief that they will lead to more substantial and sustained changes in behaviors related to cancer prevention, detection, and treatment than would single-level interventions. It is important to understand how intervention components are related to patient outcomes and identify barriers to implementation. Designs that permit such assessments are uncommon, however. Thus, an important way of expanding our knowledge about multilevel interventions would be to assess the impact of interventions at different levels on patients as well as the independent and synergistic effects of influences from different levels. It also would be useful to assess the impact of interventions on outcomes at different levels. Multilevel interventions are much more expensive and complicated to implement and evaluate than are single-level interventions. Given how little evidence there is about the value of multilevel interventions, however, it is incumbent upon those arguing for this approach to do multilevel research that explicates the contributions that interventions at different levels make to the desired outcomes. Only then will we know whether multilevel interventions are better than more focused interventions and gain greater insights into the kinds of interventions that can be implemented effectively and efficiently to improve health and health care for individuals with cancer. This chapter reviews designs for assessing multilevel interventions and analytic ways of controlling for potentially confounding variables that can account for the complex structure of multilevel data.

An Application of a New Evidence Grading System to Research on the Chronic Care Model

Methodological quality undergirds all evidence-based medicine because without strong evidence supporting or refuting the efficacy of an intervention, the movement toward basing medical decisions and practice on scientific evidence is not sustainable. Recently, the consensus that had existed regarding the hierarchy of evidence produced by a study design was challenged on the basis that existing guidelines failed to properly define key terms, weight the merits of certain non-randomized controlled trials, and employ a comprehensive list of study design limitations to render evaluative conclusions, to name a few of the challenges. The present study introduces a new grading system that overcomes, or at the very least greatly diminishes, these challenges. This new method is applied to the literature on the Chronic Care Model and the results are then compared to several of the most popular grading guidelines currently in use. These results revealed substantial differences between the guidelines in accordance with previous research that challenged existing methods. Furthermore, the present study lends support to the proposed grading guideline although further research into its validity and reliability is needed.

Cognitive and affective correlates of decisional balance regarding screening mammography in older women

Participation in screening mammography remains suboptimal. This research aimed to improve understanding of ways to facilitate screening mammography attendance. One hundred and forty-two women from Gold Coast, Australia, aged 50-75, participated in the study. Social cognitive variables were assessed as potential predictors of mammography attendance. Most participants (79%) were maintaining regular screening mammography. Greater knowledge of breast cancer was the strongest predictor of decisional balance in favor of attending screening. Women who had relapsed from screening had significantly lower breast cancer worry than those contemplating attending for the first time. The results were consistent with previous research and point to factors screening services could consider to increase uptake.

Systems of support to increase colorectal cancer screening and follow-up rates (SOS): design, challenges, and baseline characteristics of trial participants

BACKGROUND

Screening decreases colorectal cancer (CRC) morbidity and mortality, yet remains underutilized. Screening breakdowns arise from lack of uptake and failure to follow-up after a positive screening test.

OBJECTIVES

Systems of support to increase colorectal cancer screening and follow-up (SOS) is a randomized trial designed to increase: (1) CRC screening and (2) follow-up of positive screening tests. The Chronic Care Model and the Preventive Health Model inform study design.

METHODS

The setting is a large nonprofit healthcare organization. In part-1 study, patients age 50-75 due for CRC screening are randomized to one of 4 study conditions. Arm 1 receives usual care. Arm 2 receives automated support (mailed information about screening choices and fecal occult blood tests (FOBT)). Arm 3 receives automated and assisted support (a medical assistant telephone call). Arm 4 receives automated, assisted, and care management support (a registered nurse provides behavioral activation and coordination of care). In part-2, study patients with a positive FOBT or adenomas on flexible sigmoidoscopy are randomized to receive either usual care or nurse care management. Primary outcomes are: 1) the proportion with CRC screening, 2) the proportion with a complete diagnostic evaluation after a positive screening test.

RESULTS

We sent recruitment letters to 15,414 patients and 4675 were randomized. Randomly assigned treatment groups were similar in age, sex, race, education, self-reported health, and CRC screening history.

CONCLUSIONS

We will determine the effectiveness and cost effectiveness of stepped increases in systems of support to increase CRC screening and follow-up after a positive screening test over 2years.

Interventions to promote repeat breast cancer screening with mammography: a systematic review and meta-analysis

BACKGROUND

Various interventions to promote repeat use of mammography have been evaluated, but the efficacy of such interventions is not well understood.

METHODS

We searched electronic databases through August 15, 2009, and extracted data to calculate unadjusted effect estimates (odds ratios [ORs] and 95% confidence intervals [CIs]). Eligible studies were those that reported estimates of repeat screening for intervention and control groups. We tested homogeneity and computed summary odds ratios. To explore possible causes of heterogeneity, we performed stratified analyses, examined meta-regression models for 15 a priori explanatory variables, and conducted influence analyses. We used funnel plots and asymmetry tests to assess publication bias. Statistical tests were two-sided.

RESULTS

The 25 eligible studies (27 effect estimates) were statistically significantly heterogeneous (Q = 69.5, I(2) = 63%, P < .001). Although there were homogeneous subgroups in some categories of the 15 explanatory variables, heterogeneity persisted after stratification. For all but one explanatory variable, subgroup summary odds ratios were similar with overlapping confidence intervals. The summary odds ratio for the eight heterogeneous reminder-only studies was the largest observed (OR = 1.79, 95% CI = 1.41 to 2.29) and was statistically significantly greater than the summary odds ratio (P(diff) = .008) for the homogeneous group of 17 studies that used the more intensive strategies of education/motivation or counseling (OR = 1.27, 95% CI = 1.17 to 1.37). However, reminder-only studies remained statistically significantly heterogeneous, whereas the studies classified as education/motivation or counseling were homogeneous. Similarly, in meta-regression modeling, the only statistically significant predictor of the intervention effect size was intervention strategy (reminder-only vs the other two combined as the referent). Publication bias was not apparent.

CONCLUSIONS

The observed heterogeneity precludes a summary effect estimate. We also cannot conclude that reminder-only intervention strategies are more effective than alternate strategies. Additional studies are needed to identify methods or strategies that could increase repeat mammography.

Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes

BACKGROUND

In the previous version of this review, the effectiveness of interventions tailored to barriers to change was found to be uncertain.

OBJECTIVES

To assess the effectiveness of interventions tailored to address identified barriers to change on professional practice or patient outcomes.

SEARCH STRATEGY

For this update, in addition to the EPOC Register and pending files, we searched the following databases without language restrictions, from inception until August 2007: MEDLINE, EMBASE, CINAHL, BNI and HMIC. We searched the National Research Register to November 2007. We undertook further searches to October 2009 to identify potentially eligible published or ongoing trials.

SELECTION CRITERIA

Randomised controlled trials (RCTs) of interventions tailored to address prospectively identified barriers to change that reported objectively measured professional practice or healthcare outcomes in which at least one group received an intervention designed to address prospectively identified barriers to change.

DATA COLLECTION AND ANALYSIS

Two reviewers independently assessed quality and extracted data. We undertook quantitative and qualitative analyses. The quantitative analyses had two elements.1. We carried out a meta-regression to compare interventions tailored to address identified barriers to change with either no interventions or an intervention(s) not tailored to the barriers.2. We carried out heterogeneity analyses to investigate sources of differences in the effectiveness of interventions. These included the effects of: risk of bias, concealment of allocation, rigour of barrier analysis, use of theory, complexity of interventions, and the reported presence of administrative constraints.

MAIN RESULTS

We included 26 studies comparing an intervention tailored to address identified barriers to change to no intervention or an intervention(s) not tailored to the barriers. The effect sizes of these studies varied both across and within studies.Twelve studies provided enough data to be included in the quantitative analysis. A meta-regression model was fitted adjusting for baseline odds by fitting it as a covariate, to obtain the pooled odds ratio of 1.54 (95% CI, 1.16 to 2.01) from Bayesian analysis and 1.52 (95% CI, 1.27 to 1.82, P < 0.001) from classical analysis. The heterogeneity analyses found that no study attributes investigated were significantly associated with effectiveness of the interventions.

AUTHORS' CONCLUSIONS

Interventions tailored to prospectively identified barriers are more likely to improve professional practice than no intervention or dissemination of guidelines. However, the methods used to identify barriers and tailor interventions to address them need further development. Research is required to determine the effectiveness of tailored interventions in comparison with other interventions.