AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program

Volunteer bias in recruitment, retention, and blood sample donation in a randomised controlled trial involving mothers and their children at six months and two years: a longitudinal analysis

BACKGROUND

The vulnerability of clinical trials to volunteer bias is under-reported. Volunteer bias is systematic error due to differences between those who choose to participate in studies and those who do not.

METHODS AND RESULTS

This paper extends the applications of the concept of volunteer bias by using data from a trial of probiotic supplementation for childhood atopy in healthy dyads to explore 1) differences between a) trial participants and aggregated data from publicly available databases b) participants and non-participants as the trial progressed 2) impact on trial findings of weighting data according to deprivation (Townsend) fifths in the sample and target populations. 1) a) Recruits (n = 454) were less deprived than the target population, matched for area of residence and delivery dates (n = 6,893) (mean [SD] deprivation scores 0.09[4.21] and 0.79[4.08], t = 3.44, df = 511, p<0.001). b) i) As the trial progressed, representation of the most deprived decreased. These participants and smokers were less likely to be retained at 6 months (n = 430[95%]) (OR 0.29,0.13-0.67 and 0.20,0.09-0.46), and 2 years (n = 380[84%]) (aOR 0.68,0.50-0.93 and 0.55,0.28-1.09), and consent to infant blood sample donation (n = 220[48%]) (aOR 0.72,0.57-0.92 and 0.43,0.22-0.83). ii) Mothers interested in probiotics or research or reporting infants' adverse events or rashes were more likely to attend research clinics and consent to skin-prick testing. Mothers participating to help children were more likely to consent to infant blood sample donation. 2) In one trial outcome, atopic eczema, the intervention had a positive effect only in the over-represented, least deprived group. Here, data weighting attenuated risk reduction from 6.9%(0.9-13.1%) to 4.6%(-1.4-+10.5%), and OR from 0.40(0.18-0.91) to 0.56(0.26-1.21). Other findings were unchanged.

CONCLUSIONS

Potential for volunteer bias intensified during the trial, due to non-participation of the most deprived and smokers. However, these were not the only predictors of non-participation. Data weighting quantified volunteer bias and modified one important trial outcome.

TRIAL REGISTRATION

This randomised, double blind, parallel group, placebo controlled trial is registered with the International Standard Randomised Controlled Trials Register, Number (ISRCTN) 26287422. Registered title: Probiotics in the prevention of atopy in infants and children.

Use of case reports and the Adverse Event Reporting System in systematic reviews: overcoming barriers to assess the link between Crohn's disease medications and hepatosplenic T-cell lymphoma

BACKGROUND

To identify demographic and clinical characteristics associated with cases of hepatosplenic T-cell lymphoma (HSTCL) in patients with Crohn's disease, and to assess strength of evidence for a causal relationship between medications and HSTCL in Crohn's disease.

METHODS

We identified cases of HSTCL in Crohn's disease in studies included in a comparative effectiveness review of Crohn's disease medications, through a separate search of PubMed and Embase for published case reports, and from the Food and Drug Administration (FDA) Adverse Event Reporting System (AERS). We used three causality assessment tools to evaluate the relationship between medication exposure and HSTCL.

RESULTS

We found 37 unique cases of HSTCL in patients with Crohn's disease. Six cases were unique to the published literature and nine were unique to AERS. Cases were typically young (<40 years of age) and male (86%). The most commonly reported medications were anti-metabolites (97%) and anti-tumor necrosis factor alpha (anti-TNFa) medications (76%). Dose and duration of therapy were not consistently reported. Use of aminosalicylates and corticosteroids were rarely reported, despite the high prevalence of these medications in routine treatment. Using the causality assessment tools, it could only be determined that anti-metabolite and anti-TNFa therapies were possible causes of HSTCL in Crohn's disease based on the data contained in the case reports.

CONCLUSION

Systematic reviews that incorporate case reports of rare lethal events should search both published literature and AERS, but consideration should be given to the limitations of case reports. In this study, establishing a causative effect other than 'possible' between anti-metabolite or anti-TNFa therapies and HSTCL was not feasible because case reports lacked data required by the causality assessments, and because of the limited applicability of causality assessment tools for rare irreversible events. We recommend minimum reporting requirements for case reports to improve causality assessment and routine reporting of rare life-threatening events, including their absence, in clinical trials to help clinicians determine whether rare adverse events are causally related to a medication.

Risk of heart failure and edema associated with the use of pregabalin: a systematic review

BACKGROUND

Pregabalin is used in the treatment of postherpetic neuralgia, diabetic neuropathic pain, partial seizures, anxiety disorders and fibromyalgia. Recognized adverse effects associated with its use include cognitive impairment, somnolence and dizziness. Heart failure associated with pregabalin has been described, however the strength of this association has not been well characterized. To examine this further, we will conduct a systematic review of the risk of heart failure and edema associated with use of pregabalin.

METHODS/DESIGN

We will include all studies (experimental, quasi-experimental, observational, case series/reports, drug regulatory reports) that examine the use of pregabalin compared to placebo, gabapentin or conventional care. Our primary outcome is heart failure and the secondary outcomes include edema and weight gain. We will search electronic databases (MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials), and grey literature sources (trial registries, conference abstracts) to identify relevant studies. To ensure literature saturation, we will contact drug manufacturers, conduct forward citation searching, and scan the reference lists of key articles and included studies. We will not restrict inclusion by language or publication status.Two reviewers will screen citations (titles and abstracts) and full-text articles, conduct data abstraction, and appraise risk of bias. Random-effects meta-analysis will be conducted if the studies are deemed heterogeneous in terms of clinical, statistical and methodological factors but still suitable for meta-analysis.

CONCLUSIONS

The results of this review will assist physicians to better appreciate pregabalin's risk for edema or congestive heart failure and will be pertinent to the thousands of patients worldwide who are administered this medication.Our protocol was registered in the PROSPERO database (CRD42012002948).

Pharmacological treatment of neonatal seizures: a systematic review

Pharmacologic treatment options for neonatal seizures have expanded over the past 2 decades, and there is no consensus on optimal treatment strategy. We systematically reviewed the published literature to determine which medication(s) are most effective for treating neonatal seizures, by retrieving trials and observational investigations via PubMed (through August 2011) that focused on pharmacological seizure treatment of neonates (≤ 28 days old) and utilized continuous or amplitude-integrated EEG to confirm seizure diagnosis and cessation. Our search identified 557 initial articles and 14 additional studies after reference reviews, with 16 meeting inclusion criteria. Of these, 2 were randomized trials and only 3 additional investigations included comparison groups. We found limited evidence regarding the best pharmacologic treatment for neonatal seizures, but were able to devise a treatment algorithm from available data. These findings have the potential to serve as a clinical reference and to inform the design of comparative effectiveness investigations for neonatal antiepileptics.

Managed alcohol as a harm reduction intervention for alcohol addiction in populations at high risk for substance abuse

BACKGROUND

Managed alcohol programmes (MAP) are a harm reduction strategy used to minimise the personal harm and adverse societal effects that alcohol dependence can lead to by providing an alternative to zero-tolerance approaches that incorporate drinking goals (abstinence or moderation) that are compatible with the needs of the individual, and promoting access to services by offering low-threshold alternatives. This enables clients to gain access to services despite continued alcohol consumption and works to help the patient understand the risks involved in their behaviour and make decisions about their own treatment goals.

OBJECTIVES

To assess the effectiveness of MAP treatment regimens (serving limited quantities of alcohol daily to alcoholics) on their own or as compared to moderate drinking (self-controlled drinking), screening and brief intervention using a harm reduction approach, traditional abstinence-based interventions (12 step programmes) and no intervention.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL and PsycINFO up to March 2012. This search was expanded by handsearching of high-yield journals and conference proceedings that had not already been handsearched on behalf of The Cochrane Collaboration, searching reference lists of all papers and relevant reviews identified, references to ongoing and recently completed clinical trials in the National Research Register and IFPMA Clinical Trials Database (which contains ClinicalTrials.gov, Centerwatch, Current Controlled Trials and ClinicalStudyResults.gov, and Osservatorio Nazionale sulla Sperimentazione Clinica dei Medicinali). Trials registers, grey literature and reference lists were also searched. Individuals, organisations and experts in the field were contacted.

SELECTION CRITERIA

Randomised control trials (RCT), controlled clinical trials (CCT), interrupted time series (ITS) studies, and control before and after (CBA) studies involving vulnerable people aged 18 years or older who were at high risk for alcohol abuse attending MAP, defined as a structured programme that provided clients with controlled amounts of alcohol on a daily schedule, comparing no treatment, moderate drinking, brief intervention or 12-step variants.

DATA COLLECTION AND ANALYSIS

All study citations were collated into a single database. Two review author independently screened titles and abstracts and selected references potentially relevant to the review. Differences between selection lists were resolved by discussion. Two review authors independently evaluated whether studies should be included or excluded according to the eligibility criteria. In the event of a disagreement, a third author was consulted.

MAIN RESULTS

No studies were included in the review. This systematic review was intended to assess the effectiveness of a brief MAP on the reduction of incidence of harmful behaviour; however, no evidence was available to make this comparison; 22 articles were considered possibly relevant and all were excluded. Most articles were excluded because they failed to compare or consider managed alcohol as the experimental or control intervention, as well as one study (Baker 2010), which was also excluded because study participants were under 18 years of age. No study reviewed offered an intervention that was compared with managed alcohol or considered it as the intervention of interest, providing insufficient evidence to address the objectives of the review. Four studies (Aalto 2001; Baker 2010; Bertholet 2005; Tracy 2007) considered alcohol reduction as an outcome of interest, while four engaged interventions in a shelter setting or targeted vulnerable people (Baker 2010; Bradford 2005; Lapham 1993; McGlynn 1993); only one study (Kidd 2011) offered a qualitative assessment of a participant being admitted to MAP, but offered no analysis of the programme itself. These results accurately reflect the use of MAPs in current practice as existing programmes are ongoing only in a small number of sample pilot projects that target individuals with severe alcohol dependence or who consume non-beverage alcohol.

AUTHORS' CONCLUSIONS

The lack of evidence does not allow for a conclusion regarding the efficacy of MAP on their own, or as compared to brief intervention, moderate drinking, no intervention or 12-step variants. It is the review authors' opinion that it is likely to be the objective of MAPs that reduce their reportability and use in current practice, rather than a failure to provide an intervention that reduces the effects of alcohol dependence. Aiming to reduce harmful or antisocial behaviour in vulnerable individuals through the regulation of daily alcohol intake, rather than reducing harmful alcohol intake over time, provides considerable difficulty in developing measures of success from self-reported data (low treatment thresholds), monitoring long-term efficacy or establishing causal links between programme admission and a reduction in targeted behaviours, owing to the fact that prolonged participation in the programme is likely to indicate a willingness in the individual to change their behaviour patterns. More effort is needed to develop reporting measures, as well as methodologies, which address these specific challenges.

Methods for the drug effectiveness review project

The Drug Effectiveness Review Project was initiated in 2003 in response to dramatic increases in the cost of pharmaceuticals, which lessened the purchasing power of state Medicaid budgets. A collaborative group of state Medicaid agencies and other organizations formed to commission high-quality comparative effectiveness reviews to inform evidence-based decisions about drugs that would be available to Medicaid recipients. The Project is coordinated by the Center for Evidence-based Policy (CEbP) at Oregon Health & Science University (OHSU), and the systematic reviews are undertaken by the Evidence-based Practice Centers (EPCs) at OHSU and at the University of North Carolina. The reviews adhere to high standards for comparative effectiveness reviews. Because the investigators have direct, regular communication with policy-makers, the reports have direct impact on policy and decision-making, unlike many systematic reviews. The Project was an innovator of methods to involve stakeholders and continues to develop its methods in conducting reviews that are highly relevant to policy-makers. The methods used for selecting topics, developing key questions, searching, determining eligibility of studies, assessing study quality, conducting qualitative and quantitative syntheses, rating the strength of evidence, and summarizing findings are described. In addition, our on-going interactions with the policy-makers that use the reports are described.

A safety review and meta-analyses of bevacizumab and ranibizumab: off-label versus goldstandard

BACKGROUND

We set out a systemic review to evaluate whether off-label bevacizumab is as safe as licensed ranibizumab, and whether bevacizumab can be justifiably offered to patients as a treatment for age-related macular degeneration with robust evidence of no differential risk.

METHODS AND FINDINGS

Medline, Embase and the Cochrane Library were searched with no limitations of language and year of publication. We included RCTs with a minimum follow-up of one year which investigated bevacizumab or ranibizumab in direct comparison or against any other control group (indirect comparison). Direct comparison (3 trials, 1333 patients): The one year data show a significantly higher rate of ocular adverse effects (AE) with bevacizumab compared to ranibizumab (RR = 2.8; 95% CI 1.2-6.5). The proportion of patients with serious infections and gastrointestinal disorders was also higher with bevacizumab than with ranibizumab (RR = 1.3; 95% CI 1.0-1.7). Arterial thromboembolic events were equally distributed among the groups. Indirect comparison: Ranibizumab versus any control (5 trials, 4054 patients): The two year results of three landmark trials showed that while absolute rates of serious ocular AE were low (≤ 2.1%), relative harm was significantly raised (RR = 3.1; 95% CI 1.1-8.9). A significant increase in nonocular haemorrhage was also observed with ranibizumab (RR = 1.7; 95% CI 1.1-2.7). Bevacizumab versus any control (3 trials, 244 patients): We were unable to judge the safety profile of bevacizumab due to the poor quality of AE monitoring and reporting in the trials.

CONCLUSIONS

Evidence from head-to-head trials raises concern about an increased risk of ocular and multiple systemic AE with bevacizumab. Therefore, clinicians and patients should continue to carefully weight up the benefits and harms when choosing between the two treatment options. We also emphasize the need for studies that are powered not just for efficacy, but for defined safety outcomes based on the signals detected in this systematic review.

Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining

PURPOSE

The aim of this study was to demonstrate that modern data mining tools can be used as one step in reducing the labor necessary to produce and maintain systematic reviews.

METHODS

We used four continuously updated, manually curated resources that summarize MEDLINE-indexed articles in entire fields using systematic review methods (PDGene, AlzGene, and SzGene for genetic determinants of Parkinson disease, Alzheimer disease, and schizophrenia, respectively; and the Tufts Cost-Effectiveness Analysis (CEA) Registry for cost-effectiveness analyses). In each data set, we trained a classification model on citations screened up until 2009. We then evaluated the ability of the model to classify citations published in 2010 as "relevant" or "irrelevant" using human screening as the gold standard.

RESULTS

Classification models did not miss any of the 104, 65, and 179 eligible citations in PDGene, AlzGene, and SzGene, respectively, and missed only 1 of 79 in the CEA Registry (100% sensitivity for the first three and 99% for the fourth). The respective specificities were 90, 93, 90, and 73%. Had the semiautomated system been used in 2010, a human would have needed to read only 605/5,616 citations to update the PDGene registry (11%) and 555/7,298 (8%), 717/5,381 (13%), and 334/1,015 (33%) for the other three databases.

CONCLUSION

Data mining methodologies can reduce the burden of updating systematic reviews, without missing more papers than humans.

Efficacy of cognitive enhancers for Alzheimer's disease: protocol for a systematic review and network meta-analysis

BACKGROUND

Approximately 35 million people world-wide have Alzheimer's disease and this is projected to nearly double by 2030. Cognitive enhancers, including cholinesterase inhibitors (for example, donepezil, galantamine and rivastigmine) and memantine (N-methyl-D-aspartic acid (NMDA) receptor antagonist) have been approved for the treatment of Alzheimer's disease in many countries. Our objective is to evaluate the comparative effectiveness, safety, and cost of cognitive enhancers for Alzheimer's disease through a systematic review.

METHODS/DESIGN

Studies examining the efficacy, safety, and cost of cognitive enhancers compared to placebo, supportive care, and other cognitive enhancers for Alzheimer's patients will be included. The primary outcome is cognition and secondary outcomes include function, behavior, quality of life, safety, and cost. Experimental studies (randomized controlled trials, quasi-randomized controlled trials, controlled clinical trials), quasi-experimental studies (controlled before-after, interrupted time series), and observational studies (cohort, case-control studies) will be eligible for inclusion. Inclusion will not be limited by publication status, time period or language of dissemination.We will search electronic databases (for example, MEDLINE, Cochrane Central Register of Controlled Trials, EMBASE, CINAHL, Ageline) from inception onwards. The electronic database search will be supplemented by searching for grey literature (for example, conference proceedings, searches in Google and relevant organization websites). Two reviewers will independently screen the studies for inclusion using the eligibility criteria established a priori and independently extract data. Risk of bias will be assessed using the Cochrane Risk of Bias tool for experimental and quasi-experimental studies and the Newcastle Ottawa Scale for observational studies. If deemed appropriate, meta-analysis and network (that is, indirect comparisons) meta-analysis will be conducted.

DISCUSSION

Our systematic review will inform the decision of healthcare providers, policy-makers, Alzheimer's patients and family members about the use of cognitive enhancers, by improving their understanding of the costs, benefits and harms that are associated with these agents. PROSPERO REGISTRY NUMBER: CRD42012001948.

The performance of adverse effects search filters in MEDLINE and EMBASE

BACKGROUND

 Search filters can potentially improve the efficiency of searches involving electronic databases such as medline and embase. Although search filters have been developed for identifying records that contain adverse effects data, little is known about the sensitivity of such filters.

OBJECTIVES

 This study measured the sensitivity of using available adverse effects filters to retrieve papers with adverse effects data.

METHODS

  A total of 233 included studies from 26 systematic reviews of adverse effects were used for analysis. Search filters from medline and embase were tested for their sensitivity in retrieving the records included in these reviews. In addition, the sensitivity of each individual search term used in at least one search filter was measured.

RESULTS

 Subheadings proved the most useful search terms in both medline and embase. No indexing terms in medline achieved over 12% sensitivity. The sensitivity of published search filters varied in medline from 3% to 93% and in embase from 57% to 97%. Whether this level of sensitivity is acceptable will be dependent on the purpose of the search.

CONCLUSIONS

 Although no adverse effects search filter captured all the relevant records, high sensitivity could be achieved. Search filters may therefore be useful in retrieving adverse effects data.

Effectiveness of terbutaline pump for the prevention of preterm birth. A systematic review and meta-analysis

BACKGROUND

Subcutaneous terbutaline (SQ terbutaline) infusion by pump is used in pregnant women as a prolonged (beyond 48-72 h) maintenance tocolytic following acute treatment of preterm contractions. The effectiveness and safety of this maintenance tocolysis have not been clearly established. We aimed to systematically evaluate the effectiveness and safety of subcutaneous (SQ) terbutaline infusion by pump for maintenance tocolysis.

METHODOLOGY/PRINCIPAL FINDINGS

MEDLINE, EMBASE, CINAHL, the Cochrane Library, the Centre for Reviews and Dissemination databases, post-marketing surveillance data and grey literature were searched up to April 2011 for relevant experimental and observational studies. Two randomized trials, one nonrandomized trial, and 11 observational studies met inclusion criteria. Non-comparative studies were considered only for pump-related harms. We excluded case-reports but sought FDA summaries of post-marketing surveillance data. Non-English records without an English abstract were excluded. Evidence of low strength from observational studies with risk of bias favored SQ terbutaline pump for the outcomes of delivery at <32 and <37 weeks, mean days of pregnancy prolongation, and neonatal death. Observational studies of medium to high risk of bias also demonstrated benefit for other surrogate outcomes, such as birthweight and neonatal intensive care unit (NICU) admission. Several cases of maternal deaths and maternal cardiovascular events have been reported in patients receiving terbutaline tocolysis.

CONCLUSIONS/SIGNIFICANCE

Although evidence suggests that pump therapy may be beneficial as maintenance tocolysis, our confidence in its validity and reproducibility is low, suggesting that its use should be limited to the research setting. Concerns regarding safety of therapy persist.

Review of quality assessment tools for the evaluation of pharmacoepidemiological safety studies

OBJECTIVES

Pharmacoepidemiological studies are an important hypothesis-testing tool in the evaluation of postmarketing drug safety. Despite the potential to produce robust value-added data, interpretation of findings can be hindered due to well-recognised methodological limitations of these studies. Therefore, assessment of their quality is essential to evaluating their credibility. The objective of this review was to evaluate the suitability and relevance of available tools for the assessment of pharmacoepidemiological safety studies.

DESIGN

We created an a priori assessment framework consisting of reporting elements (REs) and quality assessment attributes (QAAs). A comprehensive literature search identified distinct assessment tools and the prespecified elements and attributes were evaluated.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome measure was the percentage representation of each domain, RE and QAA for the quality assessment tools.

RESULTS

A total of 61 tools were reviewed. Most tools were not designed to evaluate pharmacoepidemiological safety studies. More than 50% of the reviewed tools considered REs under the research aims, analytical approach, outcome definition and ascertainment, study population and exposure definition and ascertainment domains. REs under the discussion and interpretation, results and study team domains were considered in less than 40% of the tools. Except for the data source domain, quality attributes were considered in less than 50% of the tools.

CONCLUSIONS

Many tools failed to include critical assessment elements relevant to observational pharmacoepidemiological safety studies and did not distinguish between REs and QAAs. Further, there is a lack of considerations on the relative weights of different domains and elements. The development of a quality assessment tool would facilitate consistent, objective and evidence-based assessments of pharmacoepidemiological safety studies.

Development of the RTI item bank on risk of bias and precision of observational studies

OBJECTIVE

To create a practical and validated item bank for evaluating the risk of bias and precision of observational studies of interventions or exposures included in systematic evidence reviews.

STUDY DESIGN AND SETTING

The item bank, developed at RTI International, was created based on 1,492 questions included in earlier instruments, organized by the quality domains identified by Deeks et al. Items were eliminated and refined through face validity, cognitive, content validity, and interrater reliability testing.

RESULTS

The resulting item bank consisting of 29 questions for evaluating the risk of bias and precision of observational studies of interventions or exposures (1) captures all of the domains critical for evaluating this type of research, (2) is comprehensive and can be easily lifted "off the shelf" by different researchers, (3) can be adapted to different topic areas and study types (e.g., cohort, case-control, cross-sectional, and case series studies), and (4) provides sufficient instruction to apply the tool to varied topics.

CONCLUSION

One bank of items, with specific instructions for focusing abstractor evaluations, can be created to judge the risk of bias and precision of the variety of observational studies that may be used in systematic and comparative effectiveness reviews.

The association between systemic glucocorticoid therapy and the risk of infection in patients with rheumatoid arthritis: systematic review and meta-analyses

INTRODUCTION

Infection is a major cause of morbidity and mortality in patients with rheumatoid arthritis (RA). The objective of this study was to perform a systematic review and meta-analysis of the effect of glucocorticoid (GC) therapy on the risk of infection in patients with RA.

METHODS

A systematic review was conducted by using MEDLINE, EMBASE, CINAHL, and the Cochrane Central Register of Controlled Trials database to January 2010 to identify studies among populations of patients with RA that reported a comparison of infection incidence between patients treated with GC therapy and patients not exposed to GC therapy.

RESULTS

In total, 21 randomised controlled trials (RCTs) and 42 observational studies were included. In the RCTs, GC therapy was not associated with a risk of infection (relative risk (RR), 0.97 (95% CI, 0.69, 1.36)). Small numbers of events in the RCTs meant that a clinically important increased or decreased risk could not be ruled out. The observational studies generated a RR of 1.67 (1.49, 1.87), although significant heterogeneity was present. The increased risk (and heterogeneity) persisted when analyses were stratified by varying definitions of exposure, outcome, and adjustment for confounders. A positive dose-response effect was seen.

CONCLUSIONS

Whereas observational studies suggested an increased risk of infection with GC therapy, RCTs suggested no increased risk. Inconsistent reporting of safety outcomes in the RCTs, as well as marked heterogeneity, probable residual confounding, and publication bias in the observational studies, limits the opportunity for a definitive conclusion. Clinicians should remain vigilant for infection in patients with RA treated with GC therapy.

Risk management and comparative effectiveness research: implications of the new collaborative FDA-CMS initiative

The US Food and Drug Administration's (FDA's) expanded authority to require Risk Evaluation and Mitigation Strategies (REMS) and the increasing importance of comparative effectiveness research and reviews (CERs) under the Affordable Care Act raise a number of public policy issues for prescription drug safety. In addition, the formalization of an interface between the Centers for Medicare and Medicaid Services (CMS) and the FDA to expedite early coverage of drugs, devices and diagnostics raises a number of questions about the extension of CERs into post-marketing drug surveillance requirements. In this commentary, we address the potential applications of CERs and REMS in risk-benefit assessments.

Network meta-analysis-highly attractive but more methodological research is needed

Network meta-analysis, in the context of a systematic review, is a meta-analysis in which multiple treatments (that is, three or more) are being compared using both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator. To ensure validity of findings from network meta-analyses, the systematic review must be designed rigorously and conducted carefully. Aspects of designing and conducting a systematic review for network meta-analysis include defining the review question, specifying eligibility criteria, searching for and selecting studies, assessing risk of bias and quality of evidence, conducting a network meta-analysis, interpreting and reporting findings. This commentary summarizes the methodologic challenges and research opportunities for network meta-analysis relevant to each aspect of the systematic review process based on discussions at a network meta-analysis methodology meeting we hosted in May 2010 at the Johns Hopkins Bloomberg School of Public Health. Since this commentary reflects the discussion at that meeting, it is not intended to provide an overview of the field.

Finding evidence for comparing medical interventions: AHRQ and the Effective Health Care Program

OBJECTIVE

This article discusses search methodology to identify evidence for comparative effectiveness reviews (CERs) as practiced by the Effective Health Care program.

STUDY DESIGN AND SETTING

Review.

RESULTS

Search methods described attempt to overcome the bias inherent in the publication and distribution of clinical evidence. Bibliographic databases and search strategies are discussed with special emphasis on searching for observational studies and harms data. Other techniques described include the use of key articles, citation tracking, hand searching, and personal communications. Strategies for locating gray literature, such as clinical trial protocols and regulatory information, are described. Search reporting and other practical matters are also discussed.

CONCLUSION

Better reporting and further research on search strategies is needed to develop additional evidence-based recommendations.

A Comparative Effectiveness Analysis of Alternative Strategies to Assess Hypothalamic-Pituitary-Adrenal Axis Function After Microsurgical Resection of Pituitary Tumors

BACKGROUND:

Observational methods can be used in conjunction with the comparative effectiveness research (CER) paradigm to inform decisions between alternative patient management strategies in real-world clinical settings.

OBJECTIVE:

To present a brief review of current observational research regarding 3 strategies for predicting normal hypothalamic-pituitary-adrenal (HPA) axis function after surgical resection of pituitary tumors and to apply the CER model to compare these management alternatives.

METHODS:

We designed and conducted 2 prospective observational studies involving 183 patients undergoing microsurgical resection for pituitary tumors. These investigations yielded a comprehensive database comprising longitudinal data from multiple clinical domains. We investigated 3 potential strategies to predict normal postoperative HPA axis function in this cohort, including preoperative adrenocorticotrophic hormone stimulation testing and measurement of serum cortisol levels immediately after surgery or on the first postoperative day. We performed a focused comparative effectiveness review to help inform the decision between the 3 potential clinical management strategies.

RESULTS:

This investigation illustrates the use of observational research methods in conjunction with CER methodology as one means of informing clinical management decisions. Of the 3 strategies for assessing postoperative HPA axis function studied, preoperative and immediate postoperative adrenocorticotrophic hormone stimulation testing had the highest sensitivity, accuracy, and positive predictive value for normal HPA axis function postoperatively. The preoperative strategy was also the most cost-effective approach (12% reduction vs benchmark).

CONCLUSION:

The decision to use any of the 3 strategies outlined requires attention to a specific patient's clinical situation, but this decision may be aided by the results of this CER analysis.

Systematic review of the literature on comparative effectiveness of antiviral treatments for chronic hepatitis B infection

OBJECTIVES

To evaluate the comparative effectiveness of antiviral drugs in adults with chronic hepatitis B monoinfection for evidence-based decision-making.

METHODS

A systematic review of randomized controlled clinical trials (RCTs) published in English. Results after interferon and nucleos(t)ides analog therapies were synthesized with random-effects meta-analyses and number needed to treat (NNT).

RESULTS

Despite sustained improvements in selected biomarkers, no one drug regimen improved all intermediate outcomes. In 16 underpowered RCTs, drug treatments did not reduce mortality, liver cancer, or cirrhosis. Sustained HBV DNA clearance was achieved in one patient when two were treated with adefovir (NNT from 1 RCT=2 95%CI 1;2) or interferon alpha-2b (NNT from 2 RCTs=2 95%CI 2;4), 13 with lamivudine (NNT from 1 RCT=13 95%CI 7;1000), and 11 with peginterferon alpha-2a vs. lamivudine (NNT from 1 RCT=11 95%CI 7;25). Sustained HBeAg seroconversion was achieved in one patient when eight were treated with interferon alpha-2b (NNT from 2 RCTs=8 95%CI 5;33) or 10--with peginterferon alpha-2b vs. interferon alpha-2b (NNT from 1 RCT=10 95%CI 5;1000). Greater benefits and safety after entecavir vs. lamivudine or pegylated interferon alpha-2b vs. interferon alpha-2b require future investigation of clinical outcomes. Adverse events were common and more frequent after interferon. Treatment utilization for adverse effects is unknown.

CONCLUSIONS

Individual clinical decisions should rely on comparative effectiveness and absolute rates of intermediate outcomes and adverse events. Future research should clarify the relationship of intermediate and clinical outcomes and cost-effectiveness of drugs for evidence-based policy and clinical decisions.

[The concept of the development of S3 guidelines: additional benefit compared to traditional standards, problems and solutions]

Guidelines for the diagnosis and treatment of psychiatric disorders developed on the highest qualitative methodical standard cover reliable decision guidance and are an important instrument of quality improvement in health care. They have to be accessible with little effort and should be concise and clearly presented to be used in daily routine. Early integration of members of the different professions, patients and their care givers into the development process is essential for acceptance. The development process of a S3 guideline is complex and poses a difficult challenge to personal manpower and funding, which is met by the associations of the Association of Medical Societies in Germany. Further scientific discussion is necessary to find adequate solutions for problematic topics or potential distortion, such as the consequences of inadequate reporting of studies, insufficient recording and reporting of the risk of damage, unpublished data and lacking interest of sponsors in specific topics.

Acute adverse reactions of rapid Rituximab infusion among adult patients with Non-Hodgkin Lymphoma and Chronic Lymphocytic Leukemia

BACKGROUND

Rapid Rituximab infusion has become increasingly popular globally. Although pharmaceutical manufacturers recommend second and subsequent infusions to run over 2-3 hours, many cancer centres have changed their clinical practice based on their own research and the results from other primary studies. Such research studies claim that it is safe to administer Rituximab rapidly among cancer patients especially in Non Hodgkin Lymphoma (NHL) and Chronic Lymphocytic Leukemia (CLL). In addition, the studies suggest that the rapid infusion of Rituximab also results in benefits of cost saving and better resource utilisation. However, these studies have not been critically appraised for their validity and application to the global population. No previous systematic reviews on this topic have been identified.Objective The objective of this review was to critically appraise, synthesise and present the best available evidence related to the safety of rapid Rituximab infusion among adult patients with NHL and CLL.

INCLUSION CRITERIA

The participants of interest were adults aged 18 years old and above who had a diagnosis of NHL or CLL at any stage, have had prior exposure to Rituximab and received Rituximab with or without combination of any chemotherapy.The intervention of interest was rapid Rituximab infusion to be completed in 120 minutes or less.The studies of interest were both experimental and non-experimental studies.The primary outcomes of interest were the presence of acute adverse reactions and their severity. The secondary outcomes of interest were the management of the acute adverse reactions and patient mortality rate resulting from adverse reactions.Search strategy The search sought to identify published and unpublished studies from 1997 till 2010. A three-step search strategy was used for electronic databases, grey literature and reference lists.Methodological quality Two independent reviewers used the standard critical appraisal tool from JBI-MAStARI to assess the methodological qualities of the studies that matched with inclusion criteria.Data collection A standard data form from JBI-MAStARI was used to extract the data across all included studies.Data synthesis Proportional Meta-analysis based on DerSimonian-Laird weights for the random effects model was used for statistical pooling through Stats Direct. Heterogeneity was assessed using Cochran Q. When statistical pooling is not possible, the findings are presented in narrative summary.

RESULTS

A total of 753 patients were included in this review. All except one patient completed a total of 2298 cycles of rapid Rituximab infusions. Seventeen and one acute adverse reactions were reported among NHL and CLL patients respectively. There were five reactions which were not clearly stated if they were occurring in NHL or CLL patients. All were mild to moderate reactions except one patient developed severe reaction and withdrew from the study.

CONCLUSIONS

Rapid Rituximab infusion is safe for NHL patients especially in a 90-minute regimen. However, rapid rituximab infusion is not recommended for CLL patients due to lack of evidence.

IMPLICATIONS FOR PRACTICE

90-minute rapid Rituximab infusion with or without steroid premedication is recommended for NHL patient at second and subsequently infusions. No recommendations can be made in relation to stage of disease or the presence of bulky disease or leucocytosis. It is not recommended, based on the current evidence, to use rapid Rituximab for CLL patients.

IMPLICATIONS FOR RESEARCH

Further research is needed on the role of monoclonal antibodies development in rapid infusion, especially in the second and subsequent cycles. Currently, this review broadly establishes that rapid Rituximab infusion over 90 minutes is safe for NHL patients. However, more research and detailed analysis is needed to develop more specific guidelines.

The role of observational investigations in comparative effectiveness research

INTRODUCTION

Comparative effectiveness research (CER) seeks to inform clinical decisions between alternate treatment strategies using data that reflects real patient populations and real-world clinical scenarios for the purpose of improving patient outcomes. There are multiple clinical situations where the unique characteristics of observational investigations can inform medical decision-making within the CER paradigm. Accordingly, it is critical for clinicians to appreciate the strengths and limitations of observational research, particularly as they apply to CER.

METHODS

This review focuses on the role of observational research in CER. We discuss the concept of evidence hierarchies as they relate to observational research and CER, review the scope and nature of observational research, present the rationale for its inclusion in CER investigations, discuss potential sources of bias in observational investigations as well as strategies used to compensate for these biases, and discuss a framework to implement observational research in CER.

CONCLUSIONS

The CER paradigm recognizes the limitations of hierarchical models of evidence and favors application of a strength-of-evidence model. In this model, observational research fills gaps in randomized clinical trial data and is particularly valuable to investigate effectiveness, harms, prognosis, and infrequent outcomes as well as in circumstances where randomization is not possible and in studies of many surgical populations. Observational investigations must be designed with careful consideration of potential sources of bias and must incorporate strategies to control such bias prospectively, and their results must be reported in a uniform and transparent fashion. When these conditions can be achieved, observational research represents a valuable and critical component of modern CER.

Guide to designing, conducting, publishing and communicating results of clinical studies involving probiotic applications in human participants

The heterogeneity of human clinical trials to assess the effectiveness of probiotics presents challenges regarding interpretation and comparison. Evidence obtained from clinical trials among a population with a disease or specific risk factors may not be generalizable to healthy individuals. The evaluation of interventions in healthy persons requires careful selection of outcomes due to the absence of health indicators and the low incidence of preventable conditions. Given the tremendous resources invested in such trials, development of consistent approaches to assessing the effectiveness of probiotics would be beneficial. Furthermore, the reporting, presentation and communication of results may also affect the validity of the scientific evidence obtained from a trial. This review outlines the challenges associated with the design, implementation, data analysis and interpretation of clinical trials in humans involving probiotics. Best practices related to their design are offered along with recommendations for enhanced collaboration to advance research in this emerging field.