Uncertainty of the time of first significance in random effects cumulative meta-analysis

Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is one of the most commonly diagnosed and treated psychiatric disorders in childhood. Typically, children and adolescents with ADHD find it difficult to pay attention and they are hyperactive and impulsive. Methylphenidate is the psychostimulant most often prescribed, but the evidence on benefits and harms is uncertain. This is an update of our comprehensive systematic review on benefits and harms published in 2015.

OBJECTIVES

To assess the beneficial and harmful effects of methylphenidate for children and adolescents with ADHD.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, three other databases and two trials registers up to March 2022. In addition, we checked reference lists and requested published and unpublished data from manufacturers of methylphenidate.

SELECTION CRITERIA

We included all randomised clinical trials (RCTs) comparing methylphenidate versus placebo or no intervention in children and adolescents aged 18 years and younger with a diagnosis of ADHD. The search was not limited by publication year or language, but trial inclusion required that 75% or more of participants had a normal intellectual quotient (IQ > 70). We assessed two primary outcomes, ADHD symptoms and serious adverse events, and three secondary outcomes, adverse events considered non-serious, general behaviour, and quality of life.

DATA COLLECTION AND ANALYSIS

Two review authors independently conducted data extraction and risk of bias assessment for each trial. Six review authors including two review authors from the original publication participated in the update in 2022. We used standard Cochrane methodological procedures. Data from parallel-group trials and first-period data from cross-over trials formed the basis of our primary analyses. We undertook separate analyses using end-of-last period data from cross-over trials. We used Trial Sequential Analyses (TSA) to control for type I (5%) and type II (20%) errors, and we assessed and downgraded evidence according to the GRADE approach.

MAIN RESULTS

We included 212 trials (16,302 participants randomised); 55 parallel-group trials (8104 participants randomised), and 156 cross-over trials (8033 participants randomised) as well as one trial with a parallel phase (114 participants randomised) and a cross-over phase (165 participants randomised). The mean age of participants was 9.8 years ranging from 3 to 18 years (two trials from 3 to 21 years). The male-female ratio was 3:1. Most trials were carried out in high-income countries, and 86/212 included trials (41%) were funded or partly funded by the pharmaceutical industry. Methylphenidate treatment duration ranged from 1 to 425 days, with a mean duration of 28.8 days. Trials compared methylphenidate with placebo (200 trials) and with no intervention (12 trials). Only 165/212 trials included usable data on one or more outcomes from 14,271 participants. Of the 212 trials, we assessed 191 at high risk of bias and 21 at low risk of bias. If, however, deblinding of methylphenidate due to typical adverse events is considered, then all 212 trials were at high risk of bias.

PRIMARY OUTCOMES

methylphenidate versus placebo or no intervention may improve teacher-rated ADHD symptoms (standardised mean difference (SMD) -0.74, 95% confidence interval (CI) -0.88 to -0.61; I² = 38%; 21 trials; 1728 participants; very low-certainty evidence). This corresponds to a mean difference (MD) of -10.58 (95% CI -12.58 to -8.72) on the ADHD Rating Scale (ADHD-RS; range 0 to 72 points). The minimal clinically relevant difference is considered to be a change of 6.6 points on the ADHD-RS. Methylphenidate may not affect serious adverse events (risk ratio (RR) 0.80, 95% CI 0.39 to 1.67; I² = 0%; 26 trials, 3673 participants; very low-certainty evidence). The TSA-adjusted intervention effect was RR 0.91 (CI 0.31 to 2.68).

SECONDARY OUTCOMES

methylphenidate may cause more adverse events considered non-serious versus placebo or no intervention (RR 1.23, 95% CI 1.11 to 1.37; I² = 72%; 35 trials 5342 participants; very low-certainty evidence). The TSA-adjusted intervention effect was RR 1.22 (CI 1.08 to 1.43). Methylphenidate may improve teacher-rated general behaviour versus placebo (SMD -0.62, 95% CI -0.91 to -0.33; I² = 68%; 7 trials 792 participants; very low-certainty evidence), but may not affect quality of life (SMD 0.40, 95% CI -0.03 to 0.83; I² = 81%; 4 trials, 608 participants; very low-certainty evidence).

AUTHORS' CONCLUSIONS

The majority of our conclusions from the 2015 version of this review still apply. Our updated meta-analyses suggest that methylphenidate versus placebo or no-intervention may improve teacher-rated ADHD symptoms and general behaviour in children and adolescents with ADHD. There may be no effects on serious adverse events and quality of life. Methylphenidate may be associated with an increased risk of adverse events considered non-serious, such as sleep problems and decreased appetite. However, the certainty of the evidence for all outcomes is very low and therefore the true magnitude of effects remain unclear. Due to the frequency of non-serious adverse events associated with methylphenidate, the blinding of participants and outcome assessors is particularly challenging. To accommodate this challenge, an active placebo should be sought and utilised. It may be difficult to find such a drug, but identifying a substance that could mimic the easily recognised adverse effects of methylphenidate would avert the unblinding that detrimentally affects current randomised trials. Future systematic reviews should investigate the subgroups of patients with ADHD that may benefit most and least from methylphenidate. This could be done with individual participant data to investigate predictors and modifiers like age, comorbidity, and ADHD subtypes.

Pharmacological interventions for people with borderline personality disorder

BACKGROUND

Among people with a diagnosis of borderline personality disorder (BPD) who are engaged in clinical care, prescription rates of psychotropic medications are high, despite the fact that medication use is off-label as a treatment for BPD. Nevertheless, people with BPD often receive several psychotropic drugs at a time for sustained periods.

OBJECTIVES

To assess the effects of pharmacological treatment for people with BPD.

SEARCH METHODS

For this update, we searched CENTRAL, MEDLINE, Embase, 14 other databases and four trials registers up to February 2022. We contacted researchers working in the field to ask for additional data from published and unpublished trials, and handsearched relevant journals. We did not restrict the search by year of publication, language or type of publication.

SELECTION CRITERIA

Randomised controlled trials comparing pharmacological treatment to placebo, other pharmacologic treatments or a combination of pharmacologic treatments in people of all ages with a formal diagnosis of BPD. The primary outcomes were BPD symptom severity, self-harm, suicide-related outcomes, and psychosocial functioning. Secondary outcomes were individual BPD symptoms, depression, attrition and adverse events.

DATA COLLECTION AND ANALYSIS

At least two review authors independently selected trials, extracted data, assessed risk of bias using Cochrane's risk of bias tool and assessed the certainty of the evidence using the GRADE approach. We performed data analysis using Review Manager 5 and quantified the statistical reliability of the data using Trial Sequential Analysis.

MAIN RESULTS

We included 46 randomised controlled trials (2769 participants) in this review, 45 of which were eligible for quantitative analysis and comprised 2752 participants with BPD in total. This is 18 more trials than the 2010 review on this topic. Participants were predominantly female except for one trial that included men only. The mean age ranged from 16.2 to 39.7 years across the included trials. Twenty-nine different types of medications compared to placebo or other medications were included in the analyses. Seventeen trials were funded or partially funded by the pharmaceutical industry, 10 were funded by universities or research foundations, eight received no funding, and 11 had unclear funding. For all reported effect sizes, negative effect estimates indicate beneficial effects by active medication. Compared with placebo, no difference in effects were observed on any of the primary outcomes at the end of treatment for any medication. Compared with placebo, medication may have little to no effect on BPD symptom severity, although the evidence is of very low certainty (antipsychotics: SMD -0.18, 95% confidence interval (CI) -0.45 to 0.08; 8 trials, 951 participants; antidepressants: SMD -0.27, 95% CI -0.65 to 1.18; 2 trials, 87 participants; mood stabilisers: SMD -0.07, 95% CI -0.43 to 0.57; 4 trials, 265 participants). The evidence is very uncertain about the effect of medication compared with placebo on self-harm, indicating little to no effect (antipsychotics: RR 0.66, 95% CI 0.15 to 2.84; 2 trials, 76 participants; antidepressants: MD 0.45 points on the Overt Aggression Scale-Modified-Self-Injury item (0-5 points), 95% CI -10.55 to 11.45; 1 trial, 20 participants; mood stabilisers: RR 1.08, 95% CI 0.79 to 1.48; 1 trial, 276 participants). The evidence is also very uncertain about the effect of medication compared with placebo on suicide-related outcomes, with little to no effect (antipsychotics: SMD 0.05, 95 % CI -0.18 to 0.29; 7 trials, 854 participants; antidepressants: SMD -0.26, 95% CI -1.62 to 1.09; 2 trials, 45 participants; mood stabilisers: SMD -0.36, 95% CI -1.96 to 1.25; 2 trials, 44 participants). Very low-certainty evidence shows little to no difference between medication and placebo on psychosocial functioning (antipsychotics: SMD -0.16, 95% CI -0.33 to 0.00; 7 trials, 904 participants; antidepressants: SMD -0.25, 95% CI -0.57 to 0.06; 4 trials, 161 participants; mood stabilisers: SMD -0.01, 95% CI -0.28 to 0.26; 2 trials, 214 participants). Low-certainty evidence suggests that antipsychotics may slightly reduce interpersonal problems (SMD -0.21, 95% CI -0.34 to -0.08; 8 trials, 907 participants), and that mood stabilisers may result in a reduction in this outcome (SMD -0.58, 95% CI -1.14 to -0.02; 4 trials, 300 participants). Antidepressants may have little to no effect on interpersonal problems, but the corresponding evidence is very uncertain (SMD -0.07, 95% CI -0.69 to 0.55; 2 trials, 119 participants). The evidence is very uncertain about dropout rates compared with placebo by antipsychotics (RR 1.11, 95% CI 0.89 to 1.38; 13 trials, 1216 participants). Low-certainty evidence suggests there may be no difference in dropout rates between antidepressants (RR 1.07, 95% CI 0.65 to 1.76; 6 trials, 289 participants) and mood stabilisers (RR 0.89, 95% CI 0.69 to 1.15; 9 trials, 530 participants), compared to placebo. Reporting on adverse events was poor and mostly non-standardised. The available evidence on non-serious adverse events was of very low certainty for antipsychotics (RR 1.07, 95% CI 0.90 to 1.29; 5 trials, 814 participants) and mood stabilisers (RR 0.84, 95% CI 0.70 to 1.01; 1 trial, 276 participants). For antidepressants, no data on adverse events were identified.

AUTHORS' CONCLUSIONS

This review included 18 more trials than the 2010 version, so larger meta-analyses with more statistical power were feasible. We found mostly very low-certainty evidence that medication may result in no difference in any primary outcome. The rest of the secondary outcomes were inconclusive. Very limited data were available for serious adverse events. The review supports the continued understanding that no pharmacological therapy seems effective in specifically treating BPD pathology. More research is needed to understand the underlying pathophysiologic mechanisms of BPD better. Also, more trials including comorbidities such as trauma-related disorders, major depression, substance use disorders, or eating disorders are needed. Additionally, more focus should be put on male and adolescent samples.

A Comprehensive Appraisal of Meta-Analyses of Exercise-Based Stroke Rehabilitation with Trial Sequential Analysis

Meta-analysis is a common technique used to synthesise the results of multiple studies through the combination of effect size estimates and testing statistics. Numerous meta-analyses have investigated the efficacy of exercise programmes for stroke rehabilitation. However, meta-analyses may also report false-positive results because of insufficient information or random errors. Trial sequential analysis (TSA) is an advanced technique for calculating the required information size (RIS) and more restrictive statistical significance levels for the precise assessment of any specific treatment. This study used TSA to examine whether published meta-analyses in the field of stroke rehabilitation reached the RIS and whether their overall effect sizes were sufficient. A comprehensive search of six electronic databases for articles published before May 2022 was conducted. The intervention methods were divided into four primary groups, namely aerobic or resistance exercise, machine-assisted exercise, task-oriented exercise, and theory-based exercise. The primary outcome measure was gait speed and the secondary outcome measure was balance function. The data were obtained either from the meta-analyses or as raw data from the original cited texts. All data analysis was performed in TSA software. In total, 38 articles with 46 analysable results were included in the TSA. Only 17 results (37.0%) reached the RIS. In conclusion, meta-analysis interpretation is challenging. Clinicians must consider the RIS of meta-analyses before applying the results in real-world situations. TSA can provide accurate evaluations of treatment effects, which is crucial to the development of evidence-based medicine.

Demographics, Radiological Findings, and Clinical Outcomes of Klebsiella pneumonia vs. Non-Klebsiella pneumoniae Pyogenic Liver Abscess: A Systematic Review and Meta-Analysis with Trial Sequential Analysis

Pyogenic liver abscess (PLA) is a common cause of hepatobiliary sepsis. Klebsiella pneumoniae (KP) is the most common organism causing PLA. Evidence is scarce on the demographics, radiological findings, and outcomes of KPPLA versus non-KPPLA (N-KPPLA). PubMed, Embase, The Cochrane Library, and Scopus were systematically searched until 14 May 2022 for studies comparing KPPLA and N-KPPLA. Exclusion criteria were single-arm studies. Primary outcomes were mortality (30-day/in-hospital) and metastatic complications. There were 16 studies, including 5127 patients (KPPLA n = 3305, N-KPPLA n = 1822). Patients with KPPLA were younger (mean difference: −2.04 years, p = 0.02). History of hepatobiliary disease (Odds ratio (OR) 0.30, 95% CI: 0.20, 0.46) and malignancy (OR 0.26, 95% CI: 0.16, 0.42) were less common in KPPLA. KPPLA was associated with lower incidence of multiple abscesses (OR 0.52, 95% CI: 0.35, 0.76, p < 0.001) and bilobar abscesses (OR 0.60, 95% CI: 0.49, 0.74, p < 0.001). KPPLA has higher overall metastatic complications (KPPLA 9.7% vs. N-KPPLA 4.8%, OR 3.16, 95% CI: 2.00, 4.99, p < 0.001), but lower mortality (KPPLA 3.9% vs. N-KPPLA 7.6%, OR 0.51, 95% CI: 0.34, 0.78, p < 0.001). Trial sequential analysis showed conclusive evidence that KPPLA has lower mortality than N-KPPLA. In conclusion, KPPLA has lower mortality than N-KPPLA.

Control interventions in randomised trials among people with mental health disorders

BACKGROUND

Control interventions in randomised trials provide a frame of reference for the experimental interventions and enable estimations of causality. In the case of randomised trials assessing patients with mental health disorders, many different control interventions are used, and the choice of control intervention may have considerable impact on the estimated effects of the treatments being evaluated.

OBJECTIVES

To assess the benefits and harms of typical control interventions in randomised trials with patients with mental health disorders. The difference in effects between control interventions translates directly to the impact a control group has on the estimated effect of an experimental intervention. We aimed primarily to assess the difference in effects between (i) wait-list versus no-treatment, (ii) usual care versus wait-list or no-treatment, and (iii) placebo interventions (all placebo interventions combined or psychological, pharmacological, and physical placebos individually) versus wait-list or no-treatment. Wait-list patients are offered the experimental intervention by the researchers after the trial has been finalised if it offers more benefits than harms, while no-treatment participants are not offered the experimental intervention by the researchers.

SEARCH METHODS

In March 2018, we searched MEDLINE, PsycInfo, Embase, CENTRAL, and seven other databases and six trials registers.

SELECTION CRITERIA

We included randomised trials assessing patients with a mental health disorder that compared wait-list, usual care, or placebo interventions with wait-list or no-treatment .

DATA COLLECTION AND ANALYSIS

Titles, abstracts, and full texts were reviewed for eligibility. Review authors independently extracted data and assessed risk of bias using Cochrane's risk of bias tool. GRADE was used to assess the quality of the evidence. We contacted researchers working in the field to ask for data from additional published and unpublished trials. A pre-planned decision hierarchy was used to select one benefit and one harm outcome from each trial. For the assessment of benefits, we summarised continuous data as standardised mean differences (SMDs) and dichotomous data as risk ratios (RRs). We used risk differences (RDs) for the assessment of adverse events. We used random-effects models for all statistical analyses. We used subgroup analysis to explore potential causes for heterogeneity (e.g. type of placebo) and sensitivity analyses to explore the robustness of the primary analyses (e.g. fixed-effect model).

MAIN RESULTS

We included 96 randomised trials (4200 participants), ranging from 8 to 393 participants in each trial. 83 trials (3614 participants) provided usable data. The trials included 15 different mental health disorders, the most common being anxiety (25 trials), depression (16 trials), and sleep-wake disorders (11 trials). All 96 trials were assessed as high risk of bias partly because of the inability to blind participants and personnel in trials with two control interventions. The quality of evidence was rated low to very low, mostly due to risk of bias, imprecision in estimates, and heterogeneity. Only one trial compared wait-list versus no-treatment directly but the authors were not able to provide us with any usable data on the comparison. Five trials compared usual care versus wait-list or no-treatment and found a SMD -0.33 (95% CI -0.83 to 0.16, I² = 86%, 523 participants) on benefits. The difference between all placebo interventions combined versus wait-list or no-treatment was SMD -0.37 (95% CI -0.49 to -0.25, I² = 41%, 65 trials, 2446 participants) on benefits. There was evidence of some asymmetry in the funnel plot (Egger's test P value of 0.087). Almost all the trials were small. Subgroup analysis found a moderate effect in favour of psychological placebos SMD -0.49 (95% CI -0.64 to -0.30; I² = 53%, 39 trials, 1656 participants). The effect of pharmacological placebos versus wait-list or no-treatment on benefits was SMD -0.14 (95% CI -0.39 to 0.11, 9 trials, 279 participants) and the effect of physical placebos was SMD -0.21 (95% CI -0.35 to -0.08, I² = 0%, 17 trials, 896 participants). We found large variations in effect sizes in the psychological and pharmacological placebo comparisons. For specific mental health disorders, we found significant differences in favour of all placebos for sleep-wake disorders, major depressive disorder, and anxiety disorders, but the analyses were imprecise due to sparse data. We found no significant differences in harms for any of the comparisons but the analyses suffered from sparse data. When using a fixed-effect model in a sensitivity analysis on the comparison for usual care versus wait-list and no-treatment, the results were significant with an SMD of -0.46 (95 % CI -0.64 to -0.28). We reported an alternative risk of bias model where we excluded the blinding domains seeing how issues with blinding may be seen as part of the review investigation itself. However, this did not markedly change the overall risk of bias profile as most of the trials still included one or more unclear bias domains.

AUTHORS' CONCLUSIONS

We found marked variations in effects between placebo versus no-treatment and wait-list and between subtypes of placebo with the same comparisons. Almost all the trials were small with considerable methodological and clinical variability in factors such as mental health population, contents of the included control interventions, and outcome domains. All trials were assessed as high risk of bias and the evidence quality was low to very low. When researchers decide to use placebos or usual care control interventions in trials with people with mental health disorders it will often lead to lower estimated effects of the experimental intervention than when using wait-list or no-treatment controls. The choice of a control intervention therefore has considerable impact on how effective a mental health treatment appears to be. Methodological guideline development is needed to reach a consensus on future standards for the design and reporting of control interventions in mental health intervention research.

Living Systematic Reviews

Systematic reviews are difficult to keep up to date, but failure to do so leads to poor review currency and accuracy. "Living systematic review" (LSR) is an approach that aims to continually update a review, incorporating relevant new evidence as it becomes available. LSRs may be particularly important in fields where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions.This chapter describes the concept and processes of living systematic reviews. It describes the general principles of LSRs, when they might be of particular value, and how their procedures differ from conventional systematic reviews. The chapter focuses particularly on two methods of sequential meta-analysis that may be particularly useful for LSRs: Trial Sequential Analysis and Sequential Meta-Analysis, which both control for Type I error, Type II error (failing to detect a genuine effect) and take account of heterogeneity.

Association of interleukin-6 promoter polymorphism with rheumatoid arthritis: a meta-analysis with trial sequential analysis

OBJECTIVES

The association of interleukin-6 (IL-6) -174G/C (rs1800795) and IL-6 -572G/C (rs1800796) single-nucleotide polymorphism (SNP) with the risk of acquiring rheumatoid arthritis (RA) was inconsistent among previous studies. This paper aims to investigate the association between IL-6 promoter polymorphism with RA in different ethnics.

METHODS

Relevant studies were searched using Medline and Google Search engines; STATA software was used to perform the meta-analysis. Pooled odds ratios (OR) were calculated to estimate the potential genetic associations. Subgroup analysis and sensitivity analysis were applied to explore the sources of heterogeneity. Lastly, we used TSA (trial sequential analysis) software to verify the reliability of meta-analysis results.

RESULTS

A total of 18 studies were included, involving 8116 subjects (3820 RA patients and 4296 controls). We found a tendency to associate RA with the IL-6 -174G/C allele in Asians (C vs G: OR = 4.56, 95% CI = 1.85-11.23; P < 0.001); with IL-6 -572G/C genotype or allele frequencies, there was no statistical differences between RA patients and controls (P > 0.05). TSA results indicate that the current meta-analysis can draw conclusions.

CONCLUSIONS

IL-6-174G/C gene polymorphism were associated with increased risk of RA in Asians, but not in Caucasians. There was no association between IL-6 -572G/C gene polymorphism and the risk of RA. Key Points • Although the association between interleukin-6 (IL-6) promoter polymorphism and rheumatic arthritis (RA) has been discussed in the previous meta-analysis, their conclusions are inconsistent. • In this study, trial sequential analysis (TSA) was introduced into the meta-analysis, and the following two important conclusions were confirmed: (1) IL-6-174G/C gene polymorphism was associated with increased risk of RA in Asians, but not in Caucasians. (2) There was no association between IL-6 -572G/C gene polymorphism and the risk of RA.

Association of CTLA-4 (+49 A/G) polymorphism with susceptibility to autoimmune diseases: A meta-analysis with trial sequential analysis

OBJECTIVES

In recent years, more and more studies have been focusing on the association between Cytotoxic T lymphocyte antigen-4 (CTLA-4) (+49 A/G) gene polymorphism and autoimmune diseases. However, the results of previous studies are still controversial. The meta-analysis is aiming at determining the association in CTLA-4 (+49 A/G) gene rs231775 polymorphism and ankylosing spondylitis (AS), rheumatoid arthritis (RA), systemic lupus erythematosus (SLE).

METHODS

We searched PubMed, Web of Science, Chinese National Knowledge Infrastructure (CNKI) and Chinese Biomedical Database (CBM) up to November 2020, use random or fixed-effect models to perform meta-analysis to compare alleles and other genetic models, including homozygous, heterozygous, recessive and dominant models. The odds ratio (OR) with a 95% confidence interval (95% CI) was used to assess the correlation between CTLA-4 (+49 A/G) gene polymorphism and the genetic affectability of AS, RA, and SLE. Meanwhile, we used sequential trial analysis (TSA) to analyze the reliability of the results. Finally, we searched the relevant data of genome-wide association studies (GWAS) to further verify the accuracy of the experimental results.

RESULTS

47 studies with 11,893 cases and 12,032 healthy controls were included. The rs231775 G allele was relevant to high risk of autoimmune disease over all people (P < 0.05). The G allele of rs231775 was significantly related to RA susceptibility (P < 0.05), but not with AS or SLE. Subgroup analysis by ethnicity indicated that rs231775 G allele was closely related to RA in Caucasian populations and Mongolian populations (P < 0.05). A strong connection within rs231775 G allele and AS affectability was uncovered in Caucasian populations (P < 0.05). The analysis of the TSA shows that the meta-analysis can draw the conclusion.

CONCLUSION

CTLA-4 (+49 A/G) gene rs231775 G allele increases the risk of autoimmune diseases in Caucasian populations. And it also increases the risk of RA in Caucasian and Mongolian populations. More sample size and more elaborately designed studies are needed to elucidate the relationship in CTLA-4 (+49 A/G) gene rs231775 G allele and autoimmune diseases, especially AS, SLE.

[Types of evidence syntheses]

This manuscript outlines various types of review articles as forms of evidence synthesis with special regard to their strengths and limitations. Review articles not only present summarised data, but also offer an evaluation of the quality of the individual studies included in it. The validity and the reliability of outcomes of reviews is strongly dependent on the quality of the data included. For this reason, a comprehensive literature selection process is paramount. Fundamental knowledge of bias and literature assessment is also necessary when reading reviews. This article presents selected tools for evidence appraisal and evaluation of bias risk.

Psychological therapies for people with borderline personality disorder

BACKGROUND

Over the decades, a variety of psychological interventions for borderline personality disorder (BPD) have been developed. This review updates and replaces an earlier review (Stoffers-Winterling 2012).

OBJECTIVES

To assess the beneficial and harmful effects of psychological therapies for people with BPD.

SEARCH METHODS

In March 2019, we searched CENTRAL, MEDLINE, Embase, 14 other databases and four trials registers. We contacted researchers working in the field to ask for additional data from published and unpublished trials, and handsearched relevant journals. We did not restrict the search by year of publication, language or type of publication.

SELECTION CRITERIA

Randomised controlled trials comparing different psychotherapeutic interventions with treatment-as-usual (TAU; which included various kinds of psychotherapy), waiting list, no treatment or active treatments in samples of all ages, in any setting, with a formal diagnosis of BPD. The primary outcomes were BPD symptom severity, self-harm, suicide-related outcomes, and psychosocial functioning. There were 11 secondary outcomes, including individual BPD symptoms, as well as attrition and adverse effects.

DATA COLLECTION AND ANALYSIS

At least two review authors independently selected trials, extracted data, assessed risk of bias using Cochrane's 'Risk of bias' tool and assessed the certainty of the evidence using the GRADE approach. We performed data analysis using Review Manager 5 and quantified the statistical reliability of the data using Trial Sequential Analysis.

MAIN RESULTS

We included 75 randomised controlled trials (4507 participants), predominantly involving females with mean ages ranging from 14.8 to 45.7 years. More than 16 different kinds of psychotherapy were included, mostly dialectical behaviour therapy (DBT) and mentalisation-based treatment (MBT). The comparator interventions included treatment-as-usual (TAU), waiting list, and other active treatments. Treatment duration ranged from one to 36 months. Psychotherapy versus TAU Psychotherapy reduced BPD symptom severity, compared to TAU; standardised mean difference (SMD) -0.52, 95% confidence interval (CI) -0.70 to -0.33; 22 trials, 1244 participants; moderate-quality evidence. This corresponds to a mean difference (MD) of -3.6 (95% CI -4.4 to -2.08) on the Zanarini Rating Scale for BPD (range 0 to 36), a clinically relevant reduction in BPD symptom severity (minimal clinical relevant difference (MIREDIF) on this scale is -3.0 points). Psychotherapy may be more effective at reducing self-harm compared to TAU (SMD -0.32, 95% CI -0.49 to -0.14; 13 trials, 616 participants; low-quality evidence), corresponding to a MD of -0.82 (95% CI -1.25 to 0.35) on the Deliberate Self-Harm Inventory Scale (range 0 to 34). The MIREDIF of -1.25 points was not reached. Suicide-related outcomes improved compared to TAU (SMD -0.34, 95% CI -0.57 to -0.11; 13 trials, 666 participants; low-quality evidence), corresponding to a MD of -0.11 (95% CI -0.19 to -0.034) on the Suicidal Attempt Self Injury Interview. The MIREDIF of -0.17 points was not reached. Compared to TAU, psychotherapy may result in an improvement in psychosocial functioning (SMD -0.45, 95% CI -0.68 to -0.22; 22 trials, 1314 participants; low-quality evidence), corresponding to a MD of -2.8 (95% CI -4.25 to -1.38), on the Global Assessment of Functioning Scale (range 0 to 100). The MIREDIF of -4.0 points was not reached. Our additional Trial Sequential Analysis on all primary outcomes reaching significance found that the required information size was reached in all cases. A subgroup analysis comparing the different types of psychotherapy compared to TAU showed no clear evidence of a difference for BPD severity and psychosocial functioning. Psychotherapy may reduce depressive symptoms compared to TAU but the evidence is very uncertain (SMD -0.39, 95% CI -0.61 to -0.17; 22 trials, 1568 participants; very low-quality evidence), corresponding to a MD of -2.45 points on the Hamilton Depression Scale (range 0 to 50). The MIREDIF of -3.0 points was not reached. BPD-specific psychotherapy did not reduce attrition compared with TAU. Adverse effects were unclear due to too few data. Psychotherapy versus waiting list or no treatment Greater improvements in BPD symptom severity (SMD -0.49, 95% CI -0.93 to -0.05; 3 trials, 161 participants), psychosocial functioning (SMD -0.56, 95% CI -1.01 to -0.11; 5 trials, 219 participants), and depression (SMD -1.28, 95% CI -2.21 to -0.34, 6 trials, 239 participants) were observed in participants receiving psychotherapy versus waiting list or no treatment (all low-quality evidence). No evidence of a difference was found for self-harm and suicide-related outcomes. Individual treatment approaches DBT and MBT have the highest numbers of primary trials, with DBT as subject of one-third of all included trials, followed by MBT with seven RCTs. Compared to TAU, DBT was more effective at reducing BPD severity (SMD -0.60, 95% CI -1.05 to -0.14; 3 trials, 149 participants), self-harm (SMD -0.28, 95% CI -0.48 to -0.07; 7 trials, 376 participants) and improving psychosocial functioning (SMD -0.36, 95% CI -0.69 to -0.03; 6 trials, 225 participants). MBT appears to be more effective than TAU at reducing self-harm (RR 0.62, 95% CI 0.49 to 0.80; 3 trials, 252 participants), suicidality (RR 0.10, 95% CI 0.04, 0.30, 3 trials, 218 participants) and depression (SMD -0.58, 95% CI -1.22 to 0.05, 4 trials, 333 participants). All findings are based on low-quality evidence. For secondary outcomes see review text.

AUTHORS' CONCLUSIONS

Our assessments showed beneficial effects on all primary outcomes in favour of BPD-tailored psychotherapy compared with TAU. However, only the outcome of BPD severity reached the MIREDIF-defined cut-off for a clinically meaningful improvement. Subgroup analyses found no evidence of a difference in effect estimates between the different types of therapies (compared to TAU) . The pooled analysis of psychotherapy versus waiting list or no treatment found significant improvement on BPD severity, psychosocial functioning and depression at end of treatment, but these findings were based on low-quality evidence, and the true magnitude of these effects is uncertain. No clear evidence of difference was found for self-harm and suicide-related outcomes. However, compared to TAU, we observed effects in favour of DBT for BPD severity, self-harm and psychosocial functioning and, for MBT, on self-harm and suicidality at end of treatment, but these were all based on low-quality evidence. Therefore, we are unsure whether these effects would alter with the addition of more data.

Social skills training for attention deficit hyperactivity disorder (ADHD) in children aged 5 to 18 years

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) in children is associated with hyperactivity and impulsivity, attention problems, and difficulties with social interactions. Pharmacological treatment may alleviate the symptoms of ADHD but this rarely solves difficulties with social interactions. Children with ADHD may benefit from interventions designed to improve their social skills. We examined the benefits and harms of social skills training on social skills, emotional competencies, general behaviour, ADHD symptoms, performance in school of children with ADHD, and adverse events.

OBJECTIVES

To assess the beneficial and harmful effects of social skills training in children and adolescents with ADHD.

SEARCH METHODS

In July 2018, we searched CENTRAL, MEDLINE, Embase, PsycINFO, 4 other databases and two trials registers.We also searched online conference abstracts, and contacted experts in the field for information about unpublished or ongoing randomised clinical trials. We did not limit our searches by language, year of publication, or type or status of publication, and we sought translation of the relevant sections of non-English language articles.

SELECTION CRITERIA

Randomised clinical trials investigating social skills training versus either no intervention or waiting-list control, with or without pharmacological treatment of both comparison groups of children and adolescents with ADHD.

DATA COLLECTION AND ANALYSIS

We conducted the review in accordance with the Cochrane Handbook for Systematic Reviews of Intervention. We performed the analyses using Review Manager 5 software and Trial Sequential Analysis. We assessed bias according to domains for systematic errors. We assessed the certainty of the evidence with the GRADE approach.

MAIN RESULTS

We included 25 randomised clinical trials described in 45 reports. The trials included a total of 2690 participants aged between five and 17 years. In 17 trials, participants were also diagnosed with various comorbidities.The social skills interventions were described as: 1) social skills training, 2) cognitive behavioural therapy, 3) multimodal behavioural/psychosocial therapy, 4) child life and attention skills treatment, 5) life skills training, 6) the "challenging horizon programme", 7) verbal self-instruction, 8) meta-cognitive training, 9) behavioural therapy, 10) behavioural and social skills treatment, and 11) psychosocial treatment. The control interventions were no intervention or waiting list.The duration of the interventions ranged from five weeks to two years. We considered the content of the social skills interventions to be comparable and based on a cognitive-behavioural model. Most of the trials compared child social skills training or parent training combined with medication versus medication alone. Some of the experimental interventions also included teacher consultations.More than half of the trials were at high risk of bias for generation of the allocation sequence and allocation concealment. No trial reported on blinding of participants and personnel. Most of the trials did not report on differences between groups in medication for comorbid disorders. We used all eligible trials in the meta-analyses, but downgraded the certainty of the evidence to low or very low.We found no clinically relevant treatment effect of social skills interventions on the primary outcome measures: teacher-rated social skills at end of treatment (standardised mean difference (SMD) 0.11, 95% confidence interval (CI) 0.00 to 0.22; 11 trials, 1271 participants; I2 = 0%; P = 0.05); teacher-rated emotional competencies at end of treatment (SMD -0.02, 95% CI -0.72 to 0.68; two trials, 129 participants; I2 = 74%; P = 0.96); or on teacher-rated general behaviour (SMD -0.06 (negative value better), 95% CI -0.19 to 0.06; eight trials, 1002 participants; I2 = 0%; P = 0.33). The effect on the primary outcome, teacher-rated social skills at end of treatment, corresponds to a MD of 1.22 points on the social skills rating system (SSRS) scale (95% CI 0.09 to 2.36). The minimal clinical relevant difference (10%) on the SSRS is 10.0 points (range 0 to 102 points on SSRS).We found evidence in favour of social skills training on teacher-rated core ADHD symptoms at end of treatment for all eligible trials (SMD -0.26, 95% CI -0.47 to -0.05; 14 trials, 1379 participants; I2= 69%; P = 0.02), but the finding is questionable due to lack of support from sensitivity analyses, high risk of bias, lack of clinical significance, high heterogeneity, and low certainty.The studies did not report any serious or non-serious adverse events.

AUTHORS' CONCLUSIONS

The review suggests that there is little evidence to support or refute social skills training for children and adolescents with ADHD. We may need more trials that are at low risk of bias and a sufficient number of participants to determine the efficacy of social skills training versus no training for ADHD. The evidence base regarding adolescents is especially weak.

Pharmacological interventions for people with borderline personality disorder

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To assess the beneficial and harmful effects of pharmacological treatment for adolescents and adults with borderline personality disorder (BPD).

A large-scale assessment of temporal trends in meta-analyses using systematic review reports from the Cochrane Library

INTRODUCTION

Previous studies suggest that many systematic reviews contain meta-analyses that display temporal trends, such as the first study's result being more extreme than later studies' or a drift in the pooled estimate. We assessed the extent and characteristics of temporal trends using all Cochrane intervention reports published 2008-2012.

METHODS

We selected the largest meta-analysis within each report and analysed trends using methods including a Z-test (first versus subsequent estimates); generalised least squares; and cumulative sum charts. Predictors considered include meta-analysis size and review group.

RESULTS

Of 1288 meta-analyses containing at least 4 studies, the point estimate from the first study was more extreme and in the same direction as the pooled estimate in 738 (57%), with a statistically significant difference (first versus subsequent) in 165 (13%). Generalised least squares indicated trends in 717 (56%); 18% of fixed effects analyses had at least one violation of cumulative sum limits. For some methods, meta-analysis size was associated with temporal patterns and use of a random effects model, but there was no consistent association with review group.

CONCLUSIONS

All results suggest that more meta-analyses demonstrate temporal patterns than would be expected by chance. Hence, assuming the standard meta-analysis model without temporal trend is sometimes inappropriate. Factors associated with trends are likely to be context specific.

Living systematic reviews: 3. Statistical methods for updating meta-analyses

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Trial Sequential Analysis in systematic reviews with meta-analysis

BACKGROUND

Most meta-analyses in systematic reviews, including Cochrane ones, do not have sufficient statistical power to detect or refute even large intervention effects. This is why a meta-analysis ought to be regarded as an interim analysis on its way towards a required information size. The results of the meta-analyses should relate the total number of randomised participants to the estimated required meta-analytic information size accounting for statistical diversity. When the number of participants and the corresponding number of trials in a meta-analysis are insufficient, the use of the traditional 95% confidence interval or the 5% statistical significance threshold will lead to too many false positive conclusions (type I errors) and too many false negative conclusions (type II errors).

METHODS

We developed a methodology for interpreting meta-analysis results, using generally accepted, valid evidence on how to adjust thresholds for significance in randomised clinical trials when the required sample size has not been reached.

RESULTS

The Lan-DeMets trial sequential monitoring boundaries in Trial Sequential Analysis offer adjusted confidence intervals and restricted thresholds for statistical significance when the diversity-adjusted required information size and the corresponding number of required trials for the meta-analysis have not been reached. Trial Sequential Analysis provides a frequentistic approach to control both type I and type II errors. We define the required information size and the corresponding number of required trials in a meta-analysis and the diversity (D²) measure of heterogeneity. We explain the reasons for using Trial Sequential Analysis of meta-analysis when the actual information size fails to reach the required information size. We present examples drawn from traditional meta-analyses using unadjusted naïve 95% confidence intervals and 5% thresholds for statistical significance. Spurious conclusions in systematic reviews with traditional meta-analyses can be reduced using Trial Sequential Analysis. Several empirical studies have demonstrated that the Trial Sequential Analysis provides better control of type I errors and of type II errors than the traditional naïve meta-analysis.

CONCLUSIONS

Trial Sequential Analysis represents analysis of meta-analytic data, with transparent assumptions, and better control of type I and type II errors than the traditional meta-analysis using naïve unadjusted confidence intervals.

Continuously updated network meta-analysis and statistical monitoring for timely decision-making

Pairwise and network meta-analysis (NMA) are traditionally used retrospectively to assess existing evidence. However, the current evidence often undergoes several updates as new studies become available. In each update recommendations about the conclusiveness of the evidence and the need of future studies need to be made. In the context of prospective meta-analysis future studies are planned as part of the accumulation of the evidence. In this setting, multiple testing issues need to be taken into account when the meta-analysis results are interpreted. We extend ideas of sequential monitoring of meta-analysis to provide a methodological framework for updating NMAs. Based on the z-score for each network estimate (the ratio of effect size to its standard error) and the respective information gained after each study enters NMA we construct efficacy and futility stopping boundaries. A NMA treatment effect is considered conclusive when it crosses an appended stopping boundary. The methods are illustrated using a recently published NMA where we show that evidence about a particular comparison can become conclusive via indirect evidence even if no further trials address this comparison.

False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: an empirical review

OBJECTIVE

Many published meta-analyses are underpowered. We explored the role of trial sequential analysis (TSA) in assessing the reliability of conclusions in underpowered meta-analyses.

METHODS

We screened The Cochrane Database of Systematic Reviews and selected 100 meta-analyses with a binary outcome, a negative result and sufficient power. We defined a negative result as one where the 95% CI for the effect included 1.00, a positive result as one where the 95% CI did not include 1.00, and sufficient power as the required information size for 80% power, 5% type 1 error, relative risk reduction of 10% or number needed to treat of 100, and control event proportion and heterogeneity taken from the included studies. We re-conducted the meta-analyses, using conventional cumulative techniques, to measure how many false positives would have occurred if these meta-analyses had been updated after each new trial. For each false positive, we performed TSA, using three different approaches.

RESULTS

We screened 4736 systematic reviews to find 100 meta-analyses that fulfilled our inclusion criteria. Using conventional cumulative meta-analysis, false positives were present in seven of the meta-analyses (7%, 95% CI 3% to 14%), occurring more than once in three. The total number of false positives was 14 and TSA prevented 13 of these (93%, 95% CI 68% to 98%). In a post hoc analysis, we found that Cochrane meta-analyses that are negative are 1.67 times more likely to be updated (95% CI 0.92 to 2.68) than those that are positive.

CONCLUSIONS

We found false positives in 7% (95% CI 3% to 14%) of the included meta-analyses. Owing to limitations of external validity and to the decreased likelihood of updating positive meta-analyses, the true proportion of false positives in meta-analysis is probably higher. TSA prevented 93% of the false positives (95% CI 68% to 98%).

Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes

Background Cumulative meta-analysis typically involves performing an updated meta-analysis every time when new trials are added to a series of similar trials, which by definition involves multiple inspections. Neither the commonly used random effects model nor the conventional group sequential method can control the type I error for many practical situations. In our previous research, Lan et al. (Lan KKG, Hu M-X, Cappelleri JC. Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Statistica Sinica 2003; 13: 1135—45) proposed an approach based on the law of iterated logarithm (LIL) to this problem for the continuous case.

Purpose The study is an extension and generalization of our previous research to binary outcomes. Although it is based on the same LIL principle, we found the discrete case much more complex and the results from the continuous case do not apply to the binary case. The simulation study presented here is also more extensive.

Methods The LIL based method `penalizes' the Z-value of the test statistic to account for multiple tests and for the estimation of heterogeneity in treatment effects across studies. It involves an adjustment factor, which is directly related to the control of type I error and determined through extensive simulations under various conditions.

Results With an adjustment factor of 2, the LIL-based test statistics controls the overall type I error when odds ratio or relative risk is the parameter of interest. For risk difference, the adjustment factor can be reduced to 1.5. More inspections may require a larger adjustment factor, but the required adjustment factor stabilizes after 25 inspections.

Limitations It will be ideal if the adjustment factor can be obtained theoretically through a statistical model. Unfortunately, real life data are too complex and we have to solve the problem through simulation. However, for large number of inspections, the adjustment factor will have a limited effect and the type I error is controlled mainly by the LIL.

Conclusions The LIL method controls the overall type I error for a very broad range of practical situations with a binary outcome, and the LIL works properly in controlling the type I error rates as the number of inspections becomes large. Clinical Trials 2007; 4: 329—340. http://ctj.sagepub.com

Systematic Reviews of Anesthesiologic Interventions Reported as Statistically Significant: Problems with Power, Precision, and Type 1 Error Protection

BACKGROUND

The GRADE Working Group assessment of the quality of evidence is being used increasingly to inform clinical decisions and guidelines. The assessment involves explicit consideration of all sources of uncertainty. One of these sources is imprecision or random error. Many published meta-analyses are underpowered and likely to be updated in the future. When data are sparse and there are repeated updates, the risk of random error is increased. Trial Sequential Analysis (TSA) is one of several methodologies that estimates this increased risk (and decreased precision) in meta-analyses. With nominally statistically significant meta-analyses of anesthesiologic interventions, we used TSA to estimate power and imprecision in the context of sparse data and repeated updates.

METHODS

We conducted a search to identify all systematic reviews with meta-analyses that investigated an intervention that may be implemented by an anesthesiologist during the perioperative period. We randomly selected 50 meta-analyses that reported a statistically significant dichotomous outcome in their abstract. We applied TSA to these meta-analyses by using 2 main TSA approaches: relative risk reduction 20% and relative risk reduction consistent with the conventional 95% confidence limit closest to null. We calculated the power achieved by each included meta-analysis, by using each TSA approach, and we calculated the proportion that maintained statistical significance when allowing for sparse data and repeated updates.

RESULTS

From 11,870 titles, we found 682 systematic reviews that investigated anesthesiologic interventions. In the 50 sampled meta-analyses, the median number of trials included was 8 (interquartile range [IQR], 5-14), the median number of participants was 964 (IQR, 523-1736), and the median number of participants with the outcome was 202 (IQR, 96-443). By using both of our main TSA approaches, only 12% (95% CI, 5%-25%) of the meta-analyses had power ≥ 80%, and only 32% (95% CI, 20%-47%) of the meta-analyses preserved the risk of type 1 error <5%.

CONCLUSIONS

Most nominally statistically significant meta-analyses of anesthesiologic interventions are underpowered, and many do not maintain their risk of type 1 error <5% if TSA monitoring boundaries are applied. Consideration of the effect of sparse data and repeated updates is needed when assessing the imprecision of meta-analyses of anesthesiologic interventions.

Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is one of the most commonly diagnosed and treated psychiatric disorders in childhood. Typically, children with ADHD find it difficult to pay attention, they are hyperactive and impulsive.Methylphenidate is the drug most often prescribed to treat children and adolescents with ADHD but, despite its widespread use, this is the first comprehensive systematic review of its benefits and harms.

OBJECTIVES

To assess the beneficial and harmful effects of methylphenidate for children and adolescents with ADHD.

SEARCH METHODS

In February 2015 we searched six databases (CENTRAL, Ovid MEDLINE, EMBASE, CINAHL, PsycINFO, Conference Proceedings Citations Index), and two trials registers. We checked for additional trials in the reference lists of relevant reviews and included trials. We contacted the pharmaceutical companies that manufacture methylphenidate to request published and unpublished data.

SELECTION CRITERIA

We included all randomised controlled trials (RCTs) comparing methylphenidate versus placebo or no intervention in children and adolescents aged 18 years and younger with a diagnosis of ADHD. At least 75% of participants needed to have an intellectual quotient of at least 70 (i.e. normal intellectual functioning). Outcomes assessed included ADHD symptoms, serious adverse events, non-serious adverse events, general behaviour and quality of life.

DATA COLLECTION AND ANALYSIS

Seventeen review authors participated in data extraction and risk of bias assessment, and two review authors independently performed all tasks. We used standard methodological procedures expected within Cochrane. Data from parallel-group trials and first period data from cross-over trials formed the basis of our primary analyses; separate analyses were undertaken using post-cross-over data from cross-over trials. We used Trial Sequential Analyses to control for type I (5%) and type II (20%) errors, and we assessed and downgraded evidence according to the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach for high risk of bias, imprecision, indirectness, heterogeneity and publication bias.

MAIN RESULTS

The studies.We included 38 parallel-group trials (5111 participants randomised) and 147 cross-over trials (7134 participants randomised). Participants included individuals of both sexes, at a boys-to-girls ratio of 5:1, and participants' ages ranged from 3 to 18 years across most studies (in two studies ages ranged from 3 to 21 years). The average age across all studies was 9.7 years. Most participants were from high-income countries.The duration of methylphenidate treatment ranged from 1 to 425 days, with an average duration of 75 days. Methylphenidate was compared to placebo (175 trials) or no intervention (10 trials). Risk of Bias.All 185 trials were assessed to be at high risk of bias. Primary outcomes. Methylphenidate may improve teacher-rated ADHD symptoms (standardised mean difference (SMD) -0.77, 95% confidence interval (CI) -0.90 to -0.64; 19 trials, 1698 participants; very low-quality evidence). This corresponds to a mean difference (MD) of -9.6 points (95% CI -13.75 to -6.38) on the ADHD Rating Scale (ADHD-RS; range 0 to 72 points; DuPaul 1991a). A change of 6.6 points on the ADHD-RS is considered clinically to represent the minimal relevant difference. There was no evidence that methylphenidate was associated with an increase in serious (e.g. life threatening) adverse events (risk ratio (RR) 0.98, 95% CI 0.44 to 2.22; 9 trials, 1532 participants; very low-quality evidence). The Trial Sequential Analysis-adjusted intervention effect was RR 0.91 (CI 0.02 to 33.2).

SECONDARY OUTCOMES

Among those prescribed methylphenidate, 526 per 1000 (range 448 to 615) experienced non-serious adverse events, compared with 408 per 1000 in the control group. This equates to a 29% increase in the overall risk of any non-serious adverse events (RR 1.29, 95% CI 1.10 to 1.51; 21 trials, 3132 participants; very low-quality evidence). The Trial Sequential Analysis-adjusted intervention effect was RR 1.29 (CI 1.06 to 1.56). The most common non-serious adverse events were sleep problems and decreased appetite. Children in the methylphenidate group were at 60% greater risk for trouble sleeping/sleep problems (RR 1.60, 95% CI 1.15 to 2.23; 13 trials, 2416 participants), and 266% greater risk for decreased appetite (RR 3.66, 95% CI 2.56 to 5.23; 16 trials, 2962 participants) than children in the control group.Teacher-rated general behaviour seemed to improve with methylphenidate (SMD -0.87, 95% CI -1.04 to -0.71; 5 trials, 668 participants; very low-quality evidence).A change of seven points on the Child Health Questionnaire (CHQ; range 0 to 100 points; Landgraf 1998) has been deemed a minimal clinically relevant difference. The change reported in a meta-analysis of three trials corresponds to a MD of 8.0 points (95% CI 5.49 to 10.46) on the CHQ, which suggests that methylphenidate may improve parent-reported quality of life (SMD 0.61, 95% CI 0.42 to 0.80; 3 trials, 514 participants; very low-quality evidence).

AUTHORS' CONCLUSIONS

The results of meta-analyses suggest that methylphenidate may improve teacher-reported ADHD symptoms, teacher-reported general behaviour, and parent-reported quality of life among children and adolescents diagnosed with ADHD. However, the low quality of the underpinning evidence means that we cannot be certain of the magnitude of the effects. Within the short follow-up periods typical of the included trials, there is some evidence that methylphenidate is associated with increased risk of non-serious adverse events, such as sleep problems and decreased appetite, but no evidence that it increases risk of serious adverse events.Better designed trials are needed to assess the benefits of methylphenidate. Given the frequency of non-serious adverse events associated with methylphenidate, the particular difficulties for blinding of participants and outcome assessors point to the advantage of large, 'nocebo tablet' controlled trials. These use a placebo-like substance that causes adverse events in the control arm that are comparable to those associated with methylphenidate. However, for ethical reasons, such trials should first be conducted with adults, who can give their informed consent.Future trials should publish depersonalised individual participant data and report all outcomes, including adverse events. This will enable researchers conducting systematic reviews to assess differences between intervention effects according to age, sex, comorbidity, type of ADHD and dose. Finally, the findings highlight the urgent need for large RCTs of non-pharmacological treatments.

A collaborative sequential meta-analysis of individual patient data from randomized trials of endovascular therapy and tPA vs. tPA alone for acute ischemic stroke: ThRombEctomy And tPA (TREAT) analysis: statistical analysis plan for a sequential meta-analysis performed within the VISTA-Endovascular collaboration

Rationale

Endovascular treatment has been shown to restore blood flow effectively. Second-generation medical devices such as stent retrievers are now showing overwhelming efficacy in clinical trials, particularly in conjunction with intravenous recombinant tissue plasminogen activator.

Aims and Design

This statistical analysis plan utilizing a novel, sequential approach describes a prospective, individual patient data analysis of endovascular therapy in conjunction with intravenous recombinant tissue plasminogen activator agreed upon by the Thrombectomy and Tissue Plasminogen Activator Collaborative Group.

Study outcomes

This protocol will specify the primary outcome for efficacy, as ‘favorable’ outcome defined by the ordinal distribution of the modified Rankin Scale measured at three-months poststroke, but with modified Rankin Scales 5 and 6 collapsed into a single category. The primary analysis will aim to answer the questions: ‘what is the treatment effect of endovascular therapy with intravenous recombinant tissue plasminogen activator compared to intravenous tissue plasmi-nogen activator alone on full scale modified Rankin Scale at 3 months?’ and ‘to what extent do key patient characteristics influence the treatment effect of endovascular therapy?’. Key secondary outcomes include effect of endovascular therapy on death within 90 days; analyses of modified Rankin Scale using dichotomized methods; and effects of endovascular therapy on symptomatic intracranial hemorrhage. Several secondary analyses will be considered as well as expanding patient cohorts to intravenous recombinant tissue plasminogen activator-ineligible patients, should data allow.

Discussion

This collaborative meta-analysis of individual participant data from randomized trials of endovascular therapy vs. control in conjunction with intravenous thrombolysis will demonstrate the efficacy and generalizability of endovascular therapy with intravenous thrombolysis as a concomitant medication.

Constraint-induced movement therapy: trial sequential analysis applied to Cochrane collaboration systematic review results

BACKGROUND

Trial sequential analysis (TSA) may establish when firm evidence about the efficacy of interventions is reached in a cumulative meta-analysis, combining a required information size with adjusted thresholds for conservative statistical significance. Our aim was to demonstrate TSA results on randomized controlled trials (RCTs) included in a Cochrane systematic review on the effectiveness of constraint-induced movement therapy (CIMT) for stroke patients.

METHODS

We extracted data on the functional independence measure (FIM) and the action research arm test (ARAT) from RCTs that compared CIMT versus other rehabilitative techniques. Mean differences (MD) were analyzed using a random-effects model. We calculated the information size and the cumulative Z-statistic, applying the O'Brien-Fleming monitoring boundaries.

RESULTS

We included data from 14 RCTs. In the conventional meta-analysis (seven trials, 233 patients), the effect of CIMT on FIM was reported as significant (MD 2.88, 95% CI 0.08 to 5.68; P = 0.04). The diversity-adjusted required information size was 142 patients, and the cumulative Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.02 to 5.78). The effect of CIMT on ARAT (nine trials, 199 patients) was reported as significant (MD 7.78, 95% CI 1.19 to 14.37; P = 0.02). However, the diversity-adjusted required information size was 252 patients, and the Z-score did not cross the trial sequential monitoring boundary for benefit (adjusted 95% CI -0.06 to 15.62).

CONCLUSIONS

Although conventional meta-analyses of CIMT reached statistical significance, their overall results remain inconclusive and might be spurious. Researchers should not be overconfident on CIMT efficacy based on the results of meta-analyses and derived recommendations.

Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap

The current difficulties in keeping systematic reviews up to date leads to considerable inaccuracy, hampering the translation of knowledge into action. Incremental advances in conventional review updating are unlikely to lead to substantial improvements in review currency. A new approach is needed. We propose living systematic review as a contribution to evidence synthesis that combines currency with rigour to enhance the accuracy and utility of health evidence. Living systematic reviews are high quality, up-to-date online summaries of health research, updated as new research becomes available, and enabled by improved production efficiency and adherence to the norms of scholarly communication. Together with innovations in primary research reporting and the creation and use of evidence in health systems, living systematic review contributes to an emerging evidence ecosystem.

To drain or not to drain: a cumulative meta-analysis of the use of routine abdominal drains after pancreatic resection

BACKGROUND

To warrant the adoption or rejection of health care interventions in daily practice, it is important to establish the point at which the available evidence is considered sufficiently conclusive. This process must avoid bias resulting from multiple testing and take account of heterogeneity across studies. The present paper addresses the issue of whether the available evidence may be considered sufficiently conclusive to continue or discontinue the current practice of postoperative abdominal drainage after pancreatic resection.

METHODS

A systematic review was conducted of randomized and non-randomized studies comparing outcomes after routine intra-abdominal drainage with those after no drainage after pancreatic resection. Studies were retrieved from the PubMed, Cochrane Central Trial Register and EMBASE databases and meta-analysed cumulatively, adjusting for multiple testing and heterogeneity using the iterated logarithm method.

RESULTS

Three reports, describing, respectively, one randomized and two non-randomized studies with a comparative design, met the inclusion criteria predefined for primary studies reporting on drain management and complications after pancreatic resection. These studies included 89, 179 and 226 patients, respectively. The absolute differences in rates of postoperative complications in these studies were -6.4%, -9.5% and -6.3%, respectively, in favour of the no-drain groups. The cumulative risk difference in major complications, adjusted for multiple testing and heterogeneity, was -7.8%, with a 95% confidence interval of -20.2% to 4.7% (P = 0.214).

CONCLUSIONS

The routine use of abdominal drains after pancreatic resection may result in a higher risk for major complications, but the evidence is inconclusive.

Social skills training for Attention Deficit Hyperactivity Disorder (ADHD) in children aged 5 to 18 years

BACKGROUND

Attention Deficit Hyperactivity Disorder (ADHD) in children is associated with hyperactivity and impulsitivity, attention problems, and difficulties with social interactions. Pharmacological treatment may alleviate symptoms of ADHD but seldom solves difficulties with social interactions. Social skills training may benefit ADHD children in their social interactions. We examined the effects of social skills training on children's social competences, general behaviour, ADHD symptoms, and performance in school.

OBJECTIVES

To assess the effects of social skills training in children and adolescents with ADHD.

SEARCH METHODS

We searched the following electronic databases: CENTRAL (2011, Issue1), MEDLINE (1948 to March 2011), EMBASE (1980 to March 2011), ERIC (1966 to March 2011 ), AMED (1985 to June 2011), PsycINFO (1806 to March 2011), CINAHL (1980 to March 2011), and Sociological Abstracts (1952 to March 2011). We also searched the metaRegister of Controlled Trials on 15 October 2010. We did not apply any language or date restrictions to the searches. We searched online conference abstracts and contacted 176 experts in the field for possible information about unpublished or ongoing RCTs.

SELECTION CRITERIA

Randomised trials investigating social skills training for children with ADHD as a stand alone treatment or as an adjunct to pharmacological treatment.

DATA COLLECTION AND ANALYSIS

We conducted the review according to the Cochrane Handbook for Systematic Reviews of Intervention. Two authors (OJS, MS) extracted data independently using an appropriate data collection form. We performed the analyses using Review Manager 5 software.

MAIN RESULTS

We included 11 randomised trials described in 26 records (all full text articles) in the review. The trials included a total of 747 participants. All participants were between five and 12 years of age. No trials assessed adolescents. In 10 of the trials the participants suffered from different comorbidities.The duration of the interventions ranged from eight to 10 weeks (eight trials) up to two years. The types of social skills interventions were named social skills training, cognitive behavioural intervention, multimodal behavioural/psychosocial therapy, behavioural therapy/treatment, behavioural and social skills treatment, and psychosocial treatment. The content of the social skills interventions were comparable and based on a cognitive behavioural model. Most of the trials compared child social skills training and parent training plus medication versus medication alone. Some of the experimental interventions also included teacher consultations.More than half of the trials were at high risk of bias regarding generation of the allocation sequence and allocation concealment. No trial reported blinding of participants and personnel and most of the trials had no reports regarding differences between groups in collateral medication for comorbid disorders. Overall, the trials had high risk of bias due to systematic errors. Even so, as recommended by the Cochrane Handbook of Systematic Reviews of Interventions, we used all eligible trials in the meta-analysis, but the results are downgraded to low quality evidence.There were no statistically significant treatment effects either on social skills competences (positive value = better for the intervention group) (SMD 0.16; 95% CI -0.04 to 0.36; 5 trials, n = 392), on the teacher-rated general behaviour (negative value = better for the intervention group) (SMD 0.00; 95% CI -0.21 to 0.21; 3 trials, n = 358), or on the ADHD symptoms (negative value = better for the intervention group) (SMD -0.02; 95% CI -0.19 to 0.16; 6 trials, n = 515).No serious or non-serious adverse events were reported.

AUTHORS' CONCLUSIONS

The review suggests that there is little evidence to support or refute social skills training for adolescents with ADHD. There is need for more trials, with low risk of bias and with a sufficient number of participants, investigating the efficacy of social skills training versus no training for both children and adolescents.

The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis--a simulation study

BACKGROUND

Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated.

METHODS

We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error.

RESULTS

The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation.

CONCLUSIONS

Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation.

Multiple treatment comparison meta-analyses: a step forward into complexity

The use of meta-analysis has become increasingly useful for clinical and policy decision making. A recent development in meta-analysis, multiple treatment comparison (MTC) meta-analysis, provides inferences on the comparative effectiveness of interventions that may have never been directly evaluated in clinical trials. This new approach may be confusing for clinicians and methodologists and raises specific challenges relevant to certain areas of medicine. This article addresses the methodological concepts of MTC meta-analysis, including issues of heterogeneity, choice of model, and adequacy of sample sizes. We address domain-specific challenges relevant to disciplines of medicine, including baseline risks of patient populations. We conclude that MTC meta-analysis is a useful tool in the context of comparative effectiveness and requires further study, as its utility and transparency will likely predict its uptake by the research and clinical community.

Sequential methods for random-effects meta-analysis

Although meta-analyses are typically viewed as retrospective activities, they are increasingly being applied prospectively to provide up-to-date evidence on specific research questions. When meta-analyses are updated account should be taken of the possibility of false-positive findings due to repeated significance tests. We discuss the use of sequential methods for meta-analyses that incorporate random effects to allow for heterogeneity across studies. We propose a method that uses an approximate semi-Bayes procedure to update evidence on the among-study variance, starting with an informative prior distribution that might be based on findings from previous meta-analyses. We compare our methods with other approaches, including the traditional method of cumulative meta-analysis, in a simulation study and observe that it has Type I and Type II error rates close to the nominal level. We illustrate the method using an example in the treatment of bleeding peptic ulcers. Copyright © 2010 John Wiley & Sons, Ltd.

Evidence at a glance: error matrix approach for overviewing available evidence

Background

Clinical evidence continues to expand and is increasingly difficult to overview. We aimed at conceptualizing a visual assessment tool, i.e., a matrix for overviewing studies and their data in order to assess the clinical evidence at a glance.

Methods

A four-step matrix was constructed using the three dimensions of systematic error, random error, and design error. Matrix step I ranks the identified studies according to the dimensions of systematic errors and random errors. Matrix step II orders the studies according to the design errors. Matrix step III assesses the three dimensions of errors in studies. Matrix step IV assesses the size and direction of the intervention effect.

Results

The application of this four-step matrix is illustrated with two examples: peri-operative beta-blockade initialized in relation to surgery versus placebo for major non-cardiac surgery, and antiarrhythmics for maintaining sinus rhythm after cardioversion of atrial fibrillation. When clinical evidence is deemed both internally and externally valid, the size of the intervention effect is to be assessed.

Conclusion

The error matrix provides an overview of the validity of the available evidence at a glance, and may assist in deciding which interventions to use in clinical practice.

Meta-analysis: ribavirin plus interferon vs. interferon monotherapy for chronic hepatitic C - an updated Cochrane review

BACKGROUND

Multiple randomized trials have been published on antiviral treatment for chronic hepatitis C.

AIM

To meta-analyse the effect of adding ribavirin to interferon for chronic hepatitis C.

METHODS

The results of randomized trials were combined in cumulative meta-analyses. Trial sequential analyses were used to adjust for spurious results because of random errors and multiplicity. The outcome measures were undetectable hepatitis C virus RNA in serum (sustained virological response) and liver-related morbidity plus all-cause mortality.

RESULTS

We included 82 randomized trials with 12 615 patients. Trial sequential analysis established clear beneficial effect of interferon plus ribavirin vs. interferon on the sustained virological response in 1998 after nine trials (RR: 0.74; 95% CI: 0.64-0.85, P < 0.0001, 1734 patients). Subsequently, additional 73 trials were published just narrowing the confidence interval and decreasing the P-value. By contrast, trial sequential analysis found that additional evidence is needed to convincingly detect a beneficial effect of interferon plus ribavirin vs. interferon monotherapy on clinical outcomes.

CONCLUSIONS

The rationale behind several recent trials on adding ribavirin to interferon for chronic hepatitis C is debatable as the effect on virological response is established. More evidence is needed to assess if adding ribavirin to interferon improves clinical outcomes.

Interpreting meta-analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV-infected individuals

Objective:

To illustrate the utility of statistical monitoring boundaries in meta-analysis, and provide a framework in which meta-analysis can be interpreted according to the adequacy of sample size. To propose a simple method for determining how many patients need to be randomized in a future trial before a meta-analysis can be deemed conclusive.

Study design and setting:

Prospective meta-analysis of randomized clinical trials (RCTs) that evaluated the effectiveness of isoniazid chemoprophylaxis versus placebo for preventing the incidence of tuberculosis disease among human immunodeficiency virus (HIV)-positive individuals testing purified protein derivative negative. Assessment of meta-analysis precision using trial sequential analysis (TSA) with LanDeMets monitoring boundaries. Sample size determination for a future trials to make the meta-analysis conclusive according to the thresholds set by the monitoring boundaries.

Results:

The meta-analysis included nine trials comprising 2,911 trial participants and yielded a relative risk of 0.74 (95% CI, 0.53–1.04, P = 0.082, I² = 0%). To deem the meta-analysis conclusive according to the thresholds set by the monitoring boundaries, a future RCT would need to randomize 3,800 participants.

Conclusion:

Statistical monitoring boundaries provide a framework for interpreting meta-analysis according to the adequacy of sample size and project the required sample size for a future RCT to make a meta-analysis conclusive.

Sequential meta-analysis: an efficient decision-making tool

BACKGROUND

A cumulative meta-analysis of successive randomized controlled trials (RCTs) can be used to decide whether enough evidence has been obtained comparing a control and an intervention treatment or whether a new RCT should be initiated. In general, no adjustment is made for repeatedly testing the null hypothesis of treatment equivalence on cumulative data. Neither can the power of the statistical test be quantified. Recently, trial sequential analysis (TSA) was suggested to '. . . establish when firm evidence is reached in cumulative meta-analysis'. TSA is based on alpha-spending functions and necessitates a prior estimate of the total information size. Various information sizes were suggested.

PURPOSE

The aim of this study is to compare TSA with sequential meta-analysis (SMA) following Whitehead's boundaries approach.

METHODS

We compare TSA and SMA by re-analysis of a number of published examples.

RESULTS

Re-analysis of the examples shows that for an SMA: (1) no prior estimate for total information size is necessary and thus one set of boundaries suffices; (2) stopping a cumulative meta-analysis for futility is an option; (3) the power can be quantified; (4) point and interval estimates are adjusted for the multiple testing; and (5) gains in efficiency can be achieved, both for efficacy and for futility and thus ethical and economical benefits can be obtained.

LIMITATIONS

Estimates for between-trial variability are unstable for a small number of trials. The behavior of a newly proposed estimate should be subject of further investigation.

CONCLUSION

SMA is a useful tool to investigate the cumulative evidence from successive RCTs.

Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team

Background The Safety Planning, Evaluation and Reporting Team (SPERT) was formed in 2006 by the Pharmaceutical Research and Manufacturers of America.

Purpose SPERT’s goal was to propose a pharmaceutical industry standard for safety planning, data collection, evaluation, and reporting, beginning with planning first-in-human studies and continuing through the planning of the post-product-approval period.

Methods SPERT’s recommendations are based on our review of relevant literature and on consensus reached in our discussions.

Results An important recommendation is that sponsors create a Program Safety Analysis Plan early in development. We also give recommendations for the planning of repeated, cumulative meta-analyses of the safety data obtained from the studies conducted within the development program. These include clear definitions of adverse events of special interest and standardization of many aspects of data collection and study design. We describe a 3-tier system for signal detection and analysis of adverse events and highlight proposals for reducing "false positive" safety findings. We recommend that sponsors review the aggregated safety data on a regular and ongoing basis throughout the development program, rather than waiting until the time of submission.

Limitations We recognize that there may be other valid approaches.

Conclusions The proactive approach we advocate has the potential to benefit patients and health care providers by providing more comprehensive safety information at the time of new product marketing and beyond. Clinical Trials 2009; 6: 430—440. http://ctj.sagepub.com

Apparently conclusive meta-analyses may be inconclusive--Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses

BACKGROUND

Random error may cause misleading evidence in meta-analyses. The required number of participants in a meta-analysis (i.e. information size) should be at least as large as an adequately powered single trial. Trial sequential analysis (TSA) may reduce risk of random errors due to repetitive testing of accumulating data by evaluating meta-analyses not reaching the information size with monitoring boundaries. This is analogous to sequential monitoring boundaries in a single trial.

METHODS

We selected apparently conclusive (P </= 0.05) Cochrane neonatal meta-analyses. We applied heterogeneity-adjusted and unadjusted TSA on these meta-analyses by calculating the information size, the monitoring boundaries, and the cumulative Z-statistic after each trial. We identified the proportion of meta-analyses that did not reach the required information size and the proportion of these meta-analyses in which the Z-curve did not cross the monitoring boundaries.

RESULTS

Of 54 apparently conclusive meta-analyses, 39 (72%) did not reach the heterogeneity-adjusted information size required to accept or reject an intervention effect of 25% relative risk reduction. Of these 39, 19 meta-analyses (49%) were considered inconclusive, because the cumulative Z-curve did not cross the monitoring boundaries. The median number of participants required to reach the required information size was 1591 (range, 339-6149). TSA without heterogeneity adjustment largely confirmed these results.

CONCLUSIONS

Many apparently conclusive Cochrane neonatal meta-analyses may become inconclusive when the statistical analyses take into account the risk of random error due to repetitive testing.

Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?

BACKGROUND

Results from apparently conclusive meta-analyses may be false. A limited number of events from a few small trials and the associated random error may be under-recognized sources of spurious findings. The information size (IS, i.e. number of participants) required for a reliable and conclusive meta-analysis should be no less rigorous than the sample size of a single, optimally powered randomized clinical trial. If a meta-analysis is conducted before a sufficient IS is reached, it should be evaluated in a manner that accounts for the increased risk that the result might represent a chance finding (i.e. applying trial sequential monitoring boundaries).

METHODS

We analysed 33 meta-analyses with a sufficient IS to detect a treatment effect of 15% relative risk reduction (RRR). We successively monitored the results of the meta-analyses by generating interim cumulative meta-analyses after each included trial and evaluated their results using a conventional statistical criterion (alpha = 0.05) and two-sided Lan-DeMets monitoring boundaries. We examined the proportion of false positive results and important inaccuracies in estimates of treatment effects that resulted from the two approaches.

RESULTS

Using the random-effects model and final data, 12 of the meta-analyses yielded P > alpha = 0.05, and 21 yielded P </= alpha = 0.05. False positive interim results were observed in 3 out of 12 meta-analyses with P > alpha = 0.05. The monitoring boundaries eliminated all false positives. Important inaccuracies in estimates were observed in 6 out of 21 meta-analyses using the conventional P </= alpha = 0.05 and 0 out of 21 using the monitoring boundaries.

CONCLUSIONS

Evaluating statistical inference with trial sequential monitoring boundaries when meta-analyses fall short of a required IS may reduce the risk of false positive results and important inaccurate effect estimates.

Recent developments in meta-analysis

The art and science of meta-analysis, the combination of results from multiple independent studies, is now more than a century old. In the last 30 years, however, as the need for medical research and clinical practice to be based on the totality of relevant and sound evidence has been increasingly recognized, the impact of meta-analysis has grown enormously. In this paper, we review highlights of recent developments in meta-analysis in medical research. We outline in particular how emphasis has been placed on (i) heterogeneity and random-effects analyses; (ii) special consideration in different areas of application; (iii) assessing bias within and across studies; and (iv) extension of ideas to complex evidence synthesis. We conclude the paper with some remarks on ongoing challenges and possible directions for the future.

When and how to update systematic reviews

BACKGROUND

Systematic reviews are most helpful if they are up-to-date. We did a systematic review of strategies and methods describing when and how to update systematic reviews.

OBJECTIVES

To identify, describe and assess strategies and methods addressing: 1) when to update systematic reviews and 2) how to update systematic reviews.

SEARCH STRATEGY

We searched MEDLINE (1966 to December 2005), PsycINFO, the Cochrane Methodology Register (Issue 1, 2006), and hand searched the 2005 Cochrane Colloquium proceedings.

SELECTION CRITERIA

We included methodology reports, updated systematic reviews, commentaries, editorials, or other short reports describing the development, use, or comparison of strategies and methods for determining the need for updating or updating systematic reviews in healthcare.

DATA COLLECTION AND ANALYSIS

We abstracted information from each included report using a 15-item questionnaire. The strategies and methods for updating systematic reviews were assessed and compared descriptively with respect to their usefulness, comprehensiveness, advantages, and disadvantages.

MAIN RESULTS

Four updating strategies, one technique, and two statistical methods were identified. Three strategies addressed steps for updating and one strategy presented a model for assessing the need to update. One technique discussed the use of the "entry date" field in bibliographic searching. Statistical methods were cumulative meta-analysis and predicting when meta-analyses are outdated.

AUTHORS' CONCLUSIONS

Little research has been conducted on when and how to update systematic reviews and the feasibility and efficiency of the identified approaches is uncertain. These shortcomings should be addressed in future research.

Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis

BACKGROUND AND OBJECTIVE

Cumulative meta-analyses are prone to produce spurious P<0.05 because of repeated testing of significance as trial data accumulate. Information size in a meta-analysis should at least equal the sample size of an adequately powered trial. Trial sequential analysis (TSA) corresponds to group sequential analysis of a single trial and may be applied to meta-analysis to evaluate the evidence.

STUDY DESIGN AND SETTING

Six randomly selected neonatal meta-analyses with at least five trials reporting a binary outcome were examined. Low-bias heterogeneity-adjusted information size and information size determined from an assumed intervention effect of 15% were calculated. These were used for constructing trial sequential monitoring boundaries. We assessed the cumulative z-curves' crossing of P=0.05 and the boundaries.

RESULTS

Five meta-analyses showed early potentially spurious P<0.05 values. In three significant meta-analyses the cumulative z-curves crossed both boundaries, establishing firm evidence of an intervention effect. In two nonsignificant meta-analyses the cumulative z-curves crossed P=0.05, but never the boundaries, demonstrating early potentially spurious P<0.05 values. In one nonsignificant meta-analysis the cumulative z-curves never crossed P=0.05 or the boundaries.

CONCLUSION

TSAs may establish when firm evidence is reached in meta-analysis.

A systematic review identified few methods and strategies describing when and how to update systematic reviews

OBJECTIVE

Systematic reviews (SRs) are convenient summaries of evidence for health care practitioners. They form a basis for clinical practice guidelines and suggest directions for new research. SRs are most helpful if they are current; however, most of them are not being updated. This SR summarizes strategies and methods describing when and how to update SRs.

STUDY DESIGN AND SETTING

We searched MEDLINE (1966 to December 2005), PsycINFO, the Cochrane Methodology Register, and the 2005 Cochrane Colloquium proceedings to identify records describing when and how to update SRs in health care.

RESULTS

Four updating strategies, one technique, and two statistical methods were identified. Three strategies addressed steps for updating, and one strategy presented a model for assessing the need to update. One technique discussed the use of the "entry date" field in bibliographic searching. The statistical methods were cumulative meta-analysis and a test for detecting outdated meta-analyses with statistically nonsignificant results.

CONCLUSION

Little research has been conducted on when and how to update SRs in contrast to other methodological areas of conducting SRs (e.g., publication bias, variance imputation). The feasibility and efficiency of the identified approaches is uncertain. More research is needed to develop pragmatic and efficient methodologies for updating SRs.

Sequential Meta-analysis of Past Clinical Trials to Determine the Use of a New Trial

BACKGROUND

Clinical trials can be stopped early based on interim analyses or sequential analyses. In principle, sequential analyses can also be used to decide whether enough evidence has been gathered in completed trials to make further trials unnecessary. We demonstrate such an application through a retrospective analysis of clinical trials comparing ventilation methods for the treatment of preterm newborns.

METHODS

We identified 5 recent trials that compared high-frequency ventilation with conventional mechanical ventilation in the treatment of preterm newborns. Death or chronic lung disease and chronic lung disease in survivors were the primary clinical outcomes of interest. We applied sequential meta-analyses to these 5 studies.

RESULTS

After including the first study of the last 5 trials in a sequential meta-analysis, the boundary of "no clinically relevant effect" was crossed for both outcomes (death or chronic lung disease). A sensitivity analysis using a reduction in the size of assumed clinically relevant effect showed the same findings after 2 trials.

CONCLUSIONS

Sequential meta-analyses showed that a lack of clinically relevant effect had been established after the first of the 5 trials. If such an analysis had been conducted after the first or second of these clinical trials, it might have led to changes in the study design of subsequent trials or even to a reassessment of the need for further trials.