Specifying the target difference in the primary outcome for a randomised controlled trial: guidance for researchers

A protocol for the conduct of a multicentre, prospective, randomized superiority trial of surgical versus non-surgical interventions for humeral shaft fractures

Aims

Fractures of the humeral shaft represent 3% to 5% of all fractures. The most common treatment for isolated humeral diaphysis fractures in the UK is non-operative using functional bracing, which carries a low risk of complications, but is associated with a longer healing time and a greater risk of nonunion than surgery. There is an increasing trend to surgical treatment, which may lead to quicker functional recovery and lower rates of fracture nonunion than functional bracing. However, surgery carries inherent risk, including infection, bleeding, and nerve damage. The aim of this trial is to evaluate the clinical and cost-effectiveness of functional bracing compared to surgical fixation for the treatment of humeral shaft fractures.

Methods

The HUmeral SHaft (HUSH) fracture study is a multicentre, prospective randomized superiority trial of surgical versus non-surgical interventions for humeral shaft fractures in adult patients. Participants will be randomized to receive either functional bracing or surgery. With 334 participants, the trial will have 90% power to detect a clinically important difference for the Disabilities of the Arm, Shoulder and Hand questionnaire score, assuming 20% loss to follow-up. Secondary outcomes will include function, pain, quality of life, complications, cost-effectiveness, time off work, and ability to drive.

Discussion

The results of this trial will provide evidence regarding clinical and cost-effectiveness between surgical and non-surgical treatment of humeral shaft fractures. Ethical approval has been obtained from East of England - Cambridge Central Research Ethics Committee. Publication is anticipated to occur in 2024.

Using the benefit-harm trade-off method to determine the smallest worthwhile effect of intensive motor training on strength for people with spinal cord injury

STUDY DESIGN

Interviews using the benefit-harm trade-off method and an online survey.

OBJECTIVES

To determine the smallest worthwhile effect (SWE) of motor training on strength for people with spinal cord injury (SCI).

SETTING

SCI units, Australia.

METHODS

Forty people with recent SCI who had participated in motor training as part of their rehabilitation program (patient participants) and 37 physiotherapists (physiotherapist participants) working in SCI were recruited. The patient participants underwent an iterative process using the benefit-harm trade-off method to determine the SWE of motor training on strength. The physiotherapist participants were given an online survey to determine the SWE for five different scenarios. Both groups considered the SWE of a physiotherapy intervention involving an additional 12 h of motor training for 10 weeks on top of usual care. They were required to estimate the smallest improvement in strength (points on the Total Motor Score of the International Standards for Neurological Classification of SCI) to justify the effort and associated costs, risks or inconveniences of the motor training.

RESULTS

The median (interquartile range) smallest improvement in strength that patient and physiotherapist participants deemed worth the effort and associated costs, risks or inconveniences of the motor training was 3 (1-5) points, and 9 (7-13) points, respectively.

CONCLUSIONS

People with recent SCI are willing to devote 12 h a week for 10 weeks to motor training in addition to their usual care to gain small changes in strength. Physiotherapists wanted to see greater improvements to justify the intervention.

Comparison of endoscopic and microscopic management of attic cholesteatoma: A randomized controlled trial

BACKGROUND

Attic cholesteatoma is a common disease encountered by otologists.

OBJECTIVES

To compare the endoscopic approach to attic cholesteatoma with conventional microscopic technique.

MATERIAL AND METHODS

A total of 190 patients (192 ears) diagnosed with attic cholesteatoma extending to the antrum area (stages Ib and II) were randomly assigned into two groups undergoing endoscopic approach and the other microscopic technique. The outcomes were preoperative and intraoperative findings, access to hidden areas expressed in MESVI, mean operative time from first incision to ear-packing, and postoperative findings. Statistical analysis was performed by SPSS version 24.0, and P ≤ 0.05 was considered statistically significant.

RESULTS

The median Middle Ear Structural Visibility Index of the endoscopic group was better than the microscopic group (P < 0.05). The mean operating time by the endoscopic approach was less than the microscopic approach (P < 0.05). The median postoperative pain score in the endoscopic group was lower than the microscopic group (P < 0.05). In addition, there were no statistically significant differences in taste, hearing, vertigo, healing time and long term outcomes between the two groups.

CONCLUSION AND SIGNIFICANCE

Endoscopic management of limited attic cholesteatoma showed definite advantages over the conventional microscopic approach, such as providing better visualization, requiring less postoperative time, subjecting the patients to less pain, and decreasing the incidence of complications.

Structure-Function Relationships in the Rodent Streptozotocin-Induced Model for Diabetic Retinopathy: A Systematic Review

The streptozotocin (STZ)-induced rodent model is one of the most commonly employed models in preclinical drug discovery for diabetic retinopathy (DR). However, standardization and validation of experimental readouts are largely lacking. The aim of this systematic review was to identify and compare the most useful readouts of STZ-induced DR and provide recommendations for future study design based on our findings. We performed a systematic search using 2 major databases, PubMed and EMBASE. Only articles describing STZ-induced DR describing both functional and structural readouts were selected. We also assessed the risk of bias and analyzed qualitative data in the selected studies. We identified 21 studies that met our inclusion/exclusion criteria, using either rats or mice and study periods of 2 to 24 weeks. Glucose level thresholds used to define hyperglycemia were inconsistent between studies, however, most studies used either 250 or 300.6 mg/dL as a defining criterion for hyperglycemia. All included studies performed electroretinography (ERG) and reported a reduction in a-, b-, or c-wave and/or oscillatory potential amplitudes. Spectral-domain optical coherence tomography and fluorescein angiography, as well as immunohistochemical and histopathological analyses showed reductions in retinal thickness, vascular changes, and presence of inflammation. Risk of bias assessment showed that all studies had a high risk of bias due to lack of reporting or correctly following procedures. Our systematic review highlights that ERG represents the most consistent functional readout in the STZ model. However, due to the high risk of bias, caution must be used when interpreting these studies.

Protocol for a randomised controlled feasibility study examining the efficacy of brief cognitive therapy for the treatment of panic disorder in adolescents (PANDA)

BACKGROUND

Panic disorder occurs in between 1 and 3% of adolescents, is associated with high levels of co-morbidity, and without treatment, appears to have a chronic course. To improve access to effective psychological interventions, briefer versions of cognitive behaviour therapy (CBT) have been developed and evaluated for preadolescent children with anxiety disorders. However, there are currently no brief evidence-based CBT interventions for adolescents with anxiety disorders that can be delivered in less than eight sessions. Given that a brief version of cognitive therapy has been shown to be effective in adults with panic disorder, it is possible that an adapted version could be effective for adolescents with panic disorder.

METHODS

The study will examine whether a definitive trial can be conducted, based on a single-centre feasibility randomised controlled trial using several well-defined criteria. Between 30 and 48 young people (age 11-18 years) who meet diagnostic criteria for panic disorder, attending a routine clinical service will be randomly allocated to receive either (i) brief cognitive therapy or (ii) a general form of CBT treatment that is more commonly used for adolescents with anxiety disorders. Both will be delivered 1:1 by a therapist and involve five treatment sessions and two booster sessions. Young people's outcomes will be assessed at the end of treatment and at 3-month follow-up, and qualitative interviews will be conducted to examine acceptability. We will also explore outcomes 1 year after the completion of treatment.

DISCUSSION

This study will test the feasibility of a randomised controlled trial to compare brief cognitive therapy to a general form of CBT for adolescents with panic disorder in the UK. The outputs from the study will provide a clear indication of the feasibility of a future definitive trial and, if indicated, the critical resources that will be required and key information to inform the design and maximise the successful completion of the trial. This has the potential to bring direct benefits to young people and their families, as well as services and society more broadly.

TRIAL REGISTRATION

This trial is registered on the ISRCTN Registry, registration number ISRCTN14884288 , registered retrospectively on 05/12/2019.

A Multicentered Randomized Controlled Trial Comparing the Effectiveness of Pain Treatment Communication Tools in Emergency Department Patients With Back or Kidney Stone Pain

Objectives. To compare the effectiveness of 3 approaches for communicating opioid risk during an emergency department visit for a common painful condition. Methods. This parallel, multicenter randomized controlled trial was conducted at 6 geographically disparate emergency department sites in the United States. Participants included adult patients between 18 and 70 years of age presenting with kidney stone or musculoskeletal back pain. Participants were randomly assigned to 1 of 3 risk communication strategies: (1) a personalized probabilistic risk visual aid, (2) a visual aid and a video narrative, or 3) general risk information. The primary outcomes were accuracy of risk recall, reported opioid use, and treatment preference at time of discharge. Results. A total of 1301 participants were enrolled between June 2017 and August 2019. There was no difference in risk recall at 14 days between the narrative and probabilistic groups (43.7% vs 38.8%; absolute risk reduction = 4.9%; 95% confidence interval [CI] = -2.98, 12.75). The narrative group had lower rates of preference for opioids at discharge than the general risk information group (25.9% vs 33.0%; difference = 7.1%; 95% CI = 0.64, 0.97). There were no differences in reported opioid use at 14 days between the narrative, probabilistic, and general risk groups (10.5%, 10.3%, and 13.3%, respectively; P = .44). Conclusions. An emergency medicine communication tool incorporating probabilistic risk and patient narratives was more effective than general information in mitigating preferences for opioids in the treatment of pain but was not more effective with respect to opioid use or risk recall. Trial Registration. Clinical Trials.gov identifier: NCT03134092. (Am J Public Health. 2022;112(S1):S45-S55. https://doi.org/10.2105/AJPH.2021.306511).

OUP accepted manuscript

Introduction

Antimicrobial stewardship interventions (ASIs) aim to reduce the emergence of antimicrobial resistance. We sought to systematically evaluate how microbiological outcomes have been handled and analysed in randomized controlled trials (RCTs) evaluating ASIs.

Methods

We searched PubMed and Embase from 2011–21. Studies were selected if they were RCTs evaluating ASIs. A narrative synthesis approach was taken, identifying whether the study reported any microbiological data (bacterial genus/species; bacterial colony counts; prevalence of bacterial, microbiologically defined infections; and antibiotic susceptibility, measured pre-randomization or post-randomization in one arm only) or outcomes (post-randomization data compared between arms). Studies with or without microbiological data/outcomes were summarized in terms of study characteristics, methods of reporting and analysis of these outcomes.

Results

We identified 117 studies, with 34 (29.1%) collecting microbiological data and 18 (15.4%) reporting microbiological outcomes. Most studies with microbiological outcomes were conducted in secondary care (12/18, 66.7%) and targeted adult populations (14/18, 77.8%), and the intervention involved biomarker-guided rapid diagnostic testing (7/18, 38.9%). The overall quality of reporting and analysing microbiological outcomes was low and inconsistent. The selected study population in analyses and methods of handling missing data were unclear.

Conclusions

This review demonstrates that the quality of handling and reporting microbiological outcomes in RCTs of ASIs was low. The lack of consistency and clarity made it difficult to compare the findings across studies, limiting policy- and clinical decision-making. Therefore, there is a clear need for the development of guidance for handling microbiological outcomes in RCTs and adopting appropriate methods to evaluate these data carefully.

Living Systematic Reviews

Systematic reviews are difficult to keep up to date, but failure to do so leads to poor review currency and accuracy. "Living systematic review" (LSR) is an approach that aims to continually update a review, incorporating relevant new evidence as it becomes available. LSRs may be particularly important in fields where research evidence is emerging rapidly, current evidence is uncertain, and new research may change policy or practice decisions.This chapter describes the concept and processes of living systematic reviews. It describes the general principles of LSRs, when they might be of particular value, and how their procedures differ from conventional systematic reviews. The chapter focuses particularly on two methods of sequential meta-analysis that may be particularly useful for LSRs: Trial Sequential Analysis and Sequential Meta-Analysis, which both control for Type I error, Type II error (failing to detect a genuine effect) and take account of heterogeneity.

Minimal clinically important difference in means in vulnerable populations: challenges and solutions

INTRODUCTION AND MOTIVATION

Many health studies measure a continuous outcome and compare means between groups. Since means for biological data are often difficult to interpret clinically, it is common to dichotomise using a cut-point and present the 'percentage abnormal' alongside or in place of means. Examples include birthweight where 'abnormal' is defined as <2500 g (low birthweight), systolic blood pressure with abnormal defined as >140 mm Hg (high blood pressure) and lung function with varying definitions of the 'limit of normal'. In vulnerable populations with low means, for example, birthweight in a population of preterm babies, a given difference in means between two groups will represent a larger difference in the percentage with low birthweight than in a general population of babies where most will be full term. Thus, in general, the difference in percentage of patients with abnormal values for a given difference in means varies according to the reference population's mean value. This phenomenon leads to challenges in interpreting differences in means in vulnerable populations and in defining an outcome-specific minimal clinically important difference (MCID) in means since the proportion abnormal, which is useful in interpreting means, is not constant-it varies with the population mean. This has relevance for study power calculations and data analyses in vulnerable populations where a small observed difference in means may be difficult to interpret clinically and may be disregarded, even if associated with a relatively large difference in percentage abnormal which is clinically relevant.

METHODS

To address these issues, we suggest both difference in means and difference in percentage (proportion) abnormal are considered when choosing the MCID, and that both means and percentages abnormal are reported when analysing the data.

CONCLUSIONS

We describe a distributional approach to analyse proportions classified as abnormal that avoids the usual loss of precision and power associated with dichotomisation.

Do patient characteristics matter when calculating sample size for eczema clinical trials?

Background

The Patient‐Oriented Eczema Measure (POEM) is the core outcome instrument recommended for measuring patient‐reported atopic eczema symptoms in clinical trials. To ensure that the statistical significance of clinical trial results is meaningful, trials are often designed by specifying the target difference in the primary outcome as part of the sample size calculation. One method used to specify the target difference is a score that corresponds to a standardized effect size.

Objectives

to assess how the standardized effect size of POEM scores vary across age, gender, ethnicity and disease severity.

Methods

This study combined data from five UK‐based randomized clinical trials of eczema treatments in order to assess differences in self‐reported eczema symptoms (POEM) corresponding to a standardized effect size (0.5 SD of baseline POEM scores) across age, gender, ethnicity and disease severity.

Results

POEM scores corresponding to 0.5 SD_(baseline) were remarkably consistent across participants of varying ages, gender, ethnicity and disease severity from datasets of five UK trials in children (range 2.99–3.45).

Conclusions

This study provides information that can support those designing clinical trials to determine their sample size and can aid individuals interpreting trial results. Further exploration of differences in populations beyond the United Kingdom is needed.

Predictors, Outcomes, and Statistical Solutions of Missing Cases in Web-Based Psychotherapy: Methodological Replication and Elaboration Study

BACKGROUND

Missing cases present a challenge to our ability to evaluate the effects of web-based psychotherapy trials. As missing cases are often lost to follow-up, less is known about their characteristics, their likely clinical outcomes, or the likely effect of the treatment being trialed.

OBJECTIVE

The aim of this study is to explore the characteristics of missing cases, their likely treatment outcomes, and the ability of different statistical models to approximate missing posttreatment data.

METHODS

A sample of internet-delivered cognitive behavioral therapy participants in routine care (n=6701, with 36.26% missing cases at posttreatment) was used to identify predictors of dropping out of treatment and predictors that moderated clinical outcomes, such as symptoms of psychological distress, anxiety, and depression. These variables were then incorporated into a range of statistical models that approximated replacement outcomes for missing cases, and the results were compared using sensitivity and cross-validation analyses.

RESULTS

Treatment adherence, as measured by the rate of progress of an individual through the treatment modules, and higher pretreatment symptom scores were identified as the dominant predictors of missing cases probability (Nagelkerke R²=60.8%) and the rate of symptom change. Low treatment adherence, in particular, was associated with increased odds of presenting as missing cases during posttreatment assessment (eg, odds ratio 161.1:1) and, at the same time, attenuated the rate of symptom change across anxiety (up to 28% of the total symptom with 48% reduction effect), depression (up to 41% of the total with 48% symptom reduction effect), and psychological distress symptom outcomes (up to 52% of the total with 37% symptom reduction effect) at the end of the 8-week window. Reflecting this pattern of results, statistical replacement methods that overlooked the features of treatment adherence and baseline severity underestimated missing case symptom outcomes by as much as 39% at posttreatment.

CONCLUSIONS

The treatment outcomes of the cases that were missing at posttreatment were distinct from those of the remaining observed sample. Thus, overlooking the features of missing cases is likely to result in an inaccurate estimate of the effect of treatment.

Essential statistical principles of clinical trials of pain treatments

This article presents an overview of fundamental statistical principles of clinical trials of pain treatments. Statistical considerations relevant to phase 2 proof of concept and phase 3 confirmatory randomized trials investigating efficacy and safety are discussed, including (1) research design; (2) endpoints and analyses; (3) sample size determination and statistical power; (4) missing data and trial estimands; (5) data monitoring and interim analyses; and (6) interpretation of results. Although clinical trials of pharmacologic treatments are emphasized, the key issues raised by these trials are also directly applicable to clinical trials of other types of treatments, including biologics, devices, nonpharmacologic therapies (eg, physical therapy and cognitive-behavior therapy), and complementary and integrative health interventions.

Mortality trends in sickle cell patients

BACKGROUND

Sickle cell disease affects a significant portion of US patients with African descent. It continues to be one of the leading causes of frequent hospitalizations and high in-hospital morality risk. Until the approval of disease-modifying therapies in last two years, medical therapy has relied mostly on management of pain episodes and the use of hydroxyurea. We discuss the nationwide analysis of trends in in-hospital mortality in patients with Sickle Cell Disease from 2000 to 2014.

METHODS

Trends of in-hospital mortality in sickle cell patients were analyzed from a database provided by the Agency of Healthcare Research and Quality. From the data hospitalization rates and in-hospital mortality in categories by region in the US, hospital size, health insurance status, comorbidities and gender were examined. Patterns of in-hospital mortality were analyzed by logistic regression.

RESULTS

Ratio for hospitalization and mortality among the four regions described Northeast, Midwest, South, West with respective values of 0.63%, 0.65%, 0.76% and 0.89% with P = 0.008 and OR = 1.07. Odds ratio for sickle cell patients that died during hospitalization and health insurance status was OR = 0.08. Comorbidities considered in sickle cell patients; diabetes mellitus (DM), hypertension (HTN), hyperlipidemia (HLD), chronic kidney disease (CKD), smoking status. The odds ratio for comorbidities show A-fib with a value of OR = 4.47, followed by hypertension OR = 1.92, diabetes mellitus OR = 1.44 and chronic kidney disease OR = 1.29, smoking status OR = 0.60. Mortality-hospitalization ratio by gender was: males 0.77% and females 0.69% with OR = 0.87.

CONCLUSIONS

In-hospital mortality by US regions, as well as health insurance status are important measurable elements that show the impact of the disease from a public health perspective. Further and more specific data of regions by states, comorbidities by states and sex, as well as health insurance status by states will provide further insight in local mortality trends.

The effect of core stability training with ball and balloon exercise on respiratory variables in chronic non-specific low back pain: An experimental study

BACKGROUND

Studies have shown the involvement of respiratory characteristics and their relationship with impairments in non-specific low back pain (NS-LBP). The effects of core stability with a combined ball and balloon exercise (CBB) on respiratory variables had not been investigated.

OBJECTIVE

To evaluate the effectiveness of CBB on respiratory variables among NS-LBP patients.

STUDY DESIGN

pre- and post-experimental study.

PARTICIPANTS

Forty participants were assigned to an experimental group (EG) [n = 20] and control group (CG) [n = 20] based on the study criteria.

INTERVENTIONS

The EG received CBB together with routine physiotherapy and the CG received routine physiotherapy over a period of 8 weeks. Participants were instructed to carry out the exercises for 3 days per week. The training was evaluated once a week and the exercises progressed based on the level of pain.

OUTCOME MEASURES

Primary outcomes were maximum inspiratory pressure (MIP), maximum expiratory pressure (MEP) and maximum voluntary ventilation (MVV). The secondary outcomes were measured in the numeric rating scale (NRS), total faulty breathing scale (TFBS), cloth tape measure (CTM) and lumbo-pelvic stability.

RESULTS

The MIP increased significantly among the EG when compared with that in the CG (p > 0.05).The EG showed a significant increase in MVV (p = 0.04) when compared to the CG (p = 0.0001). There was a significant reduction in pain for both groups. The MEP, TFBS, chest expansion and core stability showed no changes in either group.

CONCLUSION

CBB was effective in improving respiratory variables among NS-LBP patients.

Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)

Background: Indicated primary prevention in young people at Clinical High Risk for Psychosis (CHR-P) is a promising avenue for improving outcomes of one of the most severe mental disorders but their effectiveness has recently been questioned. Methods: Umbrella review. A multi-step independent literature search of Web of Science until January 11, 2019, identified interventional meta-analyses in CHR-P individuals. The individual randomised controlled trials that were analysed by the meta-analyses were extracted. A review of ongoing trials and a simulation of living meta-analysis complemented the analysis. Results: Seven meta-analyses investigating preventive treatments in CHR-P individuals were included. None of them produced pooled effect sizes across psychological, pharmacological, or other types of interventions. The outcomes analysed encompassed risk of psychosis onset, the acceptability of treatments, the severity of attenuated positive/negative psychotic symptoms, depression, symptom-related distress, social functioning, general functioning, and quality of life. These meta-analyses were based on 20 randomised controlled trials: the vast majority defined the prevention of psychosis onset as their primary outcome of interest and only powered to large effect sizes. There was no evidence to favour any preventive intervention over any other (or control condition) for improving any of these clinical outcomes. Caution is required when making clinical recommendations for the prevention of psychosis in individuals at risk. Discussion: Prevention of psychosis from a CHR-P state has been, and should remain, the primary outcome of interventional research, refined and complemented by other clinically meaningful outcomes. Stagnation of knowledge should promote innovative and collaborative research efforts, in line with the progressive and incremental nature of medical knowledge. Advancements will most likely be associated with the development of new experimental therapeutics that are ongoing along with the ability to deconstruct the high heterogeneity within CHR-P populations. This would require the estimation of treatment-specific effect sizes through living individual participant data meta-analyses, controlling risk enrichment during recruitment, statistical power, and embedding precision medicine within youth mental health services that can accommodate sequential prognosis and advanced trial designs. Conclusions: The evidence-based challenges and proposed solutions addressed by this umbrella review can inform the next generation of research into preventive treatments for psychosis.

Using Time Trade-Off Methods to Elicit Short-Term Utilities Associated with Treatments for Bulbar Urethral Stricture

BACKGROUND

Recurrent urethral stricture is usually treated with either open urethroplasty or endoscopic urethrotomy. Both of the procedures cause short-term utility loss, which may not be captured by standard utility questionnaires due to the challenges of completing a standard instrument at the time of an acute episode of short duration, especially within a clinical trial setting. We propose to use time trade-off (TTO) methods to estimate these short-term utility losses.

OBJECTIVE

The aim was to compare the use of two alternative TTO methods to elicit patients' short-term utilities following surgical treatments for recurrent urethral stricture.

METHOD

Two variants of TTO (chained and conventional) were used. Six health profiles were developed-three for each procedure. Forty participants took part, with 20 randomly allocated to each TTO method.

RESULTS

Thirty-eight participants provided usable data for analysis. Estimated utility values decreased as the severity of the health profiles increased. There was no evidence that utility values differed between elicitation methods or procedures for mild {ranging from 0.79 (standard deviation [SD] 0.17) to 0.83 [SD 0.20]} and moderate (ranging from 0.54 [SD 0.24] to 0.67 [SD 0.21]) health states, although they appeared to differ for severe health states (ranging from 0.29 [SD 0.20] to 0.56 [SD 0.24]).

CONCLUSION

The study demonstrates the feasibility and value of eliciting patients' short-term utilities. Given the small sample size, the study findings are tentative. Further research with a larger sample size is needed to determine the appropriate TTO method to use and how the elicited utilities can be used in combination with standard cost-utility assessments to aid decision making.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Internet delivery of intensive speech and language therapy for children with cerebral palsy: a pilot randomised controlled trial

OBJECTIVES

To test the feasibility of recruitment, retention, outcome measures and internet delivery of dysarthria therapy for young people with cerebral palsy in a randomised controlled trial.

DESIGN

Mixed methods. Single blind pilot randomised controlled trial, with control offered Skype therapy at end of study. Qualitative study of the acceptability of therapy delivery via Skype.

SETTING

Nine speech and language therapy departments in northern England recruited participants to the study. Skype therapy was provided in a university setting.

PARTICIPANTS

Twenty-two children (14 M, 8 F) with dysarthria and cerebral palsy (mean age 8.8 years (SD 3.2)) agreed to take part. Participants were randomised to dysarthria therapy via Skype (n=11) or treatment as usual (n=11).

INTERVENTIONS

Children received either usual speech therapy from their local therapist for 6 weeks or dysarthria therapy via Skype from a research therapist. Usual therapy sessions varied in frequency, duration and content. Skype dysarthria therapy focused on breath control and phonation to produce clear speech at a steady rate, and comprised three 40 min sessions per week for 6 weeks.

PRIMARY AND SECONDARY OUTCOME MEASURES

Feasibility and acceptability of the trial design, intervention and outcome measures.

RESULTS

Departments recruited two to three participants. All participants agreed to random allocation. None withdrew from the study. Recordings of children's speech were made at all time points and rated by listeners. Families allocated to Skype dysarthria therapy judged internet delivery of the therapy to be acceptable. All families reported that the study design was acceptable. Treatment integrity checks suggested that the phrases practised in one therapy exercise should be reduced in length.

CONCLUSIONS

A delayed treatment design, in which dysarthria therapy is offered at the end of the study to families allocated to treatment as usual, is acceptable. A randomised controlled trial of internet delivered dysarthria therapy is feasible.

DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial

Background

A key step in the design of a RCT is the estimation of the number of participants needed in the study. The most common approach is to specify a target difference between the treatments for the primary outcome and then calculate the required sample size. The sample size is chosen to ensure that the trial will have a high probability (adequate statistical power) of detecting a target difference between the treatments should one exist.

The sample size has many implications for the conduct and interpretation of the study. Despite the critical role that the target difference has in the design of a RCT, the way in which it is determined has received little attention. In this article, we summarise the key considerations and messages from new guidance for researchers and funders on specifying the target difference, and undertaking and reporting a RCT sample size calculation. This article on choosing the target difference for a randomised controlled trial (RCT) and undertaking and reporting the sample size calculation has been dual published in the BMJ and BMC Trials journals

Methods

The DELTA² (Difference ELicitation in TriAls) project comprised five major components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a two-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of a guidance document (stage 5).

Results and Discussion

The key messages from the DELTA² guidance on determining the target difference and sample size calculation for a randomised caontrolled trial are presented. Recommendations for the subsequent reporting of the sample size calculation are also provided.

New guidance to improve sample size calculations for trials: eliciting the target difference

BACKGROUND

Sample size calculations are central to the design of health research trials. To ensure that the trial provides good evidence to answer the trial's research question, the target effect size (difference in means or proportions, odds ratio, relative risk or hazard ratio between trial arms) must be specified under the conventional approach to determining the sample size. However, until now, there has not been comprehensive guidance on how to specify this effect.

MAIN TEXT

This is a commentary on a collection of papers from two important projects, DELTA (Difference ELicitation in TriAls) and DELTA2 that aim to provide evidence-based guidance on systematically determining the target effect size, or difference and the resultant sample sizes for trials. In addition to surveying methods that researchers are using in practice, the research team met with various experts (statisticians, methodologists, clinicians and funders); reviewed guidelines from funding agencies; and reviewed recent methodological literature. The DELTA2 guidance stresses specifying important and realistic differences, and undertaking sensitivity analyses in calculating sample sizes. It gives recommendations on how to find appropriate differences, conduct the sample size calculation(s) and how to report these in grant applications, protocols and manuscripts. It is hoped that this will contribute not only to better powered studies, but better reporting and reproducibility and thinking about what a trial should be designed to achieve.

CONCLUSIONS

The DELTA researchers have developed a set of comprehensive guidance documents that are welcome and will almost certainly improve the way that trials are designed and reported.

DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial

BACKGROUND

A key step in the design of a RCT is the estimation of the number of participants needed in the study. The most common approach is to specify a target difference between the treatments for the primary outcome and then calculate the required sample size. The sample size is chosen to ensure that the trial will have a high probability (adequate statistical power) of detecting a target difference between the treatments should one exist. The sample size has many implications for the conduct and interpretation of the study. Despite the critical role that the target difference has in the design of a RCT, the way in which it is determined has received little attention. In this article, we summarise the key considerations and messages from new guidance for researchers and funders on specifying the target difference, and undertaking and reporting a RCT sample size calculation. This article on choosing the target difference for a randomised controlled trial (RCT) and undertaking and reporting the sample size calculation has been dual published in the BMJ and BMC Trials journals METHODS: The DELTA² (Difference ELicitation in TriAls) project comprised five major components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a two-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of a guidance document (stage 5).

RESULTS AND DISCUSSION

The key messages from the DELTA² guidance on determining the target difference and sample size calculation for a randomised caontrolled trial are presented. Recommendations for the subsequent reporting of the sample size calculation are also provided.

Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA2 guidance

BACKGROUND

A key step in the design of a randomised controlled trial is the estimation of the number of participants needed. The most common approach is to specify a target difference in the primary outcome between the randomised groups and then estimate the corresponding sample size. The sample size is chosen to provide reassurance that the trial will have high statistical power to detect the target difference at the planned statistical significance level. Alternative approaches are also available, though most still require specification of a target difference. The sample size has many implications for the conduct of the study, as well as incurring scientific and ethical aspects. Despite the critical role of the target difference for the primary outcome in the design of a randomised controlled trial (RCT), the manner in which it is determined has received little attention. This article reports the development of the DELTA² guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for a RCT.

METHODS

The DELTA² (Difference ELicitation in TriAls) project has five components comprising systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2), a Delphi study (stage 3), a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4), and the preparation and dissemination of a guidance document (stage 5).

RESULTS

The project started in April 2016. The literature search identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving 69 participants, along with a 2-day consensus meeting were conducted. In addition, further engagement sessions were held at two international conferences. The main guidance text was finalised on April 18, 2018, after revision informed by feedback gathered from stages 2 and 3 and from funder representatives.

DISCUSSION

The DELTA² Delphi study identified a number of areas (such as practical recommendations and examples, greater coverage of different trial designs and statistical approaches) of particular interest amongst stakeholders which new guidance was desired to meet. New relevant references were identified by the review. Such findings influenced the scope, drafting and revision of the guidance. While not all suggestions could be accommodated, it is hoped that the process has led to a more useful and practical document.

A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal

BACKGROUND

When designing a randomised controlled trial (RCT), an important consideration is the sample size required. This is calculated from several components; one of which is the target difference. This study aims to review the currently reported methods of elicitation of the target difference as well as to quantify the target differences used in Health Technology Assessment (HTA)-funded trials.

METHODS

Trials were identified from the National Institute of Health Research Health Technology Assessment journal. A total of 177 RCTs published between 2006 and 2016 were assessed for eligibility. Eligibility was established by the design of the trial and the quality of data available. The trial designs were parallel-group, superiority RCTs with a continuous primary endpoint. Data were extracted and the standardised anticipated and observed effect size estimates were calculated. Exclusion criteria was based on trials not providing enough detail in the sample size calculation and results, and trials not being of parallel-group, superiority design.

RESULTS

A total of 107 RCTs were included in the study from 102 reports. The most commonly reported method for effect size derivation was a review of evidence and use of previous research (52.3%). This was common across all clinical areas. The median standardised target effect size was 0.30 (interquartile range: 0.20-0.38), with the median standardised observed effect size 0.11 (IQR 0.05-0.29). The maximum anticipated and observed effect sizes were 0.76 and 1.18, respectively. Only two trials had anticipated target values above 0.60.

CONCLUSION

The most commonly reported method of elicitation of the target effect size is previous published research. The average target effect size was 0.3. A clear distinction between the target difference and the minimum clinically important difference is recommended when designing a trial. Transparent explanation of target difference elicitation is advised, with multiple methods including a review of evidence and opinion-seeking advised as the more optimal methods for effect size quantification.

Clinical significance in pediatric oncology randomized controlled treatment trials: a systematic review

Background

Clinical significance in a randomized controlled trial (RCT) can be determined using the minimal clinically important difference (MCID), which should inform the delta value used to determine sample size. The primary objective was to assess clinical significance in the pediatric oncology randomized controlled trial (RCT) treatment literature by evaluating: (1) the relationship between the treatment effect and the delta value as reported in the sample size calculation, and (2) the concordance between statistical and clinical significance. The secondary objective was to evaluate the reporting of methodological attributes related to clinical significance.

Methods

RCTs of pediatric cancer treatments, where a sample size calculation with a delta value was reported or could be calculated, were systematically reviewed. MEDLINE, EMBASE, and the Cochrane Childhood Cancer Group Specialized Register through CENTRAL were searched from inception to July 2016.

Results

RCTs (77 overall; 11 and 66), representing 95 (13 and 82) randomized questions were included for non-inferiority and superiority RCTs (herein, respectively). The minority (22.1% overall; 76.9 and 13.4%) of randomized questions reported conclusions based on clinical significance, and only 4.2% (15.4 and 2.4%) explicitly based the delta value on the MCID. Over half (67.4% overall; 92.3 and 63.4%) reported a confidence interval or standard error for the primary outcome experimental and control values and 12.6% (46.2 and 7.3%) reported the treatment effect, respectively. Of the 47 randomized questions in superiority trials that reported statistically non-significant findings, 25.5% were possibly clinically significant. Of the 24 randomized questions in superiority trials that were statistically significant, only 8.3% were definitely clinically significant.

Conclusions

A minority of RCTs in the pediatric oncology literature reported methodological attributes related to clinical significance and a notable portion of statistically insignificant studies were possibly clinically significance.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2925-8) contains supplementary material, which is available to authorized users.

Minimal important change values for the Oxford Knee Score and the Forgotten Joint Score at 1 year after total knee replacement

Background and purpose - Interpreting changes in Oxford Knee Score (OKS) and Forgotten Joint Score (FJS) following total knee replacement (TKR) is challenged by the lack of methodologically rigorous methods to estimate minimal important change (MIC) values. We determined MIC values by predictive modeling for the OKS and FJS in patients undergoing primary TKR. Patients and methods - We conducted a prospective cohort study in patients undergoing TKR between January 2015 and July 2016. OKS and FJS were completed preoperatively and at 1 year postoperatively, accompanied by a 7-point anchor question ranging from "better, an important improvement" to "worse, an important worsening." MIC improvement values were defined with the predictive modeling approach based on logistic regression, with patients' decisions on important improvement as dependent variable and change in OKS/FJS as independent variable. Furthermore, the MICs were adjusted for high proportions of improved patients. Results - 333/496 (67.1%) patients with a median age of 69 years (61% female) had complete data for OKS, FJS, and anchor questions at 1 year postoperatively. 85% were importantly improved. Spearman's correlations between the anchor and the change score were 0.56 for OKS, and 0.61 for FJS. Adjusted predictive MIC values (95% CI) for improvement were 8 (6-9) for OKS and 14 (10-18) for FJS. Interpretation - The MIC value of 8 for OKS and 14 for FJS corresponds to minimal improvements that the average patient finds important and aids in our understanding of whether improvements after TKR are clinically relevant.

Effect of a combined brief narrative exposure therapy with case management versus treatment as usual in primary care for patients with traumatic stress sequelae following intensive care medicine: study protocol for a multicenter randomized controlled trial (PICTURE)

BACKGROUND

Traumatic events like critical illness and intensive care are threats to life and bodily integrity and pose a risk factor for posttraumatic stress disorder (PTSD). PTSD affects the quality of life and morbidity and may increase health-care costs. Limited access to specialist care results in PTSD patients being treated in primary care settings. Narrative exposure therapy (NET) is based on the principles of cognitive behavioral therapy and has shown positive effects when delivered by health-care professionals other than psychologists. The primary aims of the PICTURE trial (from "PTSD after ICU survival") are to investigate the effectiveness and applicability of NET adapted for primary care with case management in adults diagnosed with PTSD after intensive care.

METHODS/DESIGN

This is an investigator-initiated, multi-center, primary care-based, randomized controlled two-arm parallel group, observer-blinded superiority trial conducted throughout Germany. In total, 340 adult patients with a total score of at least 20 points on the posttraumatic diagnostic scale (PDS-5) 3 months after receiving intensive care treatment will be equally randomized to two groups: NET combined with case management and improved treatment as usual (iTAU). All primary care physicians (PCPs) involved will be instructed in the diagnosis and treatment of PTSD according to current German guidelines. PCPs in the iTAU group will deliver usual care during three consultations. In the experimental group, PCPs will additionally be trained to deliver an adapted version of NET (three sessions) supported by phone-based case management by a medical assistant. At 6 and 12 months after randomization, structured blinded telephone interviews will assess patient-reported outcomes. The primary composite endpoint is the absolute change from baseline at month 6 in PTSD symptom severity measured by the PDS-5 total score, which also incorporates the death of any study patients. Secondary outcomes cover the domains depression, anxiety, disability, health-related quality-of-life, and cost-effectiveness. The principal analysis is by intention to treat.

DISCUSSION

If the superiority of the experimental intervention over usual care can be demonstrated, the combination of brief NET and case management could be a treatment option to relieve PTSD-related symptoms and to improve primary care after intensive care.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03315390 . Registered on 10 October 2017. German Clinical Trials Register, DRKS00012589 . Registered on 17 October 2017.

Contributions of a survey and retrospective cohort study to the planning of a randomised controlled trial of corticosteroids in the treatment of paediatric septic shock

Background

Randomised controlled trials (RCTs) are challenging to conduct in a paediatric critical-care environment. Background work, including surveys and observational studies, is often used to determine disease estimates, sample sizes and design protocols when planning such RCTs. Our objective was to determine the necessity of performing a survey or a retrospective chart review or both when planning an RCT on corticosteroids in the treatment of paediatric septic shock.

Methods

We compared information on corticosteroid use for moderate to severe paediatric septic shock obtained from a survey of physician beliefs and stated practices with that obtained from a retrospective cohort study. The survey was conducted between February and March 2012 and the retrospective study included children from birth to 17 years of age admitted from January 2010 to June 2011. The survey and the retrospective study were conducted at four academic tertiary care centres in Canada.

Results

Survey responses from 23 physicians and retrospective data from 81 septic shock patients were included. The survey identified time to discontinuation of vasoactive infusions as the most feasible and clinically important outcome for an RCT on corticosteroids for paediatric septic shock. The retrospective chart review provided means and standard deviations for the suggested primary outcome, from which we could estimate sample sizes and justify the minimal clinically important difference. The survey found that physicians believe that patients with severe septic shock were most likely to benefit from corticosteroid administration but the majority stated they would be unwilling to randomise such patients, suggesting a lack of individual physician equipoise. The combined information from the survey and retrospective study suggested that enrolment of patients with moderate septic shock would be more feasible but that strategies would still have to be implemented to prevent open-label corticosteroid use.

Conclusions

The survey provided valuable information on the choice of primary outcome, target population and physician equipoise. The retrospective study provided estimates of patient numbers, the minimal clinically important difference, evidence for community equipoise and physician practice patterns. Strong consideration should be given to performing both types of studies prior to conducting RCTs in paediatric critical-care environments.

Research methodology for orthopaedic surgeons, with a focus on outcome

Since improving the patient's condition is the ultimate goal of clinical care and research, this review of research methodology focuses on outcomes in the musculoskeletal field.This paper provides an overview of conceptual models, different types of outcomes and commonly assessed outcomes in orthopaedics as well as epidemiological and statistical aspects of outcomes determination, measurement and interpretation.Clinicians should determine the outcome(s) most important to patients and/or public health in collaboration with the patients, epidemiologists/statisticians and other stakeholders.Key points in outcome choice are to evaluate both the benefit and harm of a health intervention, and to consider short- and longer-term outcomes including patient-reported outcomes.Outcome estimation should aim at identifying a clinically important difference (not the same as a statistically significant difference), at presenting measures of effects with confidence intervals and at taking the necessary steps to minimize bias. Cite this article: EFORT Open Rev 2018;3 DOI: 10.1302/2058-5241.3.170064.

How should minimally important change scores for the Patient-Oriented Eczema Measure be interpreted? A validation using varied methods

Background

The Patient‐Oriented Eczema Measure (POEM), scored 0–28, is the core outcome instrument recommended for measuring patient‐reported atopic eczema symptoms in clinical trials. To date, two published studies have broadly concurred that the minimally important change (MIC) of the POEM is three points. Further assessment of the MIC of POEM in different populations, and using a variety of methods, will improve interpretability of the POEM in research and clinical practice.

Objectives

To calculate the smallest detectable change in the POEM and estimate the MIC of the POEM using a variety of methods in a trial dataset of children with moderate‐to‐severe atopic eczema.

Methods

This study used distribution‐based and anchor‐based methods to calculate the MIC of the POEM in children with moderate‐to‐severe eczema.

Results

Data were collected from 300 children. The smallest detectable change was 2·13. The MIC estimates were 1·07 (using 0·2 SD of baseline POEM scores) and 2·68 (using 0·5 SD of baseline POEM scores) based on distribution‐based methods; were 3·09–6·13 based on patient‐/parent‐reported anchor‐based methods; and were 3·23–5·38 based on investigator‐reported anchor‐based methods.

Conclusions

We recommend the following thresholds be used to interpret changes in POEM scores: ≤ 2, unlikely to be a change beyond measurement error; 2·1–2·9, a small change detected that is likely to be beyond measurement error but may not be clinically important; 3–3·9, probably a clinically important change; ≥ 4, very likely to be a clinically important change.

What's already known about this topic?

The Patient‐Oriented Eczema Measure (POEM) is recommended as the core outcome instrument for measuring patient‐reported symptoms in eczema clinical trials.

Two previous studies have examined the minimally important change (MIC) of the POEM; one in children with mild eczema and another in adults with very severe eczema.

These previous studies both concluded that the MIC in POEM is around three points.

What does this study add?

This study explored the impact of different methodologies for calculating the MIC of the POEM in children with moderate‐to‐severe eczema.

A change in POEM of less than two points is likely to be below the smallest detectable change (i.e. below measurement error) for the scale.

The MIC varied considerably depending on the method used, but a change in POEM score below three points is unlikely to be a clinically important change.

What are the clinical implications of this work?

This study aids sample size calculations for clinical trials and helps researchers, clinicians and patients to interpret changes in POEM scores in clinical trials and routine monitoring of eczema in clinical practice.

https://doi.org/10.1111/bjd.16611 available online

https://goo.gl/Uqv3dl

Invasive urodynamic testing prior to surgical treatment for stress urinary incontinence in women: cost-effectiveness and value of information analyses in the context of a mixed methods feasibility study

BACKGROUND

INVESTIGATE-I (INVasive Evaluation before Surgical Treatment of Incontinence Gives Added Therapeutic Effect?) was a mixed methods study to assess the feasibility of a future randomised controlled trial of invasive urodynamic testing (IUT) prior to surgery for stress urinary incontinence (SUI) in women. Here we report one of the study's five components, with the specific objectives of (i) exploring the cost-effectiveness of IUT compared with clinical assessment plus non-invasive tests (henceforth described as 'IUT' and 'no IUT' respectively) in women with SUI or stress-predominant mixed urinary incontinence (MUI) prior to surgery, and (ii) determining the expected net gain (ENG) from additional research.

METHODS

Study participants were women with SUI or stress-predominant MUI who had failed to respond to conservative treatments recruited from seven UK urogynaecology and female urology units. They were randomised to receive either 'IUT' or 'no IUT' before undergoing further treatment. Data from 218 women were used in the economic analysis. Cost utility, net benefit and value of information (VoI) analyses were performed within a randomised controlled pilot trial. Costs and quality-adjusted life years (QALYs) were estimated over 6 months to determine the incremental cost per QALY of 'IUT' compared to 'no IUT'. Net monetary benefit informed the VoI analysis. The VoI estimated the ENG and optimal sample size for a future definitive trial.

RESULTS

At 6 months, the mean difference in total average cost was £138 (p = 0.071) in favour of 'IUT'; there was no difference in QALYs estimated from the SF-12 (difference 0.004; p = 0.425) and EQ-5D-3L (difference - 0.004; p = 0.725); therefore, the probability of IUT being cost-effective remains uncertain. The estimated ENG was positive for further research to address this uncertainty with an optimal sample size of 404 women.

CONCLUSIONS

This is the largest economic evaluation of IUT. On average, up to 6 months after treatment, 'IUT' may be cost-saving compared to 'no IUT' because of the reduction in surgery following invasive investigation. However, uncertainty remains over the probability of 'IUT' being considered cost-effective, especially in the longer term. The VoI analysis indicated that further research would be of value.

TRIAL REGISTRATION

ISRCTN. ISRCTN71327395. Registered 7 June 2010.

Ethical implications of excessive cluster sizes in cluster randomised trials

The cluster randomised trial (CRT) is commonly used in healthcare research. It is the gold-standard study design for evaluating healthcare policy interventions. A key characteristic of this design is that as more participants are included, in a fixed number of clusters, the increase in achievable power will level off. CRTs with cluster sizes that exceed the point of levelling-off will have excessive numbers of participants, even if they do not achieve nominal levels of power. Excessively large cluster sizes may have ethical implications due to exposing trial participants unnecessarily to the burdens of both participating in the trial and the potential risks of harm associated with the intervention. We explore these issues through the use of two case studies. Where data are routinely collected, available at minimum cost and the intervention poses low risk, the ethical implications of excessively large cluster sizes are likely to be low (case study 1). However, to maximise the social benefit of the study, identification of excessive cluster sizes can allow for prespecified and fully powered secondary analyses. In the second case study, while there is no burden through trial participation (because the outcome data are routinely collected and non-identifiable), the intervention might be considered to pose some indirect risk to patients and risks to the healthcare workers. In this case study it is therefore important that the inclusion of excessively large cluster sizes is justifiable on other grounds (perhaps to show sustainability). In any randomised controlled trial, including evaluations of health policy interventions, it is important to minimise the burdens and risks to participants. Funders, researchers and research ethics committees should be aware of the ethical issues of excessively large cluster sizes in cluster trials.

Animal models of cardiac arrest: A systematic review of bias and reporting

AIM OF THE REVIEW

Animal models are essential in advancing resuscitation research but are susceptible to various biases compromising internal validity, which may explain unsuccessful transition to human clinical trials. This study aimed to assess risk of bias in animal studies of cardiac arrest.

DATA SOURCES

This study was based on a previous systematic review of all animal cardiac arrest studies published between March 8, 2011 and March 8, 2016 in PubMed and EMBASE. For this study, we focused on interventional studies and selected a random sample of 50 pig and 50 rat studies. We used a modified version of the SYRCLE's risk of bias tool for animal studies. Bias assessment was performed by two independent reviewers.

RESULTS

92% of pig studies and 88% of rat studies used randomization to assign interventions, but the methodology was unknown or insufficiently reported in 60% and 68% of the studies, respectively. Correct timing of randomization was lacking or unclear in over half of the studies. 40% of pig studies and 28% of rat studies reported insufficient baseline characteristics. When possible, blinding was not performed/reported in 68% of rat studies and 31% of pig studies. Blinding of outcome assessors was missing or inadequately reported in 65% of pig studies and 60% of rat studies. 80% of all studies lacked a sample size calculation, while 60% of pig and 80% of rat studies omitted a specified primary outcome.

CONCLUSION

This study indicates insufficient reporting and methodological shortcomings in animal models of cardiac arrest.

50 Tips for Clinical Trialists

This paper presents ‘Tips’ for researchers of brain impairment who are interested in conducting randomised controlled trials. The paper is intended for researchers who are planning to undertake their first trial, but may also be of interest to more experienced trialists or clinicians who want to further their understandings of clinical trials. The Tips include suggestions for how to design, conduct, analyse and report clinical trials.

The split-plot design was useful for evaluating complex, multilevel interventions, but there is need for improvement in its design and report

OBJECTIVES

To describe the sample size calculation, analysis and reporting of split-plot (S-P) randomized controlled trials in health care (trials that use two units of randomization: one at a cluster-level and one at a level lower than the cluster).

STUDY DESIGN AND SETTING

We carried out a comprehensive search in the EMBASE database from 1946 to 2016. Health care trials with a S-P design in human subjects were included. Three authors screened and assessed the studies, and the data were extracted on methodology and reporting standards based on CONSORT.

RESULTS

Eighteen S-P studies were included, with authors using nine different designations to describe them. Units of randomization were unclear in nine abstracts. Explicit rationale for choosing the design was not given. Ten studies presented a sample size calculation accounting for clustering; the analyses were coherent with that. Flow of participant diagrams was presented but was incomplete in 14 articles.

CONCLUSION

S-P designs can be useful complex designs but challenging to report. Researchers need to clearly describe the rationale, sample size calculation, and participant flow. We provide a suggested CONSORT style participant flow diagram to aid reporting. There is need for more research regarding sample size calculation for S-P.

Current practice in methodology and reporting of the sample size calculation in randomised trials of hip and knee osteoarthritis: a protocol for a systematic review

Background

A key aspect of the design of randomised controlled trials (RCTs) is determining the sample size. It is important that the trial sample size is appropriately calculated. The required sample size will differ by clinical area, for instance, due to the prevalence of the condition and the choice of primary outcome. Additionally, it will depend upon the choice of target difference assumed in the calculation. Focussing upon the hip and knee osteoarthritis population, this study aims to systematically review how the trial size was determined for trials of osteoarthritis, on what basis, and how well these aspects are reported.

Methods

Several electronic databases (Medline, Cochrane library, CINAHL, EMBASE, PsycINFO, PEDro and AMED) will be searched to identify articles on RCTs of hip and knee osteoarthritis published in 2016. Articles will be screened for eligibility and data extracted independently by two reviewers. Data will be extracted on study characteristics (design, population, intervention and control treatments), primary outcome, chosen sample size and justification, parameters used to calculate the sample size (including treatment effect in control arm, level of variability in primary outcome, loss to follow-up rates). Data will be summarised across the studies using appropriate summary statistics (e.g. n and %, median and interquartile range). The proportion of studies which report each key component of the sample size calculation will be presented. The reproducibility of the sample size calculation will be tested.

Discussion

The findings of this systematic review will summarise the current practice for sample size calculation in trials of hip and knee osteoarthritis. It will also provide evidence on the completeness of the reporting of the sample size calculation, reproducibility of the chosen sample size and the basis for the values used in the calculation.

Trial registration

As this review was not eligible to be registered on PROSPERO, the summary information was uploaded to Figshare to make it publicly accessible in order to avoid unnecessary duplication amongst other benefits (https://doi.org/10.6084/m9.figshare.5009027.v1); Registered January 17, 2017.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2209-8) contains supplementary material, which is available to authorized users.

Living systematic reviews: 3. Statistical methods for updating meta-analyses

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Choosing the target difference ('effect size') for a randomised controlled trial - DELTA2 guidance protocol

BACKGROUND

A key step in the design of a randomised controlled trial (RCT) is the estimation of the number of participants needed. By far the most common approach is to specify a target difference and then estimate the corresponding sample size; this sample size is chosen to provide reassurance that the trial will have high statistical power to detect such a difference between the randomised groups (at the planned statistical significance level). The sample size has many implications for the conduct of the study, as well as carrying scientific and ethical aspects to its choice. Despite the critical role of the target difference for the primary outcome in the design of an RCT, the manner in which it is determined has received little attention. This article reports the protocol of the Difference ELicitation in TriAls (DELTA2) project, which will produce guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for RCTs.

METHODS/DESIGN

The DELTA2 project has five components: systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2); a Delphi study (stage 3); a 2-day consensus meeting bringing together researchers, funders and patient representatives, as well as one-off engagement sessions at relevant stakeholder meetings (stage 4); and the preparation and dissemination of a guidance document (stage 5).

DISCUSSION

Specification of the target difference for the primary outcome is a key component of the design of an RCT. There is a need for better guidance for researchers and funders regarding specification and reporting of this aspect of trial design. The aim of this project is to produce consensus based guidance for researchers and funders.

How do you design randomised trials for smaller populations? A framework

How should we approach trial design when we can get some, but not all, of the way to the numbers required for a randomised phase III trial?We present an ordered framework for designing randomised trials to address the problem when the ideal sample size is considered larger than the number of participants that can be recruited in a reasonable time frame. Staying with the frequentist approach that is well accepted and understood in large trials, we propose a framework that includes small alterations to the design parameters. These aim to increase the numbers achievable and also potentially reduce the sample size target. The first step should always be to attempt to extend collaborations, consider broadening eligibility criteria and increase the accrual time or follow-up time. The second set of ordered considerations are the choice of research arm, outcome measures, power and target effect. If the revised design is still not feasible, in the third step we propose moving from two- to one-sided significance tests, changing the type I error rate, using covariate information at the design stage, re-randomising patients and borrowing external information.We discuss the benefits of some of these possible changes and warn against others. We illustrate, with a worked example based on the Euramos-1 trial, the application of this framework in designing a trial that is feasible, while still providing a good evidence base to evaluate a research treatment.This framework would allow appropriate evaluation of treatments when large-scale phase III trials are not possible, but where the need for high-quality randomised data is as pressing as it is for common diseases.

Potential effectiveness of copper surfaces in reducing health care-associated infection rates in a pediatric intensive and intermediate care unit: A nonrandomized controlled trial

BACKGROUND

Studies have consistently shown that copper alloyed surfaces decrease the burden of microorganisms in health care environments. This study assessed whether copper alloy surfaces decreased hospital-associated infections in pediatric intensive and intermediate care units.

METHODS

Admitted infants were assigned sequentially to a room furnished with or without a limited number of copper alloyed surfaces. Clinical and exposure to intervention data were collected on a daily basis. To avoid counting infections present prior to admission, patients who stayed in the hospital <72 hours were excluded from analysis. Health care-associated infections (HAIs) were confirmed according to protocol definitions.

RESULTS

Clinical outcomes from 515 patients were considered in our analysis: 261 patients from the intervention arm of the study, and 254 from the control arm. Crude analysis showed an HAI rate of 10.6 versus 13.0 per 1,000 patient days for copper- and non-copper-exposed patients, respectively, for a crude relative risk reduction (RRR) of 0.19 (90% confidence interval, 0.46 to -0.22). Conducting clinical trials to assess interventions that may impact HAI rates is very challenging. The results here contribute to our understanding and ability to estimate the effect size that copper alloy surfaces have on HAI acquisition.

CONCLUSIONS

Exposure of pediatric patients to copper-surfaced objects in the closed environment of the intensive care unit resulted in decreased HAI rates when compared with noncopper exposure; however, the RRR was not statistically significant. The clinical effect size warrants further consideration of this intervention as a component of a systems-based approach to control HAIs.