Reporting of randomized factorial trials was frequently inadequate

When should factorial designs be used for late-phase randomised controlled trials?

BACKGROUND

A 2×2 factorial design evaluates two interventions (A versus control and B versus control) by randomising to control, A-only, B-only or both A and B together. Extended factorial designs are also possible (e.g. 3×3 or 2×2×2). Factorial designs often require fewer resources and participants than alternative randomised controlled trials, but they are not widely used. We identified several issues that investigators considering this design need to address, before they use it in a late-phase setting.

METHODS

We surveyed journal articles published in 2000-2022 relating to designing factorial randomised controlled trials. We identified issues to consider based on these and our personal experiences.

RESULTS

We identified clinical, practical, statistical and external issues that make factorial randomised controlled trials more desirable. Clinical issues are (1) interventions can be easily co-administered; (2) risk of safety issues from co-administration above individual risks of the separate interventions is low; (3) safety or efficacy data are wanted on the combination intervention; (4) potential for interaction (e.g. effect of A differing when B administered) is low; (5) it is important to compare interventions with other interventions balanced, rather than allowing randomised interventions to affect the choice of other interventions; (6) eligibility criteria for different interventions are similar. Practical issues are (7) recruitment is not harmed by testing many interventions; (8) each intervention and associated toxicities is unlikely to reduce either adherence to the other intervention or overall follow-up; (9) blinding is easy to implement or not required. Statistical issues are (10) a suitable scale of analysis can be identified; (11) adjustment for multiplicity is not required; (12) early stopping for efficacy or lack of benefit can be done effectively. External issues are (13) adequate funding is available and (14) the trial is not intended for licensing purposes. An overarching issue (15) is that factorial design should give a lower sample size requirement than alternative designs. Across designs with varying non-adherence, retention, intervention effects and interaction effects, 2×2 factorial designs require lower sample size than a three-arm alternative when one intervention effect is reduced by no more than 24%-48% in the presence of the other intervention compared with in the absence of the other intervention.

CONCLUSIONS

Factorial designs are not widely used and should be considered more often using our issues to consider. Low potential for at most small to modest interaction is key, for example, where the interventions have different mechanisms of action or target different aspects of the disease being studied.

Distributive randomization: a pragmatic fractional factorial design to screen or evaluate multiple simultaneous interventions in a clinical trial

BACKGROUND

In some medical indications, numerous interventions have a weak presumption of efficacy, but a good track record or presumption of safety. This makes it feasible to evaluate them simultaneously. This study evaluates a pragmatic fractional factorial trial design that randomly allocates a pre-specified number of interventions to each participant, and statistically tests main intervention effects. We compare it to factorial trials, parallel-arm trials and multiple head-to-head trials, and derive some good practices for its design and analysis.

METHODS

We simulated various scenarios involving 4 to 20 candidate interventions among which 2 to 8 could be simultaneously allocated. A binary outcome was assumed. One or two interventions were assumed effective, with various interactions (positive, negative, none). Efficient combinatorics algorithms were created. Sample sizes and power were obtained by simulations in which the statistical test was either difference of proportions or multivariate logistic regression Wald test with or without interaction terms for adjustment, with Bonferroni multiplicity-adjusted alpha risk for both. Native R code is provided without need for compiling or packages.

RESULTS

Distributive trials reduce sample sizes 2- to sevenfold compared to parallel arm trials, and increase them 1- to twofold compared to factorial trials, mostly when fewer allocations than for the factorial design are possible. An unexpectedly effective intervention causes small decreases in power (< 10%) if its effect is additive, but large decreases (possibly down to 0) if not, as for factorial designs. These large decreases are prevented by using interaction terms to adjust the analysis, but these additional estimands have a sample size cost and are better pre-specified. The issue can also be managed by adding a true control arm without any intervention.

CONCLUSION

Distributive randomization is a viable design for mass parallel evaluation of interventions in constrained trial populations. It should be introduced first in clinical settings where many undercharacterized interventions are potentially available, such as disease prevention strategies, digital behavioral interventions, dietary supplements for chronic conditions, or emerging diseases. Pre-trial simulations are recommended, for which tools are provided.

Reporting of Factorial Randomized Trials: Extension of the CONSORT 2010 Statement

Importance

Transparent reporting of randomized trials is essential to facilitate critical appraisal and interpretation of results. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, reporting of factorial trials is suboptimal.

Objective

To develop a consensus-based extension to the Consolidated Standards of Reporting Trials (CONSORT) 2010 Statement for factorial trials.

Design

Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT extension for factorial trials was developed by (1) generating a list of reporting recommendations for factorial trials using a scoping review of methodological articles identified using a MEDLINE search (from inception to May 2019) and supplemented with relevant articles from the personal collections of the authors; (2) a 3-round Delphi survey between January and June 2022 to identify additional items and assess the importance of each item, completed by 104 panelists from 14 countries; and (3) a hybrid consensus meeting attended by 15 panelists to finalize the selection and wording of items for the checklist.

Findings

This CONSORT extension for factorial trials modifies 16 of the 37 items in the CONSORT 2010 checklist and adds 1 new item. The rationale for the importance of each item is provided. Key recommendations are (1) the reason for using a factorial design should be reported, including whether an interaction is hypothesized, (2) the treatment groups that form the main comparisons should be clearly identified, and (3) for each main comparison, the estimated interaction effect and its precision should be reported.

Conclusions and Relevance

This extension of the CONSORT 2010 Statement provides guidance on the reporting of factorial randomized trials and should facilitate greater understanding of and transparency in their reporting.

Consensus Statement for Protocols of Factorial Randomized Trials: Extension of the SPIRIT 2013 Statement

Importance

Trial protocols outline a trial's objectives as well as the methods (design, conduct, and analysis) that will be used to meet those objectives, and transparent reporting of trial protocols ensures objectives are clear and facilitates appraisal regarding the suitability of study methods. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, no extension of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 Statement, which provides guidance on reporting of trial protocols, for factorial trials is available.

Objective

To develop a consensus-based extension to the SPIRIT 2013 Statement for factorial trials.

Evidence Review

The SPIRIT extension for factorial trials was developed using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework. First, a list of reporting recommendations was generated using a scoping review of methodological articles identified using a MEDLINE search (inception to May 2019), which was supplemented with relevant articles from the personal collections of the authors. Second, a 3-round Delphi survey (January to June 2022, completed by 104 panelists from 14 countries) was conducted to assess the importance of items and identify additional recommendations. Third, a hybrid consensus meeting was held, attended by 15 panelists to finalize selection and wording of the checklist.

Findings

This SPIRIT extension for factorial trials modified 9 of the 33 items in the SPIRIT 2013 checklist. Key reporting recommendations were that the rationale for using a factorial design should be provided, including whether an interaction is hypothesized; the treatment groups that will form the main comparisons should be identified; and statistical methods for each main comparison should be provided, including how interactions will be assessed.

Conclusions and Relevance

In this consensus statement, 9 factorial-specific items were provided that should be addressed in all protocols of factorial trials to increase the trial's utility and transparency.

Effectiveness of adding rehabilitation of cervical related sensorimotor control to manual therapy and exercise for neck pain: A randomized controlled trial

BACKGROUND

Local neck treatments and sensorimotor training can improve cervical proprioception and balance, but it remains unclear what treatments and treatment combination achieve the best outcomes.

OBJECTIVES

To investigate the most effective interventions to improve disturbances in joint position sense (JPS) and balance and their effects on neck pain, dizziness and related features in the short- and long-terms.

DESIGN

2x2 factorial, randomized controlled trial.

METHODS

Participants with neck pain (n = 152) were randomly allocated to one of four intervention groups: i) local neck treatment (NT), ii) NT + JPS/oculomotor exercises (JPS/OC), iii) NT + balance exercises, and iv) all treatments. Participants received 12 treatments over 6 weeks. Primary outcomes were postural sway and joint position error. Secondary outcomes included gait speed, dizziness, pain intensity and disability, cervical range of motion, functional ability, and quality of life. Outcome measures were taken at baseline, posttreatment and 3-, 6- and 12-month follow-ups.

RESULTS

All four interventions resulted in short- and long-term improvements in primary and secondary outcomes with medium to large effect sizes, but JPS and balance in neck torsion improved most with the addition of a combined program of JPS/OC + balance exercises to NT while balance in neck neutral improved most with the addition of balance exercises. Adding sensorimotor training was also more effective in maintaining levels of improvement in neck pain and disability at 6- and 12-months. Effect sizes of additional treatment benefits were medium to large.

CONCLUSIONS

Adding specific training of JPS/OC and balance to NT best addresses deficits in cervical proprioception and balance.

CLINICAL TRIAL REGISTRATION NUMBER

xxxxx.

White Paper: Open Digital Health - accelerating transparent and scalable health promotion and treatment

In this White Paper, we outline recommendations from the perspective of health psychology and behavioural science, addressing three research gaps: (1) What methods in the health psychology research toolkit can be best used for developing and evaluating digital health tools? (2) What are the most feasible strategies to reuse digital health tools across populations and settings? (3) What are the main advantages and challenges of sharing (openly publishing) data, code, intervention content and design features of digital health tools? We provide actionable suggestions for researchers joining the continuously growing Open Digital Health movement, poised to revolutionise health psychology research and practice in the coming years. This White Paper is positioned in the current context of the COVID-19 pandemic, exploring how digital health tools have rapidly gained popularity in 2020-2022, when world-wide health promotion and treatment efforts rapidly shifted from face-to-face to remote delivery. This statement is written by the Directors of the not-for-profit Open Digital Health initiative (n = 6), Experts attending the European Health Psychology Society Synergy Expert Meeting (n = 17), and the initiative consultant, following a two-day meeting (19-20th August 2021).

Estimands for factorial trials

Factorial trials offer an efficient method to evaluate multiple interventions in a single trial, however the use of additional treatments can obscure research objectives, leading to inappropriate analytical methods and interpretation of results. We define a set of estimands for factorial trials, and describe a framework for applying these estimands, with the aim of clarifying trial objectives and ensuring appropriate primary and sensitivity analyses are chosen. This framework is intended for use in factorial trials where the intent is to conduct "two-trials-in-one" (ie, to separately evaluate the effects of treatments A and B), and is comprised of four steps: (i) specifying how additional treatment(s) (eg, treatment B) will be handled in the estimand, and how intercurrent events affecting the additional treatment(s) will be handled; (ii) designating the appropriate factorial estimator as the primary analysis strategy; (iii) evaluating the interaction to assess the plausibility of the assumptions underpinning the factorial estimator; and (iv) performing a sensitivity analysis using an appropriate multiarm estimator to evaluate to what extent departures from the underlying assumption of no interaction may affect results. We show that adjustment for other factors is necessary for noncollapsible effect measures (such as odds ratio), and through a trial re-analysis we find that failure to consider the estimand could lead to inappropriate interpretation of results. We conclude that careful use of the estimands framework clarifies research objectives and reduces the risk of misinterpretation of trial results, and should become a standard part of both the protocol and reporting of factorial trials.

Sample size calculation in hierarchical 2×2 factorial trials with unequal cluster sizes

Motivated by a suicide prevention trial with hierarchical treatment allocation (cluster-level and individual-level treatments), we address the sample size requirements for testing the treatment effects as well as their interaction. We assume a linear mixed model, within which two types of treatment effect estimands (controlled effect and marginal effect) are defined. For each null hypothesis corresponding to an estimand, we derive sample size formulas based on large-sample z-approximation, and provide finite-sample modifications based on a t-approximation. We relax the equal cluster size assumption and express the sample size formulas as functions of the mean and coefficient of variation of cluster sizes. We show that the sample size requirement for testing the controlled effect of the cluster-level treatment is more sensitive to cluster size variability than that for testing the controlled effect of the individual-level treatment; the same observation holds for testing the marginal effects. In addition, we show that the sample size for testing the interaction effect is proportional to that for testing the controlled or the marginal effect of the individual-level treatment. We conduct extensive simulations to validate the proposed sample size formulas, and find the empirical power agrees well with the predicted power for each test. Furthermore, the t-approximations often provide better control of type I error rate with a small number of clusters. Finally, we illustrate our sample size formulas to design the motivating suicide prevention factorial trial. The proposed methods are implemented in the R package H2x2Factorial.

A randomized factorial trial of internet-delivered cognitive behavioural therapy: An 8-week program with or without extended support and booster lesson

While internet-delivered cognitive behavioural therapy (ICBT) is effective, some patients suggest extended support post-treatment could improve care. In this randomized factorial trial, we examined the benefits of an 8-week therapist-assisted ICBT program offered with or without an optional 4-week extension of support (Factor 1) and with or without an optional booster lesson (Factor 2). Patients screened for ICBT for depression and/or anxiety were randomly assigned to the conditions (N = 434) and we examined the use of the extension and booster, differences between those who did or did not use extension or booster, and the impact of the extension or booster on outcomes, engagement, and satisfaction at 26-weeks post-enrollment. Therapists recorded time and observations with offering support during the extension and booster. In the extension group, 54.4% (n = 56) requested the extension, while in the booster group 50.9% (n = 56) accessed the booster, and in the combined group, 41.6% (n = 47) requested the extension and 51.3% (n = 58) accessed the booster. Those who requested the extension were older, and more likely to report medication and mental health service use and severe mental health-related disability at pre-treatment; they also reported putting less effort into ICBT and finding skills more difficult. The booster was more often used among those with lower symptom severity, and those who put more effort into and had more positive experiences with ICBT. As expected, those assigned to extension sent more messages to their therapist, and those assigned to booster logged in more often. Therapists also took more time to deliver ICBT with an extension (>18 min) or booster (>13 min) compared to the 8-week program, and perceived extension and booster as beneficial for some, but not all patients. Treatment satisfaction was high across conditions, and effect sizes were large from pre-treatment to 26-week follow-up on most measures. No significant group differences were found in this study. Lack of group differences, however, could reflect low use of the extension and booster. Results provide helpful information about the demand for extensions and boosters, and provide directions for future research.

Rationale and design of the BA-SCAD (Beta-blockers and Antiplatelet agents in patients with Spontaneous Coronary Artery Dissection) randomized clinical trial

INTRODUCTION Y OBJECTIVES

Spontaneous coronary artery dissection (SCAD) is a rare cause of acute coronary syndrome. Most patients are empirically treated with beta-blockers and antiplatelet drugs. The Beta-blockers and Antiplatelet agents in patients with Spontaneous Coronary Artery Dissection (BA-SCAD) is an academic, pragmatic, prospective, randomized, open-label, blinded-endpoint clinical trial, performed under the auspices of the Spanish Society of Cardiology, to assess the efficacy of pharmacological therapy in patients with SCAD.

METHODS

Using a 2 x 2 factorial design, 600 patients will be randomized (1:1/1:1) to: a) beta-blockers (yes/no) and b) "short" (1 month) vs "prolonged" (12 months) antiplatelet therapy. Only patients with preserved left ventricular ejection fraction will be randomized to beta-blockers (yes/no) because patients with reduced left ventricular ejection fraction will receive beta-blockers according to current guidelines. Similarly, only conservatively managed patients (ie, no coronary intervention) will be randomized to the antiplatelet stratum, as patients requiring coronary interventions will receive 1-year dual antiplatelet therapy. The primary efficacy endpoint includes a composite of death, myocardial infarction, stroke, coronary revascularization, recurrent SCAD, and unplanned hospitalization for acute coronary syndrome or heart failure at 1 year. The primary safety endpoint will be bleeding. All patients will be clinically followed up yearly. A comprehensive set of additional substudies (clinical, imaging, revascularization, biomarkers, inflammatory, immunologic, pharmacogenetics, and genetic) will be conducted to ensure a holistic view of this unique and challenging clinical entity.

CONCLUSIONS

The results of the BA-SCAD randomized clinical trial will advance our knowledge in the treatment of patients with SCAD. The study was registered at ClinicalTrials.gov (Identifier: NCT04850417).

A pragmatic factorial randomized controlled trial of transdiagnostic internet-delivered cognitive behavioural therapy: Exploring benefits of homework reflection questionnaires and twice-weekly therapist support

With the growing demand for internet-delivered cognitive behavioural therapy (iCBT), this pragmatic factorial (2 × 2 × 2) randomized controlled trial evaluated strategies for facilitating iCBT engagement and outcomes in routine care. Specifically, the benefits to patients and therapists of using homework reflection questionnaires and offering patients twice-weekly therapist support were examined. Patients (n = 632) accepted into iCBT for depression and/or anxiety were randomly assigned to complete homework reflection questionnaires or not (factor 1), receive once- or twice-weekly support (factor 2), and to receive care from therapists employed in one of two settings (iCBT clinic or a community mental health clinic; factor 3). Outcomes were measured at pre-treatment, and 8, 12, and 24-weeks post-enrollment. Therapist time was tracked and a focus group was conducted to examine therapist experiences. No differences in patient outcomes were found between therapists employed in the two settings; as such, these two groups were combined for further analyses. In terms of engagement, homework reflection questionnaires were associated with fewer website log-ins and days accessing iCBT; twice-weekly support was associated with more patient emails sent to therapists. Despite engagement differences, homework reflection questionnaires and twice-weekly support did not significantly impact primary outcomes; all groups showed large improvements in depression and anxiety that were maintained at 24-week follow-up. Therapists perceived a number of benefits and challenges associated with responding to homework reflection questionnaires and offering twice-weekly support; most notably the strategies did not benefit all patients. Twice-weekly support was associated with increased therapist time and organizational challenges. It is concluded that neither completion of homework questionnaires nor offering twice-weekly support significantly improve iCBT in routine care.

Study protocol: NeoCLEAR: Neonatal Champagne Lumbar punctures Every time - An RCT: a multicentre, randomised controlled 2 × 2 factorial trial to investigate techniques to increase lumbar puncture success

Background

The neonatal period carries the highest risk of bacterial meningitis (~ 1 in 5000 births), bearing high mortality (~ 10%) and morbidity (20–50%) rates. Lumbar puncture (LP) remains essential to the diagnosis of meningitis. Though LP is a common procedure in neonates, success rates are lower (50–60%) than in other patient populations. None of the currently-practised neonatal LP techniques are supported by evidence from adequately-powered, randomised controlled trials (RCTs). NeoCLEAR aims to compare two modifications to the traditional technique which are free, accessible, and commonly practised: sitting (as opposed to lying) position, and ‘early’ (as opposed to ‘late’) stylet removal.

Methods/design

Written parental informed consent permitting, infants in neonatal/maternity wards, of 27^+ 0 to 44^+ 0 weeks corrected gestational age and weighing ≥1000 g, who require an LP, will be randomly allocated to sitting or lying position, and to early or late stylet removal. The co-primary objectives are to compare success rates (the proportion of infants with cerebrospinal fluid red cell count < 10,000/mm³ on first LP procedure) in 1020 infants between the two positions, and between the two methods of stylet removal. Secondary outcomes relate to LP procedures, complications, diagnoses of meningitis, duration of antibiotics and hospital stay. A modified intention-to-treat analysis will be conducted.

Discussion

Two modifications to the traditional LP technique (sitting vs lying position; and early vs late stylet removal) will be simultaneously investigated in an efficient and appropriately-powered 2 × 2 factorial RCT design. Analysis will identify the optimal techniques (in terms of obtaining easily-interpretable cerebrospinal fluid), as well as the impact on infants, parents and healthcare systems whilst providing robust safety data. Using a pragmatic RCT design, all practitioners will be trained in all LP techniques, but there will inevitably be variation between unit practice guidelines and other aspects of individual care.

An improved LP technique would result in:

• Fewer uninterpretable samples, repeated attempts and procedures

• Reduced distress for infants and families

• Decreased antibiotic use and risk of antibiotic resistance

• Reduced healthcare costs due to fewer procedures, reduced length of stay, shorter antibiotic courses, and minimised antibiotic-associated complications

Trial registration

ISRCTN14040914. Date assigned: 26/06/2018.

Alternative clinical trial designs

High-quality clinical trials are needed to advance the care of injured patients. Traditional randomized clinical trials in trauma have challenges in generating new knowledge due to many issues, including logistical difficulties performing individual randomization, unclear pretrial estimates of treatment effect leading to often unpowered studies, and difficulty assessing the generalizability of an intervention given the heterogeneity of both patients and trauma centers. In this review, we discuss alternative clinical trial designs that can address some of these difficulties. These include pragmatic trials, cluster randomization, cluster randomized stepped wedge designs, factorial trials, and adaptive designs. Additionally, we discuss how Bayesian methods of inference may provide more knowledge to trauma and acute care surgeons compared with traditional, frequentist methods.