Multiple imputation methods for bivariate outcomes in cluster randomised trials

A scoping review of statistical methods for trial-based economic evaluations: The current state of play

The statistical quality of trial-based economic evaluations is often suboptimal, while a comprehensive overview of available statistical methods is lacking. Therefore, this review summarized and critically appraised available statistical methods for trial-based economic evaluations. A literature search was performed to identify studies on statistical methods for dealing with baseline imbalances, skewed costs and/or effects, correlated costs and effects, clustered data, longitudinal data, missing data and censoring in trial-based economic evaluations. Data was extracted on the statistical methods described, their advantages, disadvantages, relative performance and recommendations of the study. Sixty-eight studies were included. Of them, 27 (40%) assessed methods for baseline imbalances, 39 (57%) assessed methods for skewed costs and/or effects, 27 (40%) assessed methods for correlated costs and effects, 18 (26%) assessed methods for clustered data, 7 (10%) assessed methods for longitudinal data, 26 (38%) assessed methods for missing data and 10 (15%) assessed methods for censoring. All identified methods were narratively described. This review provides a comprehensive overview of available statistical methods for dealing with the most common statistical complexities in trial-based economic evaluations. Herewith, it can provide valuable input for researchers when deciding which statistical methods to use in a trial-based economic evaluation.

Children and Young People's Health Partnership Evelina London Model of Care: economic evaluation protocol of a complex system change

INTRODUCTION

The Children and Young People's Health Partnership (CYPHP) Evelina London Model of Care is a new approach to integrated care delivery for children and young people (CYP) with common health complaints and chronic conditions. CYPHP includes population health management (services shaped by data-driven understanding of population and individual needs, applied in this case to enable proactive case finding and tailored biopsychosocial care), specialist clinics with multidisciplinary health teams and training resources for professionals working with CYP. This complex health system strengthening programme has been implemented in South London since April 2018 and will be evaluated using a cluster randomised controlled trial with an embedded process evaluation. This protocol describes the within-trial and beyond-trial economic evaluation of CYPHP.

METHODS AND ANALYSIS

The economic evaluation will identify, measure and value resources and health outcome impacts of CYPHP compared with enhanced usual care from a National Health Service/Personal Social Service and a broader societal perspective. The study population includes 90 000 CYP under 16 years of age in 23 clusters (groups of general practitioner (GP) practices) to assess health service use and costs, with more detailed cost-effectiveness analysis of a targeted sample of 2138 CYP with asthma, eczema or constipation (tracer conditions). For the cost-effectiveness analysis, health outcomes will be measured using the Paediatric Quality of Life Inventory and quality-adjusted life years (QALYs) using the Child Health Utility 9 Dimensions (CHU-9D) measure. To account for changes in parental well-being, the Warwick-Edinburg Mental Well-being Scale will be integrated with QALYs in a cost-benefit analysis. The within-trial economic evaluation will be complemented by a novel long-term model that expands the analytical horizon to 10 years. Analyses will adhere to good practice guidelines and National Institute for Health and Care Excellence public health reference case.

ETHICS AND DISSEMINATION

The study has received ethical approval from South West-Cornwall and Plymouth Research Ethics Committee (REC Reference: 17/SW/0275). Results will be submitted for publication in peer-reviewed journals, made available in briefing papers for local decision-makers, and provided to the local community through website and public events. Findings will be generalisable to community-based models of care, especially in urban settings.

TRIAL REGISTRATION NUMBER

NCT03461848.

Multiple imputation for handling missing outcome data in randomized trials involving a mixture of independent and paired data

Randomized trials involving independent and paired observations occur in many areas of health research, for example in paediatrics, where studies can include infants from both single and twin births. Multiple imputation (MI) is often used to address missing outcome data in randomized trials, yet its performance in trials with independent and paired observations, where design effects can be less than or greater than one, remains to be explored. Using simulated data and through application to a trial dataset, we investigated the performance of different methods of MI for a continuous or binary outcome when followed by analysis using generalized estimating equations to account for clustering due to the pairs. We found that imputing data separately for independent and paired data, with paired data imputed in wide format, was the best performing MI method, producing unbiased point and standard error estimates for the treatment effect throughout. Ignoring clustering in the imputation model performed well in settings where the design effect due to the inclusion of paired data was close to one, but otherwise led to moderately biased variance estimates. Including a random cluster effect in the imputation model led to slightly biased point estimates for binary outcome data and variance estimates that were too small in some settings. Based on these results, we recommend researchers impute independent and paired data separately where feasible to do so. The exception is if the design effect due to the inclusion of paired data is close to one, where ignoring clustering may be appropriate.

Taking the Analysis of Trial-Based Economic Evaluations to the Next Level: The Importance of Accounting for Clustering

OBJECTIVES

The aim of this study was to assess the performance and impact of multilevel modelling (MLM) compared with ordinary least squares (OLS) regression in trial-based economic evaluations with clustered data.

METHODS

Three thousand datasets with balanced and unbalanced clusters were simulated with correlation coefficients between costs and effects of - 0.5, 0, and 0.5, and intraclass correlation coefficients (ICCs) varying between 0.05 and 0.30. Each scenario was analyzed using both MLM and OLS. Statistical uncertainty around MLM and OLS estimates was estimated using bootstrapping. Performance measures were estimated and compared between approaches, including bias, root mean squared error (RMSE) and coverage probability. Cost and effect differences, and their corresponding confidence intervals and standard errors, incremental cost-effectiveness ratios, incremental net-monetary benefits and cost-effectiveness acceptability curves were compared.

RESULTS

Cost-effectiveness outcomes were similar between OLS and MLM. MLM produced larger statistical uncertainty and coverage probabilities closer to nominal levels than OLS. The higher the ICC, the larger the effect on statistical uncertainty between MLM and OLS. Significant cost-effectiveness outcomes as estimated by OLS became non-significant when estimated by MLM. At all ICCs, MLM resulted in lower probabilities of cost effectiveness than OLS, and this difference became larger with increasing ICCs. Performance measures and cost-effectiveness outcomes were similar across scenarios with varying correlation coefficients between costs and effects.

CONCLUSIONS

Although OLS produced similar cost-effectiveness outcomes, it substantially underestimated the amount of variation in the data compared with MLM. To prevent suboptimal conclusions and a possible waste of scarce resources, it is important to use MLM in trial-based economic evaluations when data are clustered.

Novel education-based intervention to reduce inappropriate antibiotic prescribing for treatment of gonorrhoea in China: protocol for a cluster randomised controlled trial

INTRODUCTION

Inappropriate use of antibiotics to treat gonorrhoea can lead to antibiotic resistance. Education programmes may be helpful for improving physician prescribing behaviours in accordance with treatment guidelines. As traditional education based on printed materials may have limited effect on guideline-based treatment, innovative education strategies are needed. The current trial aims to assess the effectiveness of a novel education intervention to increase guideline-based treatment of gonorrhoea in China.

METHODS AND ANALYSIS

We will conduct a two-arm cluster randomised control trial at 144 hospitals (clusters) in eight Chinese provinces. The intervention will include an online training video developed on the WenJuanXing platform that covers workflows and requirements for managing a patient with uncomplicated gonorrhoea. Outpatient physicians in dermatology (dermatovenerology), urology, andrology and gynaecology will be given access to the video via a quick response code. In hospitals allocated to the control arm, physicians will continue to participate in their standard of care training programme. The primary outcome is the proportion of gonorrhoea antibiotic prescriptions adherent to Chinese national guidelines at the cluster level. In addition, to understand the reasons of physician's non-adherence to the intervention by conducting a questionnaire survey will be considered as the secondary outcome of the study.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the Medical Ethics Committee of the Chinese Academy of Medical Sciences Institute of Dermatology (2020-LS-004). All physicians will provide an informed consent prior to participating in the study. Findings of the trial will be disseminated through conferences and peer-reviewed journals, and will be used to develop training programmes for physicians.

TRIAL REGISTRATION NUMBER

ChiCTR2000029591.

Sit–stand desks to reduce sedentary behaviour in 9- to 10-year-olds: the Stand Out in Class pilot cluster RCT

Background

Sedentary behaviour (sitting) is a highly prevalent negative health behaviour, with individuals of all ages exposed to environments that promote prolonged sitting. The school classroom represents an ideal setting for environmental change through the provision of sit–stand desks.

Objectives

The aim of this study was to undertake a pilot cluster randomised controlled trial of the introduction of sit–stand desks in primary school classrooms, to inform a definitive trial. Objectives included providing information on school and participant recruitment and retention, acceptability of the intervention, and outcome measures. A preliminary estimate of the intervention’s effectiveness on the proposed primary outcome (change in weekday sitting time) for inclusion in a definitive trial was calculated, along with a preliminary assessment of potential cost-effectiveness. A full process evaluation was also undertaken.

Design

A two-armed pilot cluster randomised controlled trial with economic and qualitative evaluations. Schools were randomised on a 1 : 1 basis to the intervention (n = 4) or control (n = 4) trial arms.

Setting

Primary schools in Bradford, West Yorkshire, UK.

Participants

Children in Year 5 (i.e. aged 9–10 years).

Intervention

Six sit–stand desks replaced three standard desks (sitting six children) in the intervention classrooms for 4.5 months. Teachers were encouraged to ensure that all pupils were exposed to the sit–stand desks for at least 1 hour per day, on average, using a rotation system. Schools assigned to the control arm continued with their usual practice.

Main outcome measures

Trial feasibility outcomes included school and participant recruitment and attrition, acceptability of the intervention, and acceptability of and compliance with the proposed outcome measures [including weekday sitting measured using activPAL™ (PAL Technologies Ltd, Glasgow, UK) accelerometers, physical activity, adiposity, blood pressure, cognitive function, musculoskeletal comfort, academic progress, engagement and behaviour].

Results

Thirty-three per cent of schools approached and 75% (n = 176) of eligible children took part. At the 7-month follow-up, retention rates were 100% for schools and 97% for children. Outcome measure completion rates ranged from 63% to 97%. A preliminary estimate of intervention effectiveness, from a weighted linear regression model (adjusting for baseline sitting time and wear time) revealed a mean difference in change in sitting of –30.6 minutes per day (95% confidence interval –56.42 to –4.84 minutes per day) between the intervention and control trial arms. The process evaluation revealed that the intervention, recruitment and evaluation procedures were acceptable to teachers and children, with the exception of minor issues around activPAL attachment. A preliminary within-trial economic analysis revealed no difference between intervention and control trial arms in health and education resource use or outcomes. Long-term modelling estimated an unadjusted incremental cost-effectiveness ratio of Stand Out in Class of £78,986 per quality-adjusted life-year gained.

Conclusion

This study has provided evidence of the acceptability and feasibility of the Stand Out in Class intervention and evaluation methods. Preliminary evidence suggests that the intervention may have a positive direction of effect on weekday sitting time, which warrants testing in a full cluster randomised controlled trial. Lessons learnt from this trial will inform the planning of a definitive trial.

Trial registration

Current Controlled Trials ISRCTN12915848.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 8. See the NIHR Journals Library website for further project information.

A Comparison of Multilevel Imputation Schemes for Random Coefficient Models: Fully Conditional Specification and Joint Model Imputation with Random Covariance Matrices

Literature addressing missing data handling for random coefficient models is particularly scant, and the few studies to date have focused on the fully conditional specification framework and "reverse random coefficient" imputation. Although it has not received much attention in the literature, a joint modeling strategy that uses random within-cluster covariance matrices to preserve cluster-specific associations is a promising alternative for random coefficient analyses. This study is apparently the first to directly compare these procedures. Analytic results suggest that both imputation procedures can introduce bias-inducing incompatibilities with a random coefficient analysis model. Problems with fully conditional specification result from an incorrect distributional assumption, whereas joint imputation uses an underparameterized model that assumes uncorrelated intercepts and slopes. Monte Carlo simulations suggest that biases from these issues are tolerable if the missing data rate is 10% or lower and the sample is composed of at least 30 clusters with 15 observations per group. Furthermore, fully conditional specification tends to be superior with intraclass correlations that are typical of crosssectional data (e.g., ICC = .10), whereas the joint model is preferable with high values typical of longitudinal designs (e.g., ICC = .50).

Evaluating the impact of Marie Stopes International's digital family planning counselling application on the uptake of long-acting and permanent methods of contraception in Vietnam and Ethiopia: a study protocol for a multi-country cluster randomised controlled trial

Background

Maintaining quality of care in family planning (FP) counselling in low-resource settings is challenging. Job aids can help providers give more accurate and client-specific advice but require a provider to use them effectively and consistently. Marie Stopes International (MSI) have designed the tablet-computer based Digital Counselling Application (DCA), which prompts structured, supportive, client-specific and unbiased FP counselling. We hypothesise that a systematic exploration of clients’ fertility intentions, medical eligibility and preferences will increase their uptake of long acting and permanent methods of contraception (LAPMs).

Methods/design

We will conduct a two-armed, parallel, cluster randomised control trial across all MSI clinics (clusters) in Ethiopia (24) and Vietnam (11), randomising 18 clinics to the intervention group and 17 to the control group. Intervention providers will attend a two-day DCA-use training programme, and use DCA in their FP counselling sessions. Usual care providers will counsel clients as before. We aim to recruit 75 clients who have had FP counselling per clinic (2625 total), following them up via two telephone interviews, initially within 2 days and then at 4 months. The primary outcome is defined as the proportion of clients who report choosing a LAPM following FP counselling and will include switchers (FP counselling clients who switch from using any other FP method) and adopters (FP counselling clients who adopt any FP method having not previously been using one). We will also collect secondary outcomes at the initial follow-up (including the proportion of clients reporting being recommended a LAPM by a provider and a range of measures of client experience and satisfaction) and at the 4-month follow-up (including a range of measures of continuation rates for different FP method types). In the intervention arm, we will also conduct mixed-methods sampling to assess how providers use DCA (using an observational survey of provider–client interactions), and understand users’ experiences of receiving and giving DCA-based FP counselling (through in-depth interviews).

Discussion

This trial will provide novel information on the feasibility and acceptability of health worker delivered FP counselling using DCA, with robust evidence on its effectiveness at increasing the uptake of LAPMs in low-resource settings.

Trial registration

ISRCTN, ISRCTN11040557. Registered on 2 March 2017 (retrospectively registered).

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2815-0) contains supplementary material, which is available to authorized users.

Missing binary outcomes under covariate-dependent missingness in cluster randomised trials

Missing outcomes are a commonly occurring problem for cluster randomised trials, which can lead to biased and inefficient inference if ignored or handled inappropriately. Two approaches for analysing such trials are cluster‐level analysis and individual‐level analysis. In this study, we assessed the performance of unadjusted cluster‐level analysis, baseline covariate‐adjusted cluster‐level analysis, random effects logistic regression and generalised estimating equations when binary outcomes are missing under a baseline covariate‐dependent missingness mechanism. Missing outcomes were handled using complete records analysis and multilevel multiple imputation. We analytically show that cluster‐level analyses for estimating risk ratio using complete records are valid if the true data generating model has log link and the intervention groups have the same missingness mechanism and the same covariate effect in the outcome model. We performed a simulation study considering four different scenarios, depending on whether the missingness mechanisms are the same or different between the intervention groups and whether there is an interaction between intervention group and baseline covariate in the outcome model. On the basis of the simulation study and analytical results, we give guidance on the conditions under which each approach is valid. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have updated that review with developments in analysis of the past 13 years, with a companion article to focus on developments in design. We discuss developments in the topics of the earlier review (e.g., methods for parallel-arm GRTs, individually randomized group-treatment trials, and missing data) and in new topics, including methods to account for multiple-level clustering and alternative estimation methods (e.g., augmented generalized estimating equations, targeted maximum likelihood, and quadratic inference functions). In addition, we describe developments in analysis of alternative group designs (including stepped-wedge GRTs, network-randomized trials, and pseudocluster randomized trials), which require clustering to be accounted for in their design and analysis.

Multiple imputation methods for bivariate outcomes in cluster randomised trials

Missing observations are common in cluster randomised trials. The problem is exacerbated when modelling bivariate outcomes jointly, as the proportion of complete cases is often considerably smaller than the proportion having either of the outcomes fully observed. Approaches taken to handling such missing data include the following: complete case analysis, single‐level multiple imputation that ignores the clustering, multiple imputation with a fixed effect for each cluster and multilevel multiple imputation.

We contrasted the alternative approaches to handling missing data in a cost‐effectiveness analysis that uses data from a cluster randomised trial to evaluate an exercise intervention for care home residents.

We then conducted a simulation study to assess the performance of these approaches on bivariate continuous outcomes, in terms of confidence interval coverage and empirical bias in the estimated treatment effects. Missing‐at‐random clustered data scenarios were simulated following a full‐factorial design.

Across all the missing data mechanisms considered, the multiple imputation methods provided estimators with negligible bias, while complete case analysis resulted in biased treatment effect estimates in scenarios where the randomised treatment arm was associated with missingness. Confidence interval coverage was generally in excess of nominal levels (up to 99.8%) following fixed‐effects multiple imputation and too low following single‐level multiple imputation. Multilevel multiple imputation led to coverage levels of approximately 95% throughout. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.