A covariance estimator for GEE with improved small-sample properties

The effect of number of clusters and magnitude of within-cluster homogeneity in outcomes on the performance of four variance estimators for a marginal multivariable Cox regression model fit to clustered data in the context of observational research

Researchers often estimate the association between the hazard of a time-to-event outcome and the characteristics of individuals and the clusters in which individuals are nested. Lin and Wei's robust variance estimator is often used with a Cox regression model fit to clustered data. Recently, alternative variance estimators have been proposed: the Fay-Graubard estimator, the Kauermann-Carroll estimator, and the Mancl-DeRouen estimator. Using Monte Carlo simulations, we found that, when fitting a marginal Cox regression model with both individual-level and cluster-level covariates: (i) in the presence of weak to moderate within-cluster homogeneity of outcomes, the Lin-Wei variance estimator can result in estimates of the SE with moderate bias when the number of clusters is fewer than 20-30, while in the presence of strong within-cluster homogeneity, it can result in biased estimation even when the number of clusters is as large as 100; (ii) when the number of clusters was less than approximately 20, the Fay-Graubard variance estimator tended to result in estimates of SE with the lowest bias; (iii) when the number of clusters exceeded approximately 20, the Mancl-DeRouen estimator tended to result in estimated standard errors with the lowest bias; (iv) the Mancl-DeRouen estimator used with a t-distribution tended to result in 95% confidence that had the best performance of the estimators; (v) when the magnitude of within-cluster homogeneity in outcomes was strong or very strong, all methods resulted in confidence intervals with lower than advertised coverage rates even when the number of clusters was very large.

Effect of the Communities that HEAL intervention on receipt of behavioral therapies for opioid use disorder: A cluster randomized wait-list controlled trial

BACKGROUND

The U.S. opioid overdose crisis persists. Outpatient behavioral health services (BHS) are essential components of a comprehensive response to opioid use disorder and overdose fatalities. The Helping to End Addiction Long-Term® (HEALing) Communities Study developed the Communities That HEAL (CTH) intervention to reduce opioid overdose deaths in 67 communities in Kentucky, Ohio, New York, and Massachusetts through the implementation of evidence-based practices (EBPs), including BHS. This paper compares the rate of individuals receiving outpatient BHS in Wave 1 intervention communities (n = 34) to waitlisted Wave 2 communities (n = 33).

METHODS

Medicaid data included individuals ≥18 years of age receiving any of five BHS categories: intensive outpatient, outpatient, case management, peer support, and case management or peer support. Negative binomial regression models estimated the rate of receiving each BHS for Wave 1 and Wave 2. Effect modification analyses evaluated changes in the effect of the CTH intervention between Wave 1 and Wave 2 by research site, rurality, age, sex, and race/ethnicity.

RESULTS

No significant differences were detected between intervention and waitlisted communities in the rate of individuals receiving any of the five BHS categories. None of the interaction effects used to test the effect modification were significant.

CONCLUSIONS

Several factors should be considered when interpreting results-no significant intervention effects were observed through Medicaid claims data, the best available data source but limited in terms of capturing individuals reached by the intervention. Also, the 12-month evaluation window may have been too brief to see improved outcomes considering the time required to stand-up BHS.

TRIAL REGISTRATION

Clinical Trials.gov http://www.

CLINICALTRIALS

gov: Identifier: NCT04111939.

Maintaining the validity of inference from linear mixed models in stepped-wedge cluster randomized trials under misspecified random-effects structures

Linear mixed models are commonly used in analyzing stepped-wedge cluster randomized trials. A key consideration for analyzing a stepped-wedge cluster randomized trial is accounting for the potentially complex correlation structure, which can be achieved by specifying random-effects. The simplest random effects structure is random intercept but more complex structures such as random cluster-by-period, discrete-time decay, and more recently, the random intervention structure, have been proposed. Specifying appropriate random effects in practice can be challenging: assuming more complex correlation structures may be reasonable but they are vulnerable to computational challenges. To circumvent these challenges, robust variance estimators may be applied to linear mixed models to provide consistent estimators of standard errors of fixed effect parameters in the presence of random-effects misspecification. However, there has been no empirical investigation of robust variance estimators for stepped-wedge cluster randomized trials. In this article, we review six robust variance estimators (both standard and small-sample bias-corrected robust variance estimators) that are available for linear mixed models in R, and then describe a comprehensive simulation study to examine the performance of these robust variance estimators for stepped-wedge cluster randomized trials with a continuous outcome under different data generators. For each data generator, we investigate whether the use of a robust variance estimator with either the random intercept model or the random cluster-by-period model is sufficient to provide valid statistical inference for fixed effect parameters, when these working models are subject to random-effect misspecification. Our results indicate that the random intercept and random cluster-by-period models with robust variance estimators performed adequately. The CR3 robust variance estimator (approximate jackknife) estimator, coupled with the number of clusters minus two degrees of freedom correction, consistently gave the best coverage results, but could be slightly conservative when the number of clusters was below 16. We summarize the implications of our results for the linear mixed model analysis of stepped-wedge cluster randomized trials and offer some practical recommendations on the choice of the analytic model.

Goodness-of-fit tests for modified Poisson regression possibly producing fitted values exceeding one in binary outcome analysis

Modified Poisson regression, which estimates the regression parameters in the log-binomial regression model using the Poisson quasi-likelihood estimating equation and robust variance, is a useful tool for estimating the adjusted risk and prevalence ratio in binary outcome analysis. Although several goodness-of-fit tests have been developed for other binary regressions, few goodness-of-fit tests are available for modified Poisson regression. In this study, we proposed several goodness-of-fit tests for modified Poisson regression, including the modified Hosmer-Lemeshow test with empirical variance, Tsiatis test, normalized Pearson chi-square tests with binomial variance and Poisson variance, and normalized residual sum of squares test. The original Hosmer-Lemeshow test and normalized Pearson chi-square test with binomial variance are inappropriate for the modified Poisson regression, which can produce a fitted value exceeding 1 owing to the unconstrained parameter space. A simulation study revealed that the normalized residual sum of squares test performed well regarding the type I error probability and the power for a wrong link function. We applied the proposed goodness-of-fit tests to the analysis of cross-sectional data of patients with cancer. We recommend the normalized residual sum of squares test as a goodness-of-fit test in the modified Poisson regression.

Using Power Analysis to Choose the Unit of Randomization, Outcome, and Approach for Subgroup Analysis for a Multilevel Randomized Controlled Clinical Trial to Reduce Disparities in Cardiovascular Health

We give examples of three features in the design of randomized controlled clinical trials which can increase power and thus decrease sample size and costs. We consider an example multilevel trial with several levels of clustering. For a fixed number of independent sampling units, we show that power can vary widely with the choice of the level of randomization. We demonstrate that power and interpretability can improve by testing a multivariate outcome rather than an unweighted composite outcome. Finally, we show that using a pooled analytic approach, which analyzes data for all subgroups in a single model, improves power for testing the intervention effect compared to a stratified analysis, which analyzes data for each subgroup in a separate model. The power results are computed for a proposed prevention research study. The trial plans to randomize adults to either telehealth (intervention) or in-person treatment (control) to reduce cardiovascular risk factors. The trial outcomes will be measures of the Essential Eight, a set of scores for cardiovascular health developed by the American Heart Association which can be combined into a single composite score. The proposed trial is a multilevel study, with outcomes measured on participants, participants treated by the same provider, providers nested within clinics, and clinics nested within hospitals. Investigators suspect that the intervention effect will be greater in rural participants, who live farther from clinics than urban participants. The results use published, exact analytic methods for power calculations with continuous outcomes. We provide example code for power analyses using validated software.

Assessing treatment effect heterogeneity in the presence of missing effect modifier data in cluster-randomized trials

Understanding whether and how treatment effects vary across subgroups is crucial to inform clinical practice and recommendations. Accordingly, the assessment of heterogeneous treatment effects based on pre-specified potential effect modifiers has become a common goal in modern randomized trials. However, when one or more potential effect modifiers are missing, complete-case analysis may lead to bias and under-coverage. While statistical methods for handling missing data have been proposed and compared for individually randomized trials with missing effect modifier data, few guidelines exist for the cluster-randomized setting, where intracluster correlations in the effect modifiers, outcomes, or even missingness mechanisms may introduce further threats to accurate assessment of heterogeneous treatment effect. In this article, the performance of several missing data methods are compared through a simulation study of cluster-randomized trials with continuous outcome and missing binary effect modifier data, and further illustrated using real data from the Work, Family, and Health Study. Our results suggest that multilevel multiple imputation and Bayesian multilevel multiple imputation have better performance than other available methods, and that Bayesian multilevel multiple imputation has lower bias and closer to nominal coverage than standard multilevel multiple imputation when there are model specification or compatibility issues.

Esophageal Versus Rectal Temperature Monitoring During Whole-Body Therapeutic Hypothermia for Hypoxic-Ischemic Encephalopathy: Association with Short- and Long-Term Outcomes

OBJECTIVE

To compare the short- and long-term outcomes of infants with hypoxic-ischemic encephalopathy (HIE) treated with whole-body therapeutic hypothermia (TH), monitored by esophageal vs rectal temperature.

STUDY DESIGN

We conducted a secondary analysis of the multicenter High-Dose Erythropoietin for Asphyxia and Encephalopathy (HEAL) trial. All infants had moderate or severe HIE and were treated with whole-body TH. The primary outcome was death or neurodevelopmental impairment (NDI) at 22-36 months of age. Secondary outcomes included seizures, evidence of brain injury on magnetic resonance imaging, and complications of hypothermia. Logistic regression was used with adjustment for disease severity and site as clustering variable because cooling modality differed by site.

RESULTS

Of the 500 infants who underwent TH, 294 (59%) and 206 (41%) had esophageal and rectal temperature monitoring, respectively. There were no differences in death or NDI, seizures, or evidence of injury on magnetic resonance imaging between the 2 groups. Infants treated with TH and rectal temperature monitoring had lower odds of overcooling (OR 0.52, 95% CI 0.34-0.80) and lower odds of hypotension (OR 0.57, 95% CI 0.39-0.84) compared with those with esophageal temperature monitoring.

CONCLUSIONS

Although infants undergoing TH with esophageal monitoring were more likely to experience overcooling and hypotension, the rate of death or NDI was similar whether esophageal monitoring or rectal temperature monitoring was used. Further studies are needed to investigate whether esophageal temperature monitoring during TH is associated with an increased risk of overcooling and hypotension.

MRBEE: A bias-corrected multivariable Mendelian randomization method

Mendelian randomization (MR) is an instrumental variable approach used to infer causal relationships between exposures and outcomes, which is becoming increasingly popular because of its ability to handle summary statistics from genome-wide association studies. However, existing MR approaches often suffer the bias from weak instrumental variables, horizontal pleiotropy and sample overlap. We introduce MRBEE (MR using bias-corrected estimating equation), a multivariable MR method capable of simultaneously removing weak instrument and sample overlap bias and identifying horizontal pleiotropy. Our extensive simulations and real data analyses reveal that MRBEE provides nearly unbiased estimates of causal effects, well-controlled type I error rates and higher power than comparably robust methods and is computationally efficient. Our real data analyses result in consistent causal effect estimates and offer valuable guidance for conducting multivariable MR studies, elucidating the roles of pleiotropy, and identifying total 42 horizontal pleiotropic loci missed previously that are associated with myopia, schizophrenia, and coronary artery disease.

Two implementation strategies to support the integration of depression screening and treatment into hypertension and diabetes care in Malawi (SHARP): parallel, cluster-randomised, controlled, implementation trial

BACKGROUND

Although evidence-based treatments for depression in low-resource settings are established, implementation strategies to scale up these treatments remain poorly defined. We aimed to compare two implementation strategies in achieving high-quality integration of depression care into chronic medical care and improving mental health outcomes in patients with hypertension and diabetes.

METHODS

We conducted a parallel, cluster-randomised, controlled, implementation trial in ten health facilities across Malawi. Facilities were randomised (1:1) by covariate-constrained randomisation to either an internal champion alone (ie, basic strategy group) or an internal champion plus external supervision with audit and feedback (ie, enhanced strategy group). Champions integrated a three-element, evidence-based intervention into clinical care: universal depression screening; peer-delivered psychosocial counselling; and algorithm-guided, non-specialist antidepressant management. External supervision involved structured facility visits by Ministry officials and clinical experts to assess quality of care and provide supportive feedback approximately every 4 months. Eligible participants were adults (aged 18-65 years) seeking hypertension and diabetes care with signs of depression (Patient Health Questionnaire-9 score ≥5). Primary implementation outcomes were depression screening fidelity, treatment initiation fidelity, and follow-up treatment fidelity over the first 3 months of treatment, analysed by intention to treat. This trial is registered with ClinicalTrials.gov, NCT03711786, and is complete.

FINDINGS

Five (50%) facilities were randomised to the basic strategy and five (50%) to the enhanced strategy. Between Oct 1, 2019, and Nov 30, 2021, in the basic group, 587 patients were assessed for eligibility, of whom 301 were enrolled; in the enhanced group, 539 patients were assessed, of whom 288 were enrolled. All clinics integrated the evidence-based intervention and were included in the analyses. Of 60 774 screening-eligible visits, screening fidelity was moderate (58% in the enhanced group vs 53% in the basic group; probability difference 5% [95% CI -38% to 47%]; p=0·84) and treatment initiation fidelity was high (99% vs 98%; 0% [-3% to 3%]; p=0·89) in both groups. However, treatment follow-up fidelity was substantially higher in the enhanced group than in the basic group (82% vs 20%; 62% [36% to 89%]; p=0·0020). Depression remission was higher in the enhanced group than in the basic group (55% vs 36%; 19% [3% to 34%]; p=0·045). Serious adverse events were nine deaths (five in the basic group and four in the enhanced group) and 26 hospitalisations (20 in the basic group and six in the enhanced group); none were treatment-related.

INTERPRETATION

The enhanced implementation strategy led to an increase in fidelity in providers' follow-up treatment actions and in rates of depression remission, consistent with the literature that follow-up decisions are crucial to improving depression outcomes in integrated care models. These findings suggest that external supervision combined with an internal champion could offer an important advance in integrating depression treatment into general medical care in low-resource settings.

FUNDING

The National Institute of Mental Health.

Power calculation for detecting interaction effect in cross-sectional stepped-wedge cluster randomized trials: an important tool for disparity research

BACKGROUND

The stepped-wedge cluster randomized trial (SW-CRT) design has become popular in healthcare research. It is an appealing alternative to traditional cluster randomized trials (CRTs) since the burden of logistical issues and ethical problems can be reduced. Several approaches for sample size determination for the overall treatment effect in the SW-CRT have been proposed. However, in certain situations we are interested in examining the heterogeneity in treatment effect (HTE) between groups instead. This is equivalent to testing the interaction effect. An important example includes the aim to reduce racial disparities through healthcare delivery interventions, where the focus is the interaction between the intervention and race. Sample size determination and power calculation for detecting an interaction effect between the intervention status variable and a key covariate in the SW-CRT study has not been proposed yet for binary outcomes.

METHODS

We utilize the generalized estimating equation (GEE) method for detecting the heterogeneity in treatment effect (HTE). The variance of the estimated interaction effect is approximated based on the GEE method for the marginal models. The power is calculated based on the two-sided Wald test. The Kauermann and Carroll (KC) and the Mancl and DeRouen (MD) methods along with GEE (GEE-KC and GEE-MD) are considered as bias-correction methods.

RESULTS

Among three approaches, GEE has the largest simulated power and GEE-MD has the smallest simulated power. Given cluster size of 120, GEE has over 80% statistical power. When we have a balanced binary covariate (50%), simulated power increases compared to an unbalanced binary covariate (30%). With intermediate effect size of HTE, only cluster sizes of 100 and 120 have more than 80% power using GEE for both correlation structures. With large effect size of HTE, when cluster size is at least 60, all three approaches have more than 80% power. When we compare an increase in cluster size and increase in the number of clusters based on simulated power, the latter has a slight gain in power. When the cluster size changes from 20 to 40 with 20 clusters, power increases from 53.1% to 82.1% for GEE; 50.6% to 79.7% for GEE-KC; and 48.1% to 77.1% for GEE-MD. When the number of clusters changes from 20 to 40 with cluster size of 20, power increases from 53.1% to 82.1% for GEE; 50.6% to 81% for GEE-KC; and 48.1% to 79.8% for GEE-MD.

CONCLUSIONS

We propose three approaches for cluster size determination given the number of clusters for detecting the interaction effect in SW-CRT. GEE and GEE-KC have reasonable operating characteristics for both intermediate and large effect size of HTE.

Evaluating two small-sample corrections for fixed-effects standard errors and inferences in multilevel models with heteroscedastic, unbalanced, clustered data

Multilevel modeling (MLM) is commonly used in psychological research to model clustered data. However, data in applied research usually violate one of the essential assumptions of MLM-homogeneity of variance. While the fixed-effect estimates produced by the maximum likelihood method remain unbiased, the standard errors for the fixed effects are misestimated, resulting in inaccurate inferences and inflated or deflated type I error rates. To correct the bias in fixed effects standard errors and provide valid inferences, small-sample corrections such as the Kenward-Roger (KR) adjustment and the adjusted cluster-robust standard errors (CR-SEs) with the Satterthwaite approximation for t tests have been used. The current study compares KR with random slope (RS) models and the adjusted CR-SEs with ordinary least squares (OLS), random intercept (RI) and RS models to analyze small, heteroscedastic, clustered data using a Monte Carlo simulation. Results show the KR procedure with RS models has large biases and inflated type I error rates for between-cluster effects in the presence of level 2 heteroscedasticity. In contrast, the adjusted CR-SEs generally yield results with acceptable biases and maintain type I error rates close to the nominal level for all examined models. Thus, when the interest is only in within-cluster effect, any model with the adjusted CR-SEs could be used. However, when the interest is to make accurate inferences of the between-cluster effect, researchers should use the adjusted CR-SEs with RS to have higher power and guard against unmodeled heterogeneity. We reanalyzed an example in Snijders & Bosker (2012) to demonstrate the use of the adjusted CR-SEs with different models.

Designing individually randomized group treatment trials with repeated outcome measurements using generalized estimating equations

Individually randomized group treatment (IRGT) trials, in which the clustering of outcome is induced by group-based treatment delivery, are increasingly popular in public health research. IRGT trials frequently incorporate longitudinal measurements, of which the proper sample size calculations should account for correlation structures reflecting both the treatment-induced clustering and repeated outcome measurements. Given the relatively sparse literature on designing longitudinal IRGT trials, we propose sample size procedures for continuous and binary outcomes based on the generalized estimating equations approach, employing the block exchangeable correlation structures with different correlation parameters for the treatment arm and for the control arm, and surveying five marginal mean models with different assumptions of time effect: no-time constant treatment effect, linear-time constant treatment effect, categorical-time constant treatment effect, linear time by treatment interaction, and categorical time by treatment interaction. Closed-form sample size formulas are derived for continuous outcomes, which depends on the eigenvalues of the correlation matrices; detailed numerical sample size procedures are proposed for binary outcomes. Through simulations, we demonstrate that the empirical power agrees well with the predicted power, for as few as eight groups formed in the treatment arm, when data are analyzed using the matrix-adjusted estimating equations for the correlation parameters with a bias-corrected sandwich variance estimator.

Compare the marginal effects for environmental exposure and biomonitoring data with repeated measurements and values below the limit of detection

BACKGROUND

Environmental exposure and biomonitoring data with repeated measurements from environmental and occupational studies are commonly right-skewed and in the presence of limits of detection (LOD). However, existing model has not been discussed for small-sample properties and highly skewed data with non-detects and repeated measurements.

OBJECTIVE

Marginal modeling provides an alternative to analyzing longitudinal and cluster data, in which the parameter interpretations are with respect to marginal or population-averaged means.

METHODS

We outlined the theories of three marginal models, i.e., generalized estimating equations (GEE), quadratic inference functions (QIF), and generalized method of moments (GMM). With these approaches, we proposed to incorporate the fill-in methods, including single and multiple value imputation techniques, such that any measurements less than the limit of detection are assigned values.

RESULTS

We demonstrated that the GEE method works well in terms of estimating the regression parameters in small sample sizes, while the QIF and GMM outperform in large-sample settings, as parameter estimates are consistent and have relatively smaller mean squared error. No specific fill-in method can be deemed superior as each has its own merits.

IMPACT

Marginal modeling is firstly employed to analyze repeated measures data with non-detects, in which only the mean structure needs to be correctly provided to obtain consistent parameter estimates. After replacing non-detects through substitution methods and utilizing small-sample bias corrections, in a simulation study we found that the estimating approaches used in the marginal models have corresponding advantages under a wide range of sample sizes. We also applied the models to longitudinal and cluster working examples.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

Cluster-randomized trials (CRTs) involve randomizing entire groups of participants-called clusters-to treatment arms but are often comprised of a limited or fixed number of available clusters. While covariate adjustment can account for chance imbalances between treatment arms and increase statistical efficiency in individually randomized trials, analytical methods for individual-level covariate adjustment in small CRTs have received little attention to date. In this paper, we systematically investigate, through extensive simulations, the operating characteristics of propensity score weighting and multivariable regression as two individual-level covariate adjustment strategies for estimating the participant-average causal effect in small CRTs with a rare binary outcome and identify scenarios where each adjustment strategy has a relative efficiency advantage over the other to make practical recommendations. We also examine the finite-sample performance of the bias-corrected sandwich variance estimators associated with propensity score weighting and multivariable regression for quantifying the uncertainty in estimating the participant-average treatment effect. To illustrate the methods for individual-level covariate adjustment, we reanalyze a recent CRT testing a sedation protocol in 31 pediatric intensive care units.

Impairment of oculomotor functions in patients with early to advanced amyotrophic lateral sclerosis

Amyotrophic lateral sclerosis (ALS) can result into an incomplete locked in state (iLIS), in which communication depends on eye tracking computer devices. Oculomotor function impairments in ALS have been reported, but there is little research, particularly with respect to patients in iLIS. In the present study, we compared reflexive and executive oculomotor function by means of an eye tracking test battery between three groups: advanced ALS patients in iLIS (n = 22), patients in early to middle ALS stages (n = 44) and healthy subjects (n = 32). Patients with ALS showed significant deteriorations in oculomotor functions, with stronger impairments in iLIS. More specifically, ALS patients produced visually guided prosaccades with longer latencies and more frequent hypometria compared to healthy subjects. Longest latencies were obtained in iLIS patients, with a stronger prolongation for vertical than for horizontal prosaccades. ALS patients made more antisaccade errors and generated antisaccades with longer latencies. Smooth pursuit was also impaired in ALS. In the earlier ALS stages, bulbar onset patients presented stronger antisaccade and smooth pursuit deficits than spinal onset patients. Our findings reveal a relevant deterioration of important oculomotor functions in ALS, which increases in iLIS. It includes impairments of reflexive eye movements to loss of executive inhibitory control, indicating a progressing pathological involvement of prefrontal, midbrain and brainstem areas. The assessment of oculomotor functions may therefore provide clinically relevant bio- and progression marker, particularly in advanced ALS.

Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial

BACKGROUND

Recent work has shown that cluster-randomised trials can estimate two distinct estimands: the participant-average and cluster-average treatment effects. These can differ when participant outcomes or the treatment effect depends on the cluster size (termed informative cluster size). In this case, estimators that target one estimand (such as the analysis of unweighted cluster-level summaries, which targets the cluster-average effect) may be biased for the other. Furthermore, commonly used estimators such as mixed-effects models or generalised estimating equations with an exchangeable correlation structure can be biased for both estimands. However, there has been little empirical research into whether informative cluster size is likely to occur in practice.

METHOD

We re-analysed a cluster-randomised trial comparing two different thresholds for red blood cell transfusion in patients with acute upper gastrointestinal bleeding to explore whether estimates for the participant- and cluster-average effects differed, to provide empirical evidence for whether informative cluster size may be present. For each outcome, we first estimated a participant-average effect using independence estimating equations, which are unbiased under informative cluster size. We then compared this to two further methods: (1) a cluster-average effect estimated using either weighted independence estimating equations or unweighted cluster-level summaries, and (2) estimates from a mixed-effects model or generalised estimating equations with an exchangeable correlation structure. We then performed a small simulation study to evaluate whether observed differences between cluster- and participant-average estimates were likely to occur even if no informative cluster size was present.

RESULTS

For most outcomes, treatment effect estimates from different methods were similar. However, differences of >10% occurred between participant- and cluster-average estimates for 5 of 17 outcomes (29%). We also observed several notable differences between estimates from mixed-effects models or generalised estimating equations with an exchangeable correlation structure and those based on independence estimating equations. For example, for the EQ-5D VAS score, the independence estimating equation estimate of the participant-average difference was 4.15 (95% confidence interval: -3.37 to 11.66), compared with 2.84 (95% confidence interval: -7.37 to 13.04) for the cluster-average independence estimating equation estimate, and 3.23 (95% confidence interval: -6.70 to 13.16) from a mixed-effects model. Similarly, for thromboembolic/ischaemic events, the independence estimating equation estimate for the participant-average odds ratio was 0.43 (95% confidence interval: 0.07 to 2.48), compared with 0.33 (95% confidence interval: 0.06 to 1.77) from the cluster-average estimator.

CONCLUSION

In this re-analysis, we found that estimates from the various approaches could differ, which may be due to the presence of informative cluster size. Careful consideration of the estimand and the plausibility of assumptions underpinning each estimator can help ensure an appropriate analysis methods are used. Independence estimating equations and the analysis of cluster-level summaries (with appropriate weighting for each to correspond to either the participant-average or cluster-average treatment effect) are a desirable choice when informative cluster size is deemed possible, due to their unbiasedness in this setting.

Improving the mixed model for repeated measures to robustly increase precision in randomized trials

In randomized trials, repeated measures of the outcome are routinely collected. The mixed model for repeated measures (MMRM) leverages the information from these repeated outcome measures, and is often used for the primary analysis to estimate the average treatment effect at the primary endpoint. MMRM, however, can suffer from bias and precision loss when it models intermediate outcomes incorrectly, and hence fails to use the post-randomization information harmlessly. This paper proposes an extension of the commonly used MMRM, called IMMRM, that improves the robustness and optimizes the precision gain from covariate adjustment, stratified randomization, and adjustment for intermediate outcome measures. Under regularity conditions and missing completely at random, we prove that the IMMRM estimator for the average treatment effect is robust to arbitrary model misspecification and is asymptotically equal or more precise than the analysis of covariance (ANCOVA) estimator and the MMRM estimator. Under missing at random, IMMRM is less likely to be misspecified than MMRM, and we demonstrate via simulation studies that IMMRM continues to have less bias and smaller variance. Our results are further supported by a re-analysis of a randomized trial for the treatment of diabetes.

Is Early Childhood Development Care at Public Health Facilities in Pakistan Effective? A Cluster Randomized Controlled Trial

BACKGROUND

Significant brain development in children occurs from birth to 2 years, with environment playing an important role. Stimulation interventions are widely known to be effective in enhancing early childhood development (ECD). This study aims to assess the feasibility and effectiveness of integrating ECD care delivered by lady health visitors (LHVs) at public health facilities in rural Pakistan.

METHOD

A cluster randomized controlled trial was conducted through public health facilities in 2 districts of Punjab, Pakistan. A total of 22 clusters (rural health centers and subdistrict hospitals) were randomly allocated to receive routine care (control: n=11 clusters, 406 mother-child pairs) or counseling (intervention: n=11 clusters, 398 mother-child pairs). All children aged 11-12 months without any congenital abnormality were eligible for enrollment. The intervention was delivered by the LHVs to mothers with children aged 12-24 months in 3 quarterly sessions.

RESULTS

The primary outcome was the prevention of ECD delays in children aged 24 months (assessed with the Ages and Stages Questionnaire-3). Analysis was done on an intention-to-treat basis. A total of 804 mother-child pairs were registered in the study, of which 26 (3.3%) pairs were lost to follow-up at the endpoint. The proportion of children with 2 or more developmental delays was significantly less in the intervention arm (13%) as compared to the control arm (41%) at an endpoint (odds ratio=0.21; 95% confidence interval=0.11, 0.42). Children in the intervention arm also had significantly better anthropometric measurements when aged 24 months than the children in the control arm.

CONCLUSION

The integrated ECD care intervention for children aged 12-24 months at public health facilities was found to be effective in enhancing ECD and reducing the proportion of children with global development delays.

Bias-corrected and doubly robust inference for the three-level longitudinal cluster-randomized trials with missing continuous outcomes and small number of clusters: Simulation study and application to a study for adults with serious mental illnesses

Longitudinal cluster-randomized designs have been popular tools for comparative effective research in clinical trials. The methodologies for the three-level hierarchical design with longitudinal outcomes need to be better understood under more pragmatic settings; that is, with a small number of clusters, heterogeneous cluster sizes, and missing outcomes. Generalized estimating equations (GEEs) have been frequently used when the distribution of data and the correlation model are unknown. Standard GEEs lead to bias and an inflated type I error rate due to the small number of available clinics and non-completely random missing data in longitudinal outcomes. We evaluate the performance of inverse probability weighted (IPW) estimating equations, with and without augmentation, for two types of missing data in continuous outcomes and individual-level treatment allocation mechanisms combined with two bias-corrected variance estimators. Our intensive simulation results suggest that the proposed augmented IPW method with bias-corrected variance estimation successfully prevents the inflation of false positive findings and improves efficiency when the number of clinics is small, with moderate to severe missing outcomes. Our findings are expected to aid researchers in choosing appropriate analysis methods for three-level longitudinal cluster-randomized designs. The proposed approaches were applied to analyze data from a longitudinal cluster-randomized clinical trial involving adults with serious mental illnesses.

Restricted mean survival time to estimate an intervention effect in a cluster randomized trial

For time-to-event outcomes, the difference in restricted mean survival time is a measure of the intervention effect, an alternative to the hazard ratio, corresponding to the expected survival duration gain due to the intervention up to a predefined time t*. We extended two existing approaches of restricted mean survival time estimation for independent data to clustered data in the framework of cluster randomized trials: one based on the direct integration of Kaplan-Meier curves and the other based on pseudo-values regression. Then, we conducted a simulation study to assess and compare the statistical performance of the proposed methods, varying the number and size of clusters, the degree of clustering, and the magnitude of the intervention effect under proportional and non-proportional hazards assumption. We found that the extended methods well estimated the variance and controlled the type I error if there was a sufficient number of clusters (≥ 50) under both proportional and non-proportional hazards assumption. For cluster randomized trials with a limited number of clusters (< 50), a permutation test for pseudo-values regression was implemented and corrected the type I error. We also provided a procedure to estimate permutation-based confidence intervals which produced adequate coverage. All the extended methods performed similarly, but the pseudo-values regression offered the possibility to adjust for covariates. Finally, we illustrated each considered method with a cluster randomized trial evaluating the effectiveness of an asthma-control education program.

A flexible micro-randomized trial design and sample size considerations

Technological advancements have made it possible to deliver mobile health interventions to individuals. A novel framework that has emerged from such advancements is the just-in-time adaptive intervention, which aims to suggest the right support to the individuals when their needs arise. The micro-randomized trial design has been proposed recently to test the proximal effects of the components of these just-in-time adaptive interventions. However, the extant micro-randomized trial framework only considers components with a fixed number of categories added at the beginning of the study. We propose a more flexible micro-randomized trial design which allows addition of more categories to the components during the study. Note that the number and timing of the categories added during the study need to be fixed initially. The proposed design is motivated by collaboration on the Diabetes and Mental Health Adaptive Notification Tracking and Evaluation study, which learns to deliver effective text messages to encourage physical activity among patients with diabetes and depression. We developed a new test statistic and the corresponding sample size calculator for the flexible micro-randomized trial using an approach similar to the generalized estimating equation for longitudinal data. Simulation studies were conducted to evaluate the sample size calculators and an R shiny application for the calculators was developed.

Marginal proportional hazards models for clustered interval-censored data with time-dependent covariates

The Botswana Combination Prevention Project was a cluster-randomized HIV prevention trial whose follow-up period coincided with Botswana's national adoption of a universal test and treat strategy for HIV management. Of interest is whether, and to what extent, this change in policy modified the preventative effects of the study intervention. To address such questions, we adopt a stratified proportional hazards model for clustered interval-censored data with time-dependent covariates and develop a composite expectation maximization algorithm that facilitates estimation of model parameters without placing parametric assumptions on either the baseline hazard functions or the within-cluster dependence structure. We show that the resulting estimators for the regression parameters are consistent and asymptotically normal. We also propose and provide theoretical justification for the use of the profile composite likelihood function to construct a robust sandwich estimator for the variance. We characterize the finite-sample performance and robustness of these estimators through extensive simulation studies. Finally, we conclude by applying this stratified proportional hazards model to a re-analysis of the Botswana Combination Prevention Project, with the national adoption of a universal test and treat strategy now modeled as a time-dependent covariate.

Sample size considerations for micro-randomized trials with binary proximal outcomes

Micro-randomized trials (MRTs) are a novel experimental design for developing mobile health interventions. Participants are repeatedly randomized in an MRT, resulting in longitudinal data with time-varying treatments. Causal excursion effects are the main quantities of interest in MRT primary and secondary analyses. We consider MRTs where the proximal outcome is binary and the randomization probability is constant or time-varying but not data-dependent. We develop a sample size formula for detecting a nonzero marginal excursion effect. We prove that the formula guarantees power under a set of working assumptions. We demonstrate via simulation that violations of certain working assumptions do not affect the power, and for those that do, we point out the direction in which the power changes. We then propose practical guidelines for using the sample size formula. As an illustration, the formula is used to size an MRT on interventions for excessive drinking. The sample size calculator is implemented in R package MRTSampleSizeBinary and an interactive R Shiny app. This work can be used in trial planning for a wide range of MRTs with binary proximal outcomes.

Urschitz M, Rascher W. A Complex Intervention to Prevent Medication-Related Hospital Admissions

BACKGROUND

Children are often treated off-label and are at a disadvantage in pharmacotherapy. The aim of this study was to implement and evaluate a quality assurance measure (PaedPharm) for pediatric pharmacotherapy whose purpose is to reduce medication-related hospitalizations among children and adolescents.

METHODS

PaedPharm consisted of the digital pediatric drug information system PaedAMIS, pediatric pharmaceutical quality circles (PaedZirk), and an adverse drug event (ADE) reporting system (PaedReport). The intervention was implemented in a cluster-randomized trial (DRKS 00013924) in 12 regions, with a pediatric and adolescent medicine clinic in each and a total of 152 surrounding private practitioners, in 6 sequences over 8 quarters. In addition to the proportion of ADE-related hospital admissions (primary endpoint), comprehensive process evaluation included other endpoints such as coverage, user acceptance, and relevance to practice.

RESULTS

41 829 inpatient admissions were recorded, of which 5101 were patients of physicians who participated in our study. 4.1% of admissions were ADE-related under control conditions, and 3.1% under intervention conditions (95% CI: [2.3; 5.9] and [1.8; 4.5], respectively). A model-based comparison yielded an intervention effect of 0.73 (population-based odds ratio; [0.39; 1.37]; p = 0.33). PaedAMIS achieved moderate user acceptance and PaedZirk achieved high user acceptance.

CONCLUSION

The introduction of PaedPharm was associated with a decrease in medication-related hospitalizations that did not reach statistical significance. The process evaluation revealed broad acceptance of the intervention in outpatient pediatrics and adolescent medicine.

ORTH.Ord: An R package for analyzing correlated ordinal outcomes using alternating logistic regressions with orthogonalized residuals

BACKGROUND AND OBJECTIVES

Marginal models with generalized estimating equations (GEE) are usually recommended for analyzing correlated ordinal outcomes which are commonly seen in a longitudinal study or clustered randomized trial (CRT). Within-cluster association is often of interest in longitudinal studies or CRTs, and can be estimated with paired estimating equations. However, the estimators for within-cluster association parameters and variances may be subject to finite-sample biases when the number of clusters is small. The objective of this article is to introduce a newly developed R package ORTH.Ord for analyzing correlated ordinal outcomes using GEE models with finite-sample bias corrections.

METHODS

The R package ORTH.Ord implements a modified version of alternating logistic regressions with estimation based on orthogonalized residuals (ORTH), which use paired estimating equations to jointly estimate parameters in marginal mean and association models. The within-cluster association between ordinal responses is modeled by global pairwise odds ratios (POR). The R package also provides a finite-sample bias correction to POR parameter estimates based on matrix multiplicative adjusted orthogonalized residuals (MMORTH) for correcting estimating equations, and bias-corrected sandwich estimators with different options for covariance estimation.

RESULTS

A simulation study shows that MMORTH provides less biased global POR estimates and coverage of their 95% confidence intervals closer to the nominal level than uncorrected ORTH. An analysis of patient-reported outcomes from an orthognathic surgery clinical trial illustrates features of ORTH.Ord.

CONCLUSIONS

This article provides an overview of the ORTH method with bias-correction on both estimating equations and sandwich estimators for analyzing correlated ordinal data, describes the features of the ORTH.Ord R package, evaluates the performance of the package using a simulation study, and finally illustrates its application in an analysis of a clinical trial.

A comparison of bias-adjusted generalized estimating equations for sparse binary data in small-sample longitudinal studies

Using a generalized estimating equation (GEE) can lead to a bias in regression coefficients for a small sample or sparse data. The bias-corrected GEE (BCGEE) and penalized GEE (PGEE) were proposed to resolve the small-sample bias. Moreover, the standard sandwich covariance estimator leads to a bias of standard error for small samples; several modified covariance estimators have been proposed to address this issue. We review the modified GEEs and modified covariance estimators, and evaluate their performance in sparse binary data from small-sample longitudinal studies. The simulation results showed that GEE and BCGEE often failed to achieve convergence, whereas the convergence proportion for PGEE was quite high. The bias for the regression coefficients was generally in the ascending order of PGEE < $$ < $$ BCGEE < $$ < $$ GEE. However, PGEE and BCGEE did not sufficiently remove the bias involving 20-30 subjects with unequal exposure levels with a 5% response rate. The coverage probability (CP) of the confidence interval for BCGEE was relatively poor compared with GEE and PGEE. The CP with the sandwich covariance estimator deteriorated regardless of the GEE methods under the small sample size and low response rate, whereas the CP with the modified covariance estimators-such as Morel's method-was relatively acceptable. PGEE will be the reasonable way for analyzing sparse binary data in small-sample studies. Instead of using the standard sandwich covariance estimator, one should always apply the modified covariance estimators for analyzing these data.

The optimal pre-post allocation for randomized clinical trials

BACKGROUND

In pre-post designs, analysis of covariance (ANCOVA) is a standard technique to detect the treatment effect with a continuous variable measured at baseline and follow-up. For measurements subject to a high degree of variability, it may be advisable to repeat the pre-treatment and/or follow-up assessments. In general, repeating the follow-up measurements is more advantageous than repeating the pre-treatment measurements, while the latter can still be valuable and improve efficiency in clinical trials.

METHODS

In this article, we report investigations of using multiple pre-treatment and post-treatment measurements in randomized clinical trials. We consider the sample size formula for ANCOVA under general correlation structures with the pre-treatment mean included as the covariate and the mean follow-up value included as the response. We propose an optimal experimental design of multiple pre-post allocations under a specified constraint, that is, given the total number of pre-post treatment visits. The optimal number of the pre-treatment measurements is derived. For non-linear models, closed-form formulas for sample size/power calculations are generally unavailable, but we conduct Monte Carlo simulation studies instead.

RESULTS

Theoretical formulas and simulation studies show the benefits of repeating the pre-treatment measurements in pre-post randomized studies. The optimal pre-post allocation derived from the ANCOVA extends well to binary measurements in simulation studies, using logistic regression and generalized estimating equations (GEE).

CONCLUSIONS

Repeating baselines and follow-up assessments is a valuable and efficient technique in pre-post design. The proposed optimal pre-post allocation designs can minimize the sample size, i.e., achieve maximum power.

A Pay-It-Forward Approach to Improve Chlamydia and Gonorrhea Testing Uptake Among Female Sex Workers in China: Venue-Based Superiority Cluster Randomized Controlled Trial

BACKGROUND

Regular chlamydia and gonorrhea testing are essential for key populations, such as female sex workers (FSWs). However, testing cost, stigma, and lack of access prevent FSWs in low- and middle-income countries from receiving chlamydia and gonorrhea testing. A social innovation to address these problems is "pay it forward," where an individual receives a gift (free testing) and then asks whether they would like to give a gift to another person in the community.

OBJECTIVE

This cluster randomized controlled trial examined the effectiveness and cost of the pay-it-forward strategy in increasing access to chlamydia and gonorrhea testing among FSWs in China.

METHODS

This trial integrated a pay-it-forward approach into a community-based HIV outreach service. FSWs (aged 18 years or older) were invited by an outreach team from 4 Chinese cities (clusters) to receive free HIV testing. The 4 clusters were randomized into 2 study arms in a 1:1 ratio: a pay-it-forward arm (offered chlamydia and gonorrhea testing as a gift) and a standard-of-care arm (out-of-pocket cost for testing: US $11). The primary outcome was chlamydia and gonorrhea test uptake, as ascertained by administrative records. We conducted an economic evaluation using a microcosting approach from a health provider perspective, reporting our results in US dollars (at 2021 exchange rates).

RESULTS

Overall, 480 FSWs were recruited from 4 cities (120 per city). Most FSWs were aged ≥30 years (313/480, 65.2%), were married (283/480, 59%), had an annual income

CONCLUSIONS

The pay-it-forward strategy has the potential to enhance chlamydia and gonorrhea testing for Chinese FSWs and may be useful for scaling up preventive services. Further implementation research is needed to inform the transition of pay-it-forward research into practice.

TRIAL REGISTRATION

Chinese Clinical Trial Registry ChiCTR2000037653; https://www.chictr.org.cn/showprojen.aspx?proj=57233.

Collapse

Key Words

• China
• HIV
• STD
• care
• chlamydia
• community
• cost
• female sex workers
• gonorrhea
• implementation
• pay-it-forward
• prevention
• research
• stigma
• testing
• women

Collapse

MESH Headings

• Humans
• Female
• Sex Workers
• Sex Work
• Gonorrhea/diagnosis
• Gonorrhea/epidemiology
• China/epidemiology
• Chlamydia

Collapse

Grants

• 001 World Health Organization
• K24 AI143471 NIAID NIH HHS
• R01 AI158826 NIAID NIH HHS
• R34 MH119963 NIMH NIH HHS

Collapse

Affiliation(s)

Weiming Tang Dermatology Hospital of Southern Medical University, Guangzhou, China Southern Medical University Institute for Global Health, Guangzhou, China University of North Carolina at Chapel Hill Project-China, Guangzhou, China
Yewei Xie University of North Carolina at Chapel Hill Project-China, Guangzhou, China Health Service & System Research programme, Duke-NUS Medical School, Singapore, Singapore
Mingzhou Xiong Dermatology Hospital of Southern Medical University, Guangzhou, China Southern Medical University Institute for Global Health, Guangzhou, China
Dan Wu University of North Carolina at Chapel Hill Project-China, Guangzhou, China Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom West China School of Public Health, Sichuan University, Chengdu, China
Jason J Ong Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom Central Clinical School, Faculty of Medicine, Monash University, Melbourne, Australia
Teodora Elvira Wi Department of Global HIV, Hepatitis and STI Programmes, World Health Organization Headquarters, Geneva, Switzerland
Bin Yang Dermatology Hospital of Southern Medical University, Guangzhou, China Southern Medical University Institute for Global Health, Guangzhou, China
Joseph D Tucker University of North Carolina at Chapel Hill Project-China, Guangzhou, China Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom
Cheng Wang Dermatology Hospital of Southern Medical University, Guangzhou, China Southern Medical University Institute for Global Health, Guangzhou, China

Collapse

Improving sandwich variance estimation for marginal Cox analysis of cluster randomized trials

Cluster randomized trials (CRTs) frequently recruit a small number of clusters, therefore necessitating the application of small-sample corrections for valid inference. A recent systematic review indicated that CRTs reporting right-censored, time-to-event outcomes are not uncommon and that the marginal Cox proportional hazards model is one of the common approaches used for primary analysis. While small-sample corrections have been studied under marginal models with continuous, binary, and count outcomes, no prior research has been devoted to the development and evaluation of bias-corrected sandwich variance estimators when clustered time-to-event outcomes are analyzed by the marginal Cox model. To improve current practice, we propose nine bias-corrected sandwich variance estimators for the analysis of CRTs using the marginal Cox model and report on a simulation study to evaluate their small-sample properties. Our results indicate that the optimal choice of bias-corrected sandwich variance estimator for CRTs with survival outcomes can depend on the variability of cluster sizes and can also slightly differ whether it is evaluated according to relative bias or type I error rate. Finally, we illustrate the new variance estimators in a real-world CRT where the conclusion about intervention effectiveness differs depending on the use of small-sample bias corrections. The proposed sandwich variance estimators are implemented in an R package CoxBcv.

GEEMAEE: A SAS macro for the analysis of correlated outcomes based on GEE and finite-sample adjustments with application to cluster randomized trials

BACKGROUND AND OBJECTIVES

Generalized estimating equations (GEE) are used to analyze correlated outcomes in marginal regression models with population-averaged interpretations of exposure effects. Limitations of popular software for GEE include: (i) user choice is restricted to a small set of within-cluster pairwise correlation (intra-class correlation; ICC) structures; and (ii) inference on ICC parameters is usually not possible because the precision of their estimates is not quantified. This is important because ICC values inform the design of cluster randomized trials. Beyond the standard GEE implementation, use of paired estimating equations (Prentice 1988) provides: (i) flexible specification of models for pairwise correlations and (ii) standard errors for ICC estimates. However, most GEEs give biased estimates of standard errors and correlations when the number of clusters is small (roughly, ≤40). Consequently, there is a need for software to provide GEE analysis with finite-sample bias-corrections.

METHODS

The SAS macro GEEMAEE implements paired estimating equations to simultaneously estimate parameters in marginal mean and ICC models. It provides bias-corrected standard errors and uses matrix-adjusted estimating equations (MAEE) for bias-corrected estimation of correlations. Several built-in correlation matrix options, rarely found in software, are offered for multi-period, cluster randomized trials and similarly structured longitudinal observational data structures. Additional options include user-specified correlation structures and deletion diagnostics, namely Cooks' Distance and DBETA statistics that estimate the influence of observations, cluster-periods (when applicable) and clusters.

RESULTS

GEEMAEE is illustrated for a binary and a count outcome in two stepped wedge cluster randomized trials and a binary outcome in a longitudinal study of disease surveillance. Use of MAEE resulted in larger values of correlation estimates compared to uncorrected estimating equations. Use of bias-corrected variance estimators resulted in (appropriately) larger values of standard errors compared to the usual sandwich estimators. Deletion diagnostics identified the clusters and cluster-periods having the most influence.

CONCLUSIONS

The SAS macro GEEMAEE provides regression analysis for clustered or longitudinal responses, and is particularly useful when the number of clusters is small. Flexible specification and bias-corrected estimation of pairwise correlation parameters and standard errors are key features of the software to provide valid inference in real-world settings.

Implementing the WHO Labour Care Guide to reduce the use of Caesarean section in four hospitals in India: protocol and statistical analysis plan for a pragmatic, stepped-wedge, cluster-randomized pilot trial

BACKGROUND

The World Health Organization (WHO) Labour Care Guide (LCG) is a paper-based labour monitoring tool designed to facilitate the implementation of WHO's latest guidelines for effective, respectful care during labour and childbirth. Implementing the LCG into routine intrapartum care requires a strategy that improves healthcare provider practices during labour and childbirth. Such a strategy might optimize the use of Caesarean section (CS), along with potential benefits on the use of other obstetric interventions, maternal and perinatal health outcomes, and women's experience of care. However, the effects of a strategy to implement the LCG have not been evaluated in a randomised trial. This study aims to: (1) develop and optimise a strategy for implementing the LCG (formative phase); and (2) To evaluate the implementation of the LCG strategy compared with usual care (trial phase).

METHODS

In the formative phase, we will co-design the LCG strategy with key stakeholders informed by facility assessments and provider surveys, which will be field tested in one hospital. The LCG strategy includes a LCG training program, ongoing supportive supervision from senior clinical staff, and audit and feedback using the Robson Classification. We will then conduct a stepped-wedge, cluster-randomized pilot trial in four public hospitals in India, to evaluate the effect of the LCG strategy intervention compared to usual care (simplified WHO partograph). The primary outcome is the CS rate in nulliparous women with singleton, term, cephalic pregnancies in spontaneous labour (Robson Group 1). Secondary outcomes include clinical and process of care outcomes, as well as women's experience of care outcomes. We will also conduct a process evaluation during the trial, using standardized facility assessments, in-depth interviews and surveys with providers, audits of completed LCGs, labour ward observations and document reviews. An economic evaluation will consider implementation costs and cost-effectiveness.

DISCUSSION

Findings of this trial will guide clinicians, administrators and policymakers on how to effectively implement the LCG, and what (if any) effects the LCG strategy has on process of care, health and experience outcomes. The trial findings will inform the rollout of LCG internationally.

TRIAL REGISTRATION

CTRI/2021/01/030695 (Protocol version 1.4, 25 April 2022).

A general method for calculating power for GEE analysis of complete and incomplete stepped wedge cluster randomized trials

Stepped wedge designs have uni-directional crossovers at randomly assigned time points (steps) where clusters switch from control to intervention condition. Incomplete stepped wedge designs are increasingly used in cluster randomized trials of health care interventions and have periods without data collection due to logistical, resource and patient-centered considerations. The development of sample size formulae for stepped wedge trials has primarily focused on complete designs and continuous responses. Addressing this gap, a general, fast, non-simulation based power procedure is proposed for generalized estimating equations analysis of complete and incomplete stepped wedge designs and its predicted power is compared to simulated power for binary and continuous responses. An extensive set of simulations for six and twelve clusters is based upon the Connect-Home trial with an incomplete stepped wedge design. Results show that empirical test size is well controlled using a t-test with bias-corrected sandwich variance estimator for as few as six clusters. Analytical power agrees well with a simulated power in scenarios with twelve clusters. For six clusters, analytical power is similar to simulated power with estimation using the correctly specified model-based variance estimator. To explore the impact of study design choice on power, the proposed fast GEE power method is applied to the Connect-Home trial design, four alternative incomplete stepped wedge designs and one complete design.

Evaluation of a Complex Intervention to Strengthen Participation-Centred Care for Children with Special Healthcare Needs: Protocol of the Stepped Wedge Cluster Randomised PART-CHILD Trial

INTRODUCTION

Participation is an important dimension of healthy child development and is associated with higher self-rated health, educational attainment and civic engagement. Many children with special healthcare needs (SHCN) experience limited participation and are thus at risk for adverse health and developmental outcomes. Despite this, interventions that promote participation in healthcare are scarce. We therefore evaluate the effectiveness of a complex age- and condition-generic intervention that strengthens participation-centred care involving parents and their children with SHCN by, inter alia, assessing preferences, specifying participation goals and facilitating shared decision-making in care.

METHODS AND ANALYSIS

In this study protocol we describe the design and procedures for an unblinded, stepped wedge, cluster randomised trial conducted in 15 German interdisciplinary healthcare facilities providing services for children aged 0-18 years with SHCN. Sites are randomised to five periods in which they switch from control to intervention condition in blocks of three. The intervention includes: (1) team training focused on participation-centred care, (2) introduction of a new software facilitating participation-focused documentation and (3) implementation support promoting the transfer of training content into routine care. Study sites deliver routine care while in the control condition. As primary outcome, the degree of perceived shared decision-making with parents (CollaboRATE^pediatric parent scale), a potential antecedent of achieving participation goals in everyday life, is assessed on one randomly selected day per week during the entire study period, directly following care appointments. We aim to sample 70 parents per study site and period. Additionally, participation of children is assessed within a closed embedded cohort with three parent and patient surveys. Intervention effectiveness will be modelled with a marginal model for correlated binary outcomes using generalised estimation equations and complete cases. A comprehensive mixed-methods process evaluation complements the effectiveness analyses.

Evaluation of the whole school restorative practices project: One-year impact on discipline incidents

The current study addressed the need for experimental research on Restorative Practices (RP) in its evaluation of the Morningside Center for Teaching Social Responsibility's Whole School RP Project. The study was conducted in a large Northeastern city using a cluster randomized controlled design in 18 elementary, middle, and high schools. In a single year, before the COVID-19 pandemic, and with data from 5878 students, the study found that overall, students in the RP Project schools were less likely to receive a discipline incident record (11.1%) as compared to students in the comparison schools (18.2%). However, differential treatment effects based on sex, race/ethnicity, and disability status were not found. The findings suggest prevention efforts are a cornerstone of practice/policy reforms to reduce the use of exclusionary discipline. Findings also suggest multi-year initiatives are needed to address discipline disparities.

Design and analysis of cluster randomized trials with time-to-event outcomes under the additive hazards mixed model

A primary focus of current methods for cluster randomized trials (CRTs) has been for continuous, binary, and count outcomes, with relatively less attention given to right-censored, time-to-event outcomes. In this article, we detail considerations for sample size requirement and statistical inference in CRTs with time-to-event outcomes when the intervention effect parameter is specified through the additive hazards mixed model (AHMM), which includes a frailty term to explicitly account for the dependency between the failure times. First, we discuss improved inference for the treatment effect parameter via bias-corrected sandwich variance estimators and randomization-based test under AHMM, addressing potential small-sample biases in CRTs. Next, we derive a new sample size formula for AHMM analysis of CRTs accommodating both equal and unequal cluster sizes. When the cluster sizes vary, our sample size formula depends on the mean and coefficient of variation of cluster sizes, based on which we articulate the impact of cluster size variation in CRTs with time-to-event outcomes. Furthermore, we obtain the insight that the classical variance inflation factor for CRTs with a non-censored outcome can in fact apply to CRTs with a time-to-event outcome, providing that an appropriate definition of the intraclass correlation coefficient is considered under AHMM. Simulation studies are carried out to illustrate key design and analysis considerations in CRTs with a small to moderate number of clusters. The proposed sample size procedure and analytical methods are further illustrated using the context of the STrategies to Reduce Injuries and Develop Confidence in Elders CRT.

A Multilevel Integrated Intervention to Reduce the Impact of HIV Stigma on HIV Treatment Outcomes Among Adolescents Living With HIV in Uganda: Protocol for a Randomized Controlled Trial

BACKGROUND

HIV stigma remains a formidable barrier to HIV treatment adherence among school-attending adolescents living with HIV, owing to high levels of HIV stigma within schools, rigid school structures and routines, lack of adherence support, and food insecurity. Thus, this protocol paper presents an evidence-informed multilevel intervention that will simultaneously address family- and school-related barriers to HIV treatment adherence and care engagement among adolescents living with HIV attending boarding schools in Uganda.

OBJECTIVE

The proposed intervention-Multilevel Suubi (MSuubi)-has the following objectives: examine the impact of M-Suubi on HIV viral suppression (primary outcome) and adherence to HIV treatment, including keeping appointments, pharmacy refills, pill counts, and retention in care; examine the effect of M-Suubi on HIV stigma (internalized, anticipated, and enacted), with secondary analyses to explore hypothesized mechanisms of change (eg, depression) and intervention mediation; assess the cost and cost-effectiveness of each intervention condition; and qualitatively examine participants' experiences with HIV stigma, HIV treatment adherence, and intervention and educators' attitudes toward adolescents living with HIV and experiences with group-based HIV stigma reduction for educators, and program or policy implementation after training.

METHODS

MSuubi is a 5-year multilevel mixed methods randomized controlled trial targeting adolescents living with HIV aged 10 to 17 years enrolled in a primary or secondary school with a boarding section. This longitudinal study will use a 3-arm cluster randomized design across 42 HIV clinics in southwestern Uganda. Participants will be randomized at the clinic level to 1 of the 3 study conditions (n=14 schools; n=280 students per study arm). These include the bolstered usual care (consisting of the literature on antiretroviral therapy adherence promotion and stigma reduction), multiple family groups for HIV stigma reduction plus family economic empowerment (MFG-HIVSR plus FEE), and Group-based HIV stigma reduction for educators (GED-HIVSR). Adolescents randomized to the GED-HIVSR treatment arm will also receive the MFG-HIVSR plus FEE treatment. MSuubi will be provided for 20 months, with assessments at baseline and 12, 24, and 36 months.

RESULTS

This study was funded in September 2021. Participant screening and recruitment began in April 2022, with 158 dyads enrolled as of May 2022. Dissemination of the main study findings is anticipated in 2025.

CONCLUSIONS

MSuubi will assess the effects of a combined intervention (family-based economic empowerment, financial literacy education, and school-based HIV stigma) on HIV stigma among adolescents living with HIV in Uganda. The results will expand our understanding of effective intervention strategies for reducing stigma among HIV-infected and noninfected populations in Uganda and improving HIV treatment outcomes among adolescents living with HIV in sub-Saharan Africa.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05307250; https://clinicaltrials.gov/ct2/show/NCT05307250.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/40101.

Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial

BACKGROUND

The HEALing (Helping to End Addiction Long-term^SM) Communities Study (HCS) is a multi-site parallel group cluster randomized wait-list comparison trial designed to evaluate the effect of the Communities That Heal (CTH) intervention compared to usual care on opioid overdose deaths. Covariate-constrained randomization (CCR) was applied to balance the community-level baseline covariates in the HCS. The purpose of this paper is to evaluate the performance of model-based tests and permutation tests in the HCS setting. We conducted a simulation study to evaluate type I error rates and power for model-based and permutation tests for the multi-site HCS as well as for a subgroup analysis of a single state (Massachusetts). We also investigated whether the maximum degree of imbalance in the CCR design has an impact on the performance of the tests.

METHODS

The primary outcome, the number of opioid overdose deaths, is count data assessed at the community level that will be analyzed using a negative binomial regression model. We conducted a simulation study to evaluate the type I error rates and power for 3 tests: (1) Wald-type t-test with small-sample corrected empirical standard error estimates, (2) Wald-type z-test with model-based standard error estimates, and (3) permutation test with test statistics calculated by the difference in average residuals for the two groups.

RESULTS

Our simulation results demonstrated that Wald-type t-tests with small-sample corrected empirical standard error estimates from the negative binomial regression model maintained proper type I error. Wald-type z-tests with model-based standard error estimates were anti-conservative. Permutation tests preserved type I error rates if the constrained space was not too small. For all tests, the power was high to detect the hypothesized 40% reduction in opioid overdose deaths for the intervention vs. comparison group both for the overall HCS and the subgroup analysis of Massachusetts (MA).

CONCLUSIONS

Based on the results of our simulation study, the Wald-type t-test with small-sample corrected empirical standard error estimates from a negative binomial regression model is a valid and appropriate approach for analyzing cluster-level count data from the HEALing Communities Study.

TRIAL REGISTRATION

ClinicalTrials.gov http://www.

CLINICALTRIALS

gov ; Identifier: NCT04111939.

Cluster randomised trials with a binary outcome and a small number of clusters: comparison of individual and cluster level analysis method

BACKGROUND

Cluster randomised trials (CRTs) are often designed with a small number of clusters, but it is not clear which analysis methods are optimal when the outcome is binary. This simulation study aimed to determine (i) whether cluster-level analysis (CL), generalised linear mixed models (GLMM), and generalised estimating equations with sandwich variance (GEE) approaches maintain acceptable type-one error including the impact of non-normality of cluster effects and low prevalence, and if so (ii) which methods have the greatest power. We simulated CRTs with 8-30 clusters, altering the cluster-size, outcome prevalence, intracluster correlation coefficient, and cluster effect distribution. We analysed each dataset with weighted and unweighted CL; GLMM with adaptive quadrature and restricted pseudolikelihood; GEE with Kauermann-and-Carroll and Fay-and-Graubard sandwich variance using independent and exchangeable working correlation matrices. P-values were from a t-distribution with degrees of freedom (DoF) as clusters minus cluster-level parameters; GLMM pseudolikelihood also used Satterthwaite and Kenward-Roger DoF.

RESULTS

Unweighted CL, GLMM pseudolikelihood, and Fay-and-Graubard GEE with independent or exchangeable working correlation matrix controlled type-one error in > 97% scenarios with clusters minus parameters DoF. Cluster-effect distribution and prevalence of outcome did not usually affect analysis method performance. GEE had the least power. With 20-30 clusters, GLMM had greater power than CL with varying cluster-size but similar power otherwise; with fewer clusters, GLMM had lower power with common cluster-size, similar power with medium variation, and greater power with large variation in cluster-size.

CONCLUSION

We recommend that CRTs with ≤ 30 clusters and a binary outcome use an unweighted CL or restricted pseudolikelihood GLMM both with DoF clusters minus cluster-level parameters.

Marginal modeling of cluster-period means and intraclass correlations in stepped wedge designs with binary outcomes

Stepped wedge cluster randomized trials (SW-CRTs) with binary outcomes are increasingly used in prevention and implementation studies. Marginal models represent a flexible tool for analyzing SW-CRTs with population-averaged interpretations, but the joint estimation of the mean and intraclass correlation coefficients (ICCs) can be computationally intensive due to large cluster-period sizes. Motivated by the need for marginal inference in SW-CRTs, we propose a simple and efficient estimating equations approach to analyze cluster-period means. We show that the quasi-score for the marginal mean defined from individual-level observations can be reformulated as the quasi-score for the same marginal mean defined from the cluster-period means. An additional mapping of the individual-level ICCs into correlations for the cluster-period means further provides a rigorous justification for the cluster-period approach. The proposed approach addresses a long-recognized computational burden associated with estimating equations defined based on individual-level observations, and enables fast point and interval estimation of the intervention effect and correlations. We further propose matrix-adjusted estimating equations to improve the finite-sample inference for ICCs. By providing a valid approach to estimate ICCs within the class of generalized linear models for correlated binary outcomes, this article operationalizes key recommendations from the CONSORT extension to SW-CRTs, including the reporting of ICCs.

Transmission of SARS-CoV-2 among children and staff in German daycare centres

In daycare centres, the close contact of children with other children and employees favours the transmission of infections. The majority of children <6 years attend daycare programmes in Germany, but the role of daycare centres in the SARS-CoV-2 pandemic is unclear. We investigated the transmission risk in daycare centres and the spread of SARS-CoV-2 to associated households. 30 daycare groups with at least one recent laboratory-confirmed SARS-CoV-2 case were enrolled in the study (10/2020–06/2021). Close contact persons within daycare and households were examined over a 12-day period (repeated SARS-CoV-2 PCR tests, genetic sequencing of viruses, symptom diary). Households were interviewed to gain comprehensive information on each outbreak. We determined primary cases for all daycare groups. The number of secondary cases varied considerably between daycare groups. The pooled secondary attack rate (SAR) across all 30 daycare centres was 9.6%. The SAR tended to be higher when the Alpha variant was detected (15.9% vs. 5.1% with evidence of wild type). The household SAR was 53.3%. Exposed daycare children were less likely to get infected with SARS-CoV-2 than employees (7.7% vs. 15.5%). Containment measures in daycare programmes are critical to reduce SARS-CoV-2 transmission, especially to avoid spread to associated households.

Consideration of the adaptive randomization allocation ratio in the presence of treatment group heteroscedasticity in clinical trials

For randomized clinical trials, subjects' variance structures may vary over time among treatment groups, resulting in the heteroscedasticity of residuals in a regression analysis. Commonly used methods that assume equal variance among all treatment groups may not be able to control for a type I error. When the variances are indeed the same across treatment groups, an equal randomization allocation ratio will yield the greatest study power. However, out of ethical concern or urgent need for rare disease clinical trials, more patients may have to be allocated to the study drug arm. In these situations, an unequal randomization ratio should be considered. We propose a group variance-covariance and structures-based method to adapt the randomization ratio after interim analysis. We use simulations to compare commonly used statistical methods for continuous endpoints in assessing the impact of heteroscedasticity in equal and unequal randomization ratios and examine the extent to which the findings are affected by missing data.

Accounting for Heteroskedasticity Resulting from Between-Group Differences in Multilevel Models

Homogeneity of variance (HOV) is a well-known but often untested assumption in the context of multilevel models (MLMs). However, depending on how large the violation is, how different group sizes are, and the variance pairing, standard errors can be over or underestimated even when using MLMs, resulting in questionable inferential tests. We evaluate several tests (e.g., the H statistic, Breusch Pagan, Levene's test) that can be used with MLMs to assess violations of HOV. Although the traditional robust standard errors used with MLMs require at least 50 clusters to be effective, we assess a robust standard error adjustment (i.e., the CR2 estimator) that can be used even with a few clusters. Findings are assessed using a Monte Carlo simulation and are further illustrated using an applied example. We show that explicitly modeling the heterogenous variance structures or using the CR2 estimator are both effective at ameliorating the issues associated with the fixed effects of the regression model related to violations of HOV resulting from between-group differences.

Clustered restricted mean survival time regression

For multicenter randomized trials or multilevel observational studies, the Cox regression model has long been the primary approach to study the effects of covariates on time-to-event outcomes. A critical assumption of the Cox model is the proportionality of the hazard functions for modeled covariates, violations of which can result in ambiguous interpretations of the hazard ratio estimates. To address this issue, the restricted mean survival time (RMST), defined as the mean survival time up to a fixed time in a target population, has been recommended as a model-free target parameter. In this article, we generalize the RMST regression model to clustered data by directly modeling the RMST as a continuous function of restriction times with covariates while properly accounting for within-cluster correlations to achieve valid inference. The proposed method estimates regression coefficients via weighted generalized estimating equations, coupled with a cluster-robust sandwich variance estimator to achieve asymptotically valid inference with a sufficient number of clusters. In small-sample scenarios where a limited number of clusters are available, however, the proposed sandwich variance estimator can exhibit negative bias in capturing the variability of regression coefficient estimates. To overcome this limitation, we further propose and examine bias-corrected sandwich variance estimators to reduce the negative bias of the cluster-robust sandwich variance estimator. We study the finite-sample operating characteristics of proposed methods through simulations and reanalyze two multicenter randomized trials.

Stepped Wedge Cluster Randomized Trials: A Methodological Overview

BACKGROUND

Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials.

METHODS

We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs.

RESULTS

We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed.

CONCLUSIONS

Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.

Receiver operating characteristic estimation and threshold selection criteria in three-class classification problems for clustered data

Statistical evaluation of diagnostic tests, and, more generally, of biomarkers, is a constantly developing field, in which complexity of the assessment increases with the complexity of the design under which data are collected. One particularly prevalent type of data is clustered data, where individual units are naturally nested into clusters. In these cases, Bias can arise from omission, in the evaluation process, of cluster-level effects and/or individual covariates. Focusing on the three-class case and for continuous-valued diagnostic tests, we investigate how to exploit the clustered structure of data within a linear-mixed model approach, both when the assumption of normality holds and when it does not. We provide a method for the estimation of covariate-specific receiver operating characteristic surfaces and discuss methods for the choice of optimal thresholds, proposing three possible estimators. A proof of consistency and asymptotic normality of the proposed threshold estimators is given. All considered methods are evaluated by extensive simulation experiments. As an application, we study the use of the Lysosomal Associated Membrane Protein Family Member 5 gene expression as a biomarker to distinguish among three types of glutamatergic neurons.

Finite-sample adjustments in variance estimators for clustered competing risks regression

The marginal Fine-Gray proportional subdistribution hazards model is a popular approach to directly study the association between covariates and the cumulative incidence function with clustered competing risks data, which often arise in multicenter randomized trials or multilevel observational studies. To account for the within-cluster correlations between failure times, the uncertainty of the regression parameters estimators is quantified by the robust sandwich variance estimator, which may have unsatisfactory performance with a limited number of clusters. To overcome this limitation, we propose four bias-corrected variance estimators to reduce the negative bias of the usual sandwich variance estimator, extending the bias-correction techniques from generalized estimating equations with noncensored exponential family outcomes to clustered competing risks outcomes. We further compare their finite-sample operating characteristics through simulations and two real data examples. In particular, we found the Mancl and DeRouen (MD) type sandwich variance estimator generally has the smallest bias. Furthermore, with a small number of clusters, the Wald t -confidence interval with the MD sandwich variance estimator carries close to nominal coverage for the cluster-level effect parameter. The t -confidence intervals based on the sandwich variance estimator with any one of the three types of multiplicative bias correction or the z -confidence interval with the Morel, Bokossa and Neerchal (MBN) type sandwich variance estimator have close to nominal coverage for the individual-level effect parameter. Finally, we develop a user-friendly R package crrcbcv implementing the proposed sandwich variance estimators to assist practical applications.

A weighted Jackknife approach utilizing linear model based-estimators for clustered data

Small number of clusters combined with cluster level heterogeneity poses a great challenge for the data analysis. We have published a weighted Jackknife approach to address this issue applying weighted cluster means as the basic estimators. The current study proposes a new version of the weighted delete-one-cluster Jackknife analytic framework, which employs Ordinary Least Squares or Generalized Least Squares estimators as the fundamentals. Algorithms for computing estimated variances of the study estimators have also been derived. Wald test statistics can be further obtained, and the statistical comparison in the outcome means of two conditions is determined using the cluster permutation procedure. The simulation studies show that the proposed framework produces estimates with higher precision and improved power for statistical hypothesis testing compared to other methods.