Generalizability of the findings from a randomized controlled trial of a web-based substance use disorder intervention

Enhancing the psychological well-being and sleep quality of healthcare providers with a multimodal psychological support program: a randomized controlled trial

Background

The COVID-19 pandemic significantly challenged the global healthcare system, especially frontline healthcare professionals, such as those working in intensive care units (ICUs). In late 2022, a sudden increase in COVID-19 cases in China led to a large number of ICU admissions, requiring new ICU staff (non-ICU professionals to work in ICUs), exacerbating their stress. This study aimed to develop an effective stress management strategy for new ICU professionals, focusing on reducing the detrimental effects of stress on their psychological state. We hypothesized that the online multimodal psychological support (MPS) program might improve the psychological well-being and sleep quality of the participants.

Methods

This single-center, single-blind randomized controlled trial included new ICU staff during the COVID-19 pandemic. Participants were randomly assigned to either an intervention (online psychological support, MPS) or a control (routine wellness care, RWC) group for 28 days, and assessments were conducted before intervention (baseline), after intervention, and at the 1-month follow-up. The intervention included music therapy, sleep hygiene education, psychoeducation, and relaxation training, tailored to address common psychiatric issues experienced by healthcare professionals during the pandemic. The primary outcome was a DASS-21 score 28 days after the end of the intervention.

Results

One hundred and one professionals eventually participated in the study, 47 in the MPS group and 54 in the RWC group. No significant differences were observed in the overall psychological well-being immediately after the end of the intervention. However, the MPS group showed improved sleep and sustained lower stress levels, anxiety, and depression scores at the 1-month follow-up, significantly improving the severity of insomnia (marginal mean difference -2.028; SE 1.00; p = 0.044).

Conclusion

The online multimodal psychological support program effectively enhanced the psychological well-being and sleep quality of new ICU staff demonstrating the potential of off line training in managing stress and improving health outcomes during crises. The findings of this study emphasize the importance of accessible, flexible psychological support, especially in high-stress environments such as ICUs during pandemics.

Generalizability of clinical trial efficacy results to a real-world population: An example in migraine prevention

BACKGROUND

Health care decision makers are often concerned about the external validity of randomized controlled trials (RCTs), as their results may not apply to certain patients in the real world who intend to receive treatment.

OBJECTIVE

To demonstrate a methodology for assessing the generalizability of clinical trial results to a real-world population, before sufficient and appropriate real-world effectiveness data are available, using individual patient-level data from an RCT and aggregated baseline data from a real-world French registry in migraine.

METHODS

The analyses were conducted in 2 steps. First, individual patient-level baseline data from the multinational CONQUER RCT were weighted to match aggregated real-world InovPain registry patient characteristic data. Matched patient characteristics were sex, age, migraine type and duration, number of monthly migraine headache days, and number of monthly headache days at baseline. Second, the weighted CONQUER patient data were used to reanalyze the primary endpoint of CONQUER (least squares mean change from baseline in the number of monthly migraine headache days during the 3-month double-blind treatment phase) using predefined methodology. Sensitivity analyses were conducted to assess the robustness of findings.

RESULTS

A total of 462 patients with migraine were randomized and treated with galcanezumab or placebo in CONQUER; aggregated InovPain data were available from 130 patients with migraine. We identified no important differences in baseline patient characteristics between the 2 prespecified populations, suggesting good external validity for CONQUER. Although this limited the extent of observed differences between the original and matched CONQUER populations, weighting of CONQUER data did help harmonize the 2 datasets and allow the results obtained in CONQUER to be generalized to patients more representative of the real-world French population with migraine. Results of weighted analyses suggested that galcanezumab would be superior to placebo for reducing monthly migraine headache days in a clinical trial in patients with episodic or chronic migraine who reflected the characteristics of patients eligible to receive the drug in France.

CONCLUSIONS

Findings suggest that our methods may be helpful for assessing the generalizability of clinical trial results to a real-world population before the availability of substantial real-world clinical data.

Two-stage matching-adjusted indirect comparison

BACKGROUND

Anchored covariate-adjusted indirect comparisons inform reimbursement decisions where there are no head-to-head trials between the treatments of interest, there is a common comparator arm shared by the studies, and there are patient-level data limitations. Matching-adjusted indirect comparison (MAIC), based on propensity score weighting, is the most widely used covariate-adjusted indirect comparison method in health technology assessment. MAIC has poor precision and is inefficient when the effective sample size after weighting is small.

METHODS

A modular extension to MAIC, termed two-stage matching-adjusted indirect comparison (2SMAIC), is proposed. This uses two parametric models. One estimates the treatment assignment mechanism in the study with individual patient data (IPD), the other estimates the trial assignment mechanism. The first model produces inverse probability weights that are combined with the odds weights produced by the second model. The resulting weights seek to balance covariates between treatment arms and across studies. A simulation study provides proof-of-principle in an indirect comparison performed across two randomized trials. Nevertheless, 2SMAIC can be applied in situations where the IPD trial is observational, by including potential confounders in the treatment assignment model. The simulation study also explores the use of weight truncation in combination with MAIC for the first time.

RESULTS

Despite enforcing randomization and knowing the true treatment assignment mechanism in the IPD trial, 2SMAIC yields improved precision and efficiency with respect to MAIC in all scenarios, while maintaining similarly low levels of bias. The two-stage approach is effective when sample sizes in the IPD trial are low, as it controls for chance imbalances in prognostic baseline covariates between study arms. It is not as effective when overlap between the trials' target populations is poor and the extremity of the weights is high. In these scenarios, truncation leads to substantial precision and efficiency gains but induces considerable bias. The combination of a two-stage approach with truncation produces the highest precision and efficiency improvements.

CONCLUSIONS

Two-stage approaches to MAIC can increase precision and efficiency with respect to the standard approach by adjusting for empirical imbalances in prognostic covariates in the IPD trial. Further modules could be incorporated for additional variance reduction or to account for missingness and non-compliance in the IPD trial.

Data management in substance use disorder treatment research: Implications from data harmonization of National Institute on Drug Abuse-funded randomized controlled trials

Background:

Secondary analysis of data from completed randomized controlled trials is a critical and efficient way to maximize the potential benefits from past research. De-identified primary data from completed randomized controlled trials have been increasingly available in recent years; however, the lack of standardized data products is a major barrier to further use of these valuable data. Pre-statistical harmonization of data structure, variables, and codebooks across randomized controlled trials would facilitate secondary data analysis, including meta-analyses and comparative effectiveness studies. We describe a pre-statistical data harmonization initiative to standardize de-identified primary data from substance use disorder treatment randomized controlled trials funded by the National Institute on Drug Abuse available on the National Institute on Drug Abuse Data Share website.

Methods:

Standardized datasets and codebooks with consistent data structures, variable names, labels, and definitions were developed for 36 completed randomized controlled trials. Common data domains were identified to bundle data files from individual randomized controlled trials according to relevant concepts. Variables were harmonized if at least two randomized controlled trials used the same instruments. The structures of the harmonized data were determined based on the feedback from clinical trialists and substance use disorder research experts.

Results:

We have created a harmonized database of variables across 36 randomized controlled trials with a build-in label and a brief definition for each variable. Data files from the randomized controlled trials have been consistently categorized into eight domains (enrollment, demographics, adherence, adverse events, physical health measures, mental-behavioral-cognitive health measures, self-reported substance use measures, and biologic substance use measures). Standardized codebooks and concordance tables have also been developed to help identify instruments and variables of interest more easily.

Conclusion:

The harmonized data of randomized controlled trials of substance use disorder treatments can potentially promote future secondary data analysis of completed randomized controlled trials, allowing combining data from multiple randomized controlled trials and provide guidance for future randomized controlled trials in substance use disorder treatment research.

Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long‐standing concern when applying trial results to real‐world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real‐world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.