Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis

Selecting outcomes for pragmatic clinical trials in dementia care: The IMPACT Collaboratory iLibrary

BACKGROUND

Many interventions improve care and outcomes for people with Alzheimer's Disease and related dementias (ADRD), yet are never disseminated. Pragmatic trials facilitate the adoption and dissemination of best practices, but gaps in pragmatic outcome measurement are a critical obstacle. Our objectives are (1) to describe the development and structure of the IMbedded Pragmatic ADRD Clinical Trials Collaboratory (IMPACT) iLibrary of potential outcome measures for ADRD pragmatic trials, and (2) to assess their pragmatic characteristics.

METHODS

We identified potential outcome measures from several sources: a database of administrative and clinical outcome measures from ADRD clinical trials registered in ClinicalTrials.gov, published reviews, and IMPACT pilot pragmatic trial outcome measures. The iLibrary reports (a) number of items, (b) completion time, (c) readability for diverse populations, (d) cost or copyright barriers to use, (e) method of administration, (f) assessor training burden, and (g) feasibility of data capture and interpretation in routine care; a summary of pragmatic characteristics of each outcome measure (high, moderate, low); items or descriptions of items; and links to primary citations regarding development or psychometric properties.

RESULTS

We included 140 outcome measures in the iLibrary: 66 administrative (100% were pragmatic) and 74 clinical (52% were pragmatic). The most commonly addressed outcome domains from administrative assessments included physical function, quality of care or communication concerns, and psychological symptoms or distress behaviors. The most commonly addressed outcome domains from clinical assessments were psychological symptoms or distress behaviors, physical function, cognitive function, and health-related quality of life.

CONCLUSIONS

Pragmatic outcome measures are brief, meaningful to diverse populations, easily scored and interpreted by clinicians, and available in electronic format for analysis. The iLibrary can facilitate the selection of measures for a wide range of outcomes relevant to people with ADRD and their care partners.

Involving patients and the public In sTatistIcal Analysis pLans (INITIAL): A delphi survey

BACKGROUND

Patient and public involvement (PPI) in trials aims to enhance research by improving its relevance and transparency. Planning for statistical analysis begins at the design stage of a trial within the protocol and is refined and detailed in a Statistical Analysis Plan (SAP). While PPI is common in design and protocol development it is less common within SAPs. This study aimed to reach consensus on the most important and relevant statistical analysis items within an SAP to involve patients and the public.

METHODS

We developed a UK-based, two-round Delphi survey through an iterative consultation with public partners, statisticians, and trialists. The consultation process started with 55 items from international guidance for statistical analysis plans. We aimed to recruit at least 20 participants per key stakeholder group for inclusion in the final analysis of the Delphi survey. Participants were asked to vote on each item using a Likert scale from 1 to 9, where a rating of 1 to 3 was labelled as having 'limited importance'; 4 to 6 as 'important but not critical' and 7 to 9 as 'critical' to involve patients and the public. Results from the second round determined consensus on critical items for PPI.

RESULTS

The consultation exercise led to the inclusion of 15 statistical items in the Delphi survey. We recruited 179 participants, of whom 72% (129: 36 statisticians, 29 patients or public partners, 25 clinical researchers or methodologists, 27 trial managers, and 12 PPI coordinators) completed both rounds. Participants were on average 48 years old, 60% were female, 84% were White, 64% were based in England and 84% had at least five years' experience in trials. Four items reached consensus regarding critical importance for patient and public involvement: presentation of results to trial participants; summary and presentation of harms; interpretation and presentation of findings in an academic setting; factors impacting how well a treatment works. No consensus was reached for the remaining 11 items. In general, the results were consistent across stakeholder groups.

DISCUSSION

We identified four critical items to involve patients and the public in statistical analysis plans. The remaining 11 items did not reach consensus and need to be considered in a case-by-case basis with most responders considering patient and public involvement important (but not critical). Our research provides a platform to enable focused future efforts to improve patient and public involvement in trials and enhance the relevance of statistical analyses to patients and the public.

Patient and public involvement in pragmatic trials: online survey of corresponding authors of published trials

BACKGROUND

There are few data on patient and public involvement (PPI) in pragmatic trials. We aimed to describe the prevalence and nature of PPI within pragmatic trials, describe variation in prevalence of PPI by trial characteristics and compare prevalence of PPI reported by trial authors to that reported in trial publications.

METHODS

We applied a search filter to identify pragmatic trials published from 2014 to 2019 in MEDLINE. We invited the corresponding authors of pragmatic trials to participate in an online survey about their specific trial.

RESULTS

Of 3163 authors invited, 2585 invitations were delivered, 710 (27.5%) reported on 710 unique trials and completed the survey; 334 (47.0%) conducted PPI. Among those who conducted PPI, for many the aim was to increase the research relevance (86.3%) or quality (76.5%). Most PPI partners were engaged at protocol development stages (79.1%) and contributed to the co-design of interventions (70.9%) or recruitment or retention strategies (60.5%). Patient and public involvement was more common among trials involving children, trials conducted in the United Kingdom, cluster randomized trials, those explicitly labelled as "pragmatic" in the study manuscript, and more recent trials. Less than one-quarter of trials (22.8%) that reported PPI in the survey also reported PPI in the trial manuscript.

INTERPRETATION

Nearly half of trialists in this survey reported conducting PPI and listed several benefits of doing so, but researchers who did not conduct PPI often cited a lack of requirement for it. Patient and public involvement appears to be significantly underreported in trial publications. Consistent and standardized reporting is needed to promote transparency about PPI methods, outcomes, challenges and benefits.

Observed intervention effects for mortality in randomised clinical trials: a methodological study protocol

INTRODUCTION

It is essential to choose a realistic anticipated intervention effect when calculating a sample size for a randomised clinical trial. Unfortunately, anticipated intervention effects are often inflated, when compared with the 'true' intervention effects. This is documented for mortality in critical care trials. A similar pattern might exist across different medical specialties. This study aims to estimate the range of observed intervention effects for all-cause mortality in trials included in Cochrane Reviews, within each Cochrane Review Group.

METHODS AND ANALYSIS

We will include randomised clinical trials assessing all-cause mortality as an outcome. Trials will be identified from Cochrane Reviews published in the Cochrane Database of Systematic Reviews. Cochrane Reviews will be clustered according to the registered Cochrane Review Group (eg, Anaesthesia, Emergency and Critical Care) and the statistical analyses will be conducted for each Cochrane Review Group and overall. The median relative risk and IQR for all-cause mortality and the proportion of trials with a relative all-cause mortality risk within seven different ranges will be reported (relative risk below 0.70, 0.70-0.79, 0.80-0.89, 0.90-1.09, 1.10-1.19, 1.20-1.30 and above 1.30). Subgroup analyses will explore the effects of original design, sample size, risk of bias, disease, intervention type, follow-up length, participating centres, funding type, information size and outcome hierarchy.

ETHICS AND DISSEMINATION

Since we will use summary data from trials already approved by relevant ethical committees, this study does not require ethical approval. Regardless of our findings, the results will be published in an international peer-reviewed journal.

Health equity considerations in pragmatic trials in Alzheimer's and dementia disease: Results from a methodological review

Introduction

To improve dementia care delivery for persons across all backgrounds, it is imperative that health equity is integrated into pragmatic trials.

Methods

We reviewed 62 pragmatic trials of people with dementia published 2014 to 2019. We assessed health equity in the objectives; design, conduct, analysis; and reporting using PROGRESS-Plus which stands for Place of residence, Race/ethnicity, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital, and other factors such as age and disability.

Results

Two (3.2%) trials incorporated equity considerations into their objectives; nine (14.5%) engaged with communities; 4 (6.5%) described steps to increase enrollment from equity-relevant groups. Almost all trials (59, 95.2%) assessed baseline balance for at least one PROGRESS-Plus characteristic, but only 10 (16.1%) presented subgroup analyses across such characteristics. Differential recruitment, attrition, implementation, adherence, and applicability across PROGRESS-Plus were seldom discussed.

Discussion

Ongoing and future pragmatic trials should more rigorously integrate equity considerations in their design, conduct, and reporting.

Highlights

Few pragmatic trials are explicitly designed to inform equity-relevant objectives.Few pragmatic trials take steps to increase enrollment from equity-relevant groups.Disaggregated results across equity-relevant groups are seldom reported.Adherence to existing tools (e.g., IMPACT Best Practices, CONSORT-Equity) is key.

How do we know a treatment is good enough? A survey of non-inferiority trials

BACKGROUND

Non-inferiority and equivalence trials aim to determine whether a new treatment is good enough (non-inferior) or as good as (equivalent to) another treatment. To inform the decision about non-inferiority or equivalence, a margin is used. We aimed to identify the current methods used to determine non-inferiority or equivalence margins, as well as the main challenges and suggestions from trialists.

METHODS

We developed an online questionnaire that included both closed and open-ended questions about methods to elicit non-inferiority or equivalence margins, underlying principles, and challenges and suggestions for improvement. We recruited trialists with experience of determining a margin by contacting corresponding authors for non-inferiority or equivalence trials. We used descriptive statistics and content analysis to identify categories in qualitative data.

RESULTS

We had forty-one responses, all from non-inferiority trials. More than half of the trials were non-pharmacological (n = 21, 51%), and the most common primary outcome was clinical (n = 29, 71%). The two most used methods to determine the margin were as follows: a review of the evidence base (n = 27, 66%) and opinion seeking methods (n = 24, 59%). From those using reviews, the majority used systematic reviews or reviews of multiple RCTs to determine the margin (n = 17, 63%). From those using opinion seeking methods, the majority involved clinicians with or without other professionals (n = 19, 79%). Respondents reported that patients' opinions on the margin were sought in four trials (16%). Median confidence in overall quality of the margin was 5 out of 7 (maximum confidence); however, around a quarter of the respondents were "completely unconfident" that the margin reflected patient's views. We identified "stakeholder involvement" as the most common category to determine respondent's confidence in the quality of the margins and whether it reflected stakeholder's views. The most common suggestion to improve the definition of margins was "development of methods to involve stakeholders," and the most common challenge identified was "communication of margins."

CONCLUSIONS

Responders highlighted the need for clearer guidelines on defining a margin, more and better stakeholder involvement in its selection, and better communication tools that enable discussions about non-inferiority trials with stakeholders. Future research should focus on developing best practice recommendations.

Reporting of and explanations for under-recruitment and over-recruitment in pragmatic trials: a secondary analysis of a database of primary trial reports published from 2014 to 2019

OBJECTIVES

To describe the extent to which pragmatic trials underachieved or overachieved their target sample sizes, examine explanations and identify characteristics associated with under-recruitment and over-recruitment.

STUDY DESIGN AND SETTING

Secondary analysis of an existing database of primary trial reports published during 2014-2019, registered in ClinicalTrials.gov, self-labelled as pragmatic and with target and achieved sample sizes available.

RESULTS

Of 372 eligible trials, the prevalence of under-recruitment (achieving <90% of target sample size) was 71 (19.1%) and of over-recruitment (>110% of target) was 87 (23.4%). Under-recruiting trials commonly acknowledged that they did not achieve their targets (51, 71.8%), with the majority providing an explanation, but only 11 (12.6%) over-recruiting trials acknowledged recruitment excess. The prevalence of under-recruitment in individually randomised versus cluster randomised trials was 41 (17.0%) and 30 (22.9%), respectively; prevalence of over-recruitment was 39 (16.2%) vs 48 (36.7%), respectively. Overall, 101 025 participants were recruited to trials that did not achieve at least 90% of their target sample size. When considering trials with over-recruitment, the total number of participants recruited in excess of the target was a median (Q1-Q3) 319 (75-1478) per trial for an overall total of 555 309 more participants than targeted. In multinomial logistic regression, cluster randomisation and lower journal impact factor were significantly associated with both under-recruitment and over-recruitment, while using exclusively routinely collected data and educational/behavioural interventions were significantly associated with over-recruitment; we were unable to detect significant associations with obtaining consent, publication year, country of recruitment or public engagement.

CONCLUSIONS

A clear explanation for under-recruitment or over-recruitment in pragmatic trials should be provided to encourage transparency in research, and to inform recruitment to future trials with comparable designs. The issues and ethical implications of over-recruitment should be more widely recognised by trialists, particularly when designing cluster randomised trials.

A Longitudinal Study of Trajectories and Factors Influencing Patient-Reported Outcomes in Chronic Obstructive Pulmonary Disease

PURPOSE

To explore the trajectory of patient-reported outcomes and the factors influencing them in patients with COPD.

PATIENTS AND METHODS

The study population, 236 patients with stable COPD who attended the outpatient clinic of the Department of Respiratory and Critical Care Medicine in a tertiary care hospital in Nanning City between October 2020 and November 2021, answered the modified patient-reported outcome scale for COPD (mCOPD-PRO). Patient-reported outcomes were investigated at the time of the patient's outpatient visit (T1), 1 month after the visit (T2), 3 months after the visit (T3), and 6 months after the visit (T4). Latent class growth modeling was used to determine the number and shape of trajectories, and multinomial logistic regression analysis were used to explore influence factors of each class.

RESULTS

COPD patients' reported outcome trajectories were classified into 3 categories: health low-level group (14.80%), health risk group (54.70%), and good health group (30.50%). Logistic regression analysis showed that gender, BMI, smoking history, number of comorbidities, whether it was their first visit, and lung function classification were influential factors in patients' reported outcome trajectories (P<0.05). Female, obese, had a history of smoking, number of comorbid diseases >3, first diagnosis, and lung function class IV had a higher probability of entering the healthy low-level group.

CONCLUSION

COPD patients have poor self-reported health levels during the first 6 months after the outpatient visit, and there is group heterogeneity in patient-reported outcome trajectories; medical staff should give patients specific nursing interventions based on their current development of COPD, self-reported changes, and other relevant influencing factors.

Power analysis for cluster randomized trials with continuous co-primary endpoints

Pragmatic trials evaluating health care interventions often adopt cluster randomization due to scientific or logistical considerations. Systematic reviews have shown that co-primary endpoints are not uncommon in pragmatic trials but are seldom recognized in sample size or power calculations. While methods for power analysis based on K (K≥ 2) binary co-primary endpoints are available for cluster randomized trials (CRTs), to our knowledge, methods for continuous co-primary endpoints are not yet available. Assuming a multivariate linear mixed model that accounts for multiple types of intraclass correlation coefficients among the observations in each cluster, we derive the closed-form joint distribution of K treatment effect estimators to facilitate sample size and power determination with different types of null hypotheses under equal cluster sizes. We characterize the relationship between the power of each test and different types of correlation parameters. We further relax the equal cluster size assumption and approximate the joint distribution of the K treatment effect estimators through the mean and coefficient of variation of cluster sizes. Our simulation studies with a finite number of clusters indicate that the predicted power by our method agrees well with the empirical power, when the parameters in the multivariate linear mixed model are estimated via the expectation-maximization algorithm. An application to a real CRT is presented to illustrate the proposed method. This article is protected by copyright. All rights reserved.