Using Hybrid Effectiveness Studies to Facilitate Implementation in Community-Based Settings: Three Case Studies in Dementia Care Research

The pipeline from discovery to testing and then implementing evidence-based innovations in real-world contexts may take 2 decades or more to achieve. Implementation science innovations, such as hybrid studies that combine effectiveness and implementation research questions, may help to bridge the chasm between intervention testing and implementation in dementia care. This paper describes hybrid effectiveness studies and presents 3 examples of dementia care interventions conducted in various community-based settings. Studies that focus on outcomes and implementation processes simultaneously may result in a truncated and more efficient implementation pipeline, thereby providing older persons, their families, health care providers, and communities with the best evidence to improve quality of life and care more rapidly. We offer post-acute and long-term care researchers considerations related to study design, sampling, data collection, and analysis that they can apply to their own dementia and other chronic disease care investigations.

Coordination Between Primary Care and Women, Infants, and Children to Prevent Obesity for Infants from Low-Income Families: A Pragmatic Randomized Clinical Trial

Background: Rapid weight gain during infancy is associated with risk for later obesity, yet little research to date has examined the effect of a responsive parenting (RP) intervention with care coordination between pediatric primary care providers and Women, Infants, and Children nutritionists on infant weight. Methods: The Women, Infants, and Children Enhancements to Early Healthy Lifestyles for Baby (WEE Baby) Care study is a pragmatic, randomized clinical trial for mothers and infants (n = 288) designed to examine the effect of a patient-centered RP intervention that used advanced health information technology strategies to coordinate care to reduce rapid infant weight gain compared with standard care. General linear models examined intervention effects on infant conditional weight gain scores, weight-for-age z scores, BMI, and overweight status (BMI-for-age ≥85th percentile) from birth to age 6 months, and mothers' use of food to soothe from age 2 to 6 months. Results: There were no intervention effects on infant conditional weight gain scores or overweight status at 6 months. Infants in the RP intervention had lower mean weight-for-age z scores [M = -0.04, standard error (SE) = 0.04 vs. M = 0.05, SE = 0.04; p = 0.008] and lower mean BMI (M = 16.05, SE = 0.09 vs. M = 16.24, SE = 0.09; p = 0.03) compared with standard care. Mothers' use of emotion-based food to soothe was lower in the RP intervention compared with standard care from age 2 to 6 months [M difference = -0.32, standard deviation (SD) = 0.81 vs. 0.00, SD = 0.90; p = 0.01]. Conclusions: This pragmatic, patient-centered RP intervention did not reduce rapid infant weight gain or overweight but was associated with modestly lower infant BMI and reduced mothers' use of emotion-based food to soothe. Trial Registration: clinicaltrials.gov identifier: NCT03482908.

Career development in pragmatic clinical trials to improve care for people living with dementia

The growing number of people living with dementia (PLWD) requires a coordinated clinical response to deliver pragmatic, evidence-based interventions in frontline care settings. However, infrastructure to support such a response is lacking. Moreover, there are too few researchers conducting rigorous embedded pragmatic clinical trials (ePCTs) to make the vision of high quality, widely accessible dementia care a reality. National Institute on Aging (NIA) Imbedded Pragmatic Alzheimer's disease and Related Dementias Clinical Trials (IMPACT) Collaboratory seeks to improve the pipeline of early career researchers qualified to lead ePCTs by funding career development awards. Even with support from the Collaboratory, awardees face practical and methodological challenges to success, recently exacerbated by the COVID-19 pandemic. We first describe the training opportunities and support network for the IMPACT CDA recipients. This report then describes the unique career development challenges faced by early-career researchers involved in ePCTs for dementia care. Topics addressed include challenges in establishing a laboratory, academic promotion, mentoring and professional development, and work-life balance. Concrete suggestions to address these challenges are offered for early-career investigators, their mentors, and their supporting institutions. While some of these challenges are faced by researchers in other fields, this report seeks to provide a roadmap for expanding the work of the IMPACT Collaboratory and initiating future efforts to recruit, train, and retain talented early-career researchers involved in ePCTs for dementia care.

Real World Data Studies of Antineoplastic Drugs: How Can They Be Improved to Steer Everyday Use in the Clinic?

There is a growing interest in real world evidence when developing antineoplastic drugs owing to the shorter length of time and low costs compared to randomised controlled trials. External validity of studies in the regulatory phase can be enhanced by complementing randomised controlled trials with real world evidence. Furthermore, the use of real world evidence ensures the inclusion of patients often excluded from randomised controlled trials such as the elderly, certain ethnicities or those from certain geographical areas. This review explores approaches in which real world data may be integrated with randomised controlled trials. One approach is by using big data, especially when investigating drugs in the antineoplastic setting. This can even inform artificial intelligence thus ensuring faster and more precise diagnosis and treatment decisions. Pragmatic trials also offer an approach to examine the effectiveness of novel antineoplastic drugs without evading the benefits of randomised controlled trials. A well-designed pragmatic trial would yield results with high external validity by employing a simple study design with a large sample size and diverse settings. Although randomised controlled trials can determine efficacy of antineoplastic drugs, effectiveness in the real world may differ. The need for pragmatic trials to help guide healthcare decision-making led to the development of trials within cohorts (TWICs). TWICs make use of cohorts to conduct multiple randomised controlled trials while maintaining characteristics of real world data in routine clinical practice. Although real world data is often affected by incomplete data and biases such as selection and unmeasured biases, the use of big data and pragmatic approaches can improve the use of real world data in the development of antineoplastic drugs that can in turn steer decision-making in clinical practice.

Internet Versus Noninternet Participation in a Decentralized Clinical Trial: Lessons From the ADAPTABLE Study

Background Internet-based participation has the potential to enhance pragmatic and decentralized trials, where representative study populations and generalizability to clinical practice are key. We aimed to study the differences between internet and noninternet/telephone participants in a large remote, pragmatic trial. Methods and Results In a subanalysis of the ADAPTABLE (Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness) study, we compared internet participants with those who opted for noninternet participation. Study process measures examined included participant characteristics at consent, study medication adherence, and study retention. The clinical outcome examined was a composite of all-cause mortality, hospitalization for myocardial infarction, or hospitalization for stroke. Noninternet participants were older (mean 69.4 versus 67.4 years), more likely to be female (38.9% versus 30.2%), more likely to be Black (27.3% versus 6.0%) or Hispanic (11.1% versus 2.0%), and had a higher number of comorbid conditions. The composite clinical outcome was more than twice as high in noninternet participants. The hazard of nonadherence to the assigned aspirin dosage was 46% higher in noninternet participants than internet participants. Conclusions Noninternet participants differed from internet participants in notable demographic characteristics while having poorer baseline health. Over the course of ADAPTABLE, they also had worse clinical outcomes and greater likelihood of study drug nonadherence. These results suggest that trials focused on internet participation select for younger, healthier participants with a higher proportion of traditionally overrepresented patients. Allowing noninternet participation enhances diversity; however, additional steps may be needed to promote study retention and study medication adherence. Registration Information clinicaltrials.gov. Identifier: NCT02697916.

Research objectives and general considerations for pragmatic clinical trials of pain treatments: IMMPACT statement

ABSTRACT

Many questions regarding the clinical management of people experiencing pain and related health policy decision-making may best be answered by pragmatic controlled trials. To generate clinically relevant and widely applicable findings, such trials aim to reproduce elements of routine clinical care or are embedded within clinical workflows. In contrast with traditional efficacy trials, pragmatic trials are intended to address a broader set of external validity questions critical for stakeholders (clinicians, healthcare leaders, policymakers, insurers, and patients) in considering the adoption and use of evidence-based treatments in daily clinical care. This article summarizes methodological considerations for pragmatic trials, mainly concerning methods of fundamental importance to the internal validity of trials. The relationship between these methods and common pragmatic trials methods and goals is considered, recognizing that the resulting trial designs are highly dependent on the specific research question under investigation. The basis of this statement was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) systematic review of methods and a consensus meeting. The meeting was organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership. The consensus process was informed by expert presentations, panel and consensus discussions, and a preparatory systematic review. In the context of pragmatic trials of pain treatments, we present fundamental considerations for the planning phase of pragmatic trials, including the specification of trial objectives, the selection of adequate designs, and methods to enhance internal validity while maintaining the ability to answer pragmatic research questions.

Outcomes among patients with peripheral artery disease in the Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness (ADAPTABLE) study

BACKGROUND

We aimed to understand the effects of aspirin dose on outcomes in patients with peripheral artery disease (PAD) as well as their participation in a pragmatic randomized controlled trial.

METHODS

In a subanalysis of the Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness (ADAPTABLE) study, we compared aspirin doses (81 vs 325 mg) among participants with PAD and study participation metrics in patients with and without PAD. The primary outcome composite was all-cause mortality, nonfatal myocardial infarction, and nonfatal stroke.

RESULTS

Among 14,662 participants enrolled in ADAPTABLE with PAD status available, 3493 (23.8%) had PAD. Participants with PAD were more likely to experience the primary composite (13.76% vs 5.31%, p < 0.001), all-cause mortality (7.55% vs 3.01%, p < 0.001), myocardial infarction (5.71% vs 2.09%, p < 0.001), stroke (2.45% vs 0.86%, p < 0.001), and major bleeding (1.19% vs 0.44%, p < 0.001). A higher aspirin dose did not reduce the primary outcome in patients with PAD (13.68% vs 13.84% in 81 mg and 325 mg groups; OR 1.05, 95% CI 0.88-1.25). Participants with PAD were less likely to enroll via email (33.0% vs 41.9%, p < 0.0001), less likely to choose internet follow-up (79.2% vs 89.5%, p < 0.0001), and were more likely to change their aspirin doses (39.7% vs 30.7%, p < 0.0001).

CONCLUSIONS

ADAPTABLE participants with PAD did not benefit from a higher dose of aspirin and participated in the study differently from those without PAD. These results reinforce the need for additional PAD-specific research and suggest that different trial strategies may be needed for optimal engagement of patients with PAD. (ClinicalTrials.gov Identifier: NCT02697916).

Importance of patient engagement in the conduct of pragmatic multicenter randomized controlled trials: The ADAPTABLE experience

BACKGROUND/AIMS

Actively engaging patient partners in the conduct of trials is crucial to ensure the studies answer genuine, patient-centered, unmet clinical needs, and to facilitate participant recruitment and retention. The aim of this article is to demonstrate the feasibility of patient engagement within a large pragmatic multicenter randomized controlled trial, specifically for the purposes of dissemination of study information/updates and to favorize recruitment and retention.

METHODS

In the patient-centric, pragmatic ADAPTABLE randomized trial, transparent and timely dissemination of information on the study updates to the trial participants was undertaken to create meaningful engagement and to facilitate retention. A national panel of patient partners, the Adaptors, were directly involved in this information dissemination strategy, and study participants were engaged both nationally and locally to design recruitment methods iteratively during the conduct of the trial. All Adaptors had a lived experience with cardiovascular disease.

RESULTS

Adaptors attended bi-weekly meetings facilitated by the director of the study's patient-powered research network. They drafted and/or edited newsletters and ad hoc educational information written in a lay-friendly manner for study participants, which were regularly distributed to the ADAPTABLE community, in addition to online forums where participants could share their experience of their involvement in ADAPTABLE. To spur recruitment, a patient-driven initiative was to draft letters sharing their story, which were distributed by the local study teams. Patient partners thought that using patients' voice to provide their perspectives on why they believed this project was important would be more engaging for prospective participants than traditional approaches.

CONCLUSIONS

ADAPTABLE's experience has demonstrated the feasibility of engaging patients as partners in the conduct of a large-scale, multi-center, pragmatic randomized controlled trial. Future trials should embrace and iteratively improve this model by engaging patient partners as early as study protocol development and funding applications, and quantify its impact on the effectiveness and value of the trial.

Strategies for implementing the interRAI home care frailty scale with home-delivered meal clients

Introduction

Frailty is a complex condition that is highly associated with health decline and the loss of independence. Home-delivered meal programs are designed to provide older adults with health and nutritional support that can attenuate the risk of frailty. However, home-delivered meal agencies do not routinely assess frailty using standardized instruments, leading to uncertainty over the longitudinal impact of home-delivered meals on frailty levels. Considering this knowledge gap, this study aimed to facilitate home-delivered meal staff's implementation of a standardized frailty instrument with meal clients as part of routine programming. This article (a) describes the use of Implementation Mapping principles to develop strategies supporting frailty instrument implementation in one home-delivered meal agency and (b) examines the degree to which a combination of strategies influenced the feasibility of frailty instrument use by home-delivered meal staff at multiple time points.

Methods and materials

This retrospective observational study evaluated staff's implementation of the interRAI Home Care Frailty Scale (HCFS) with newly enrolled home-delivered meal clients at baseline-, 3-months, and 6-months. The process of implementing the HCFS was supported by five implementation strategies that were developed based on tenets of Implementation Mapping. Rates of implementation and reasons clients were lost to 3- and 6-month follow-up were evaluated using univariate analyses. Client-level data were also examined to identify demographic factors associated with attrition at both follow-up time points.

Results

Staff implemented the HCFS with 94.8% (n = 561) of eligible home-delivered meal clients at baseline. Of those clients with baseline HCFS data, staff implemented the follow-up HCFS with 43% of clients (n = 241) at 3-months and 18.0% of clients (n = 101) at 6-months. Insufficient client tracking and documentation procedures complicated staff's ability to complete the HCFS at follow-up time points.

Discussion

While the HCFS assesses important frailty domains that are relevant to home-delivered meal clients, its longitudinal implementation was complicated by several agency- and client-level factors that limited the extent to which the HCFS could be feasibly implemented over multiple time points. Future empirical studies are needed to design and test theoretically derived implementation strategies to support frailty instrument use in the home- and community-based service setting.

Health equity considerations in pragmatic trials in Alzheimer's and dementia disease: Results from a methodological review

Introduction

To improve dementia care delivery for persons across all backgrounds, it is imperative that health equity is integrated into pragmatic trials.

Methods

We reviewed 62 pragmatic trials of people with dementia published 2014 to 2019. We assessed health equity in the objectives; design, conduct, analysis; and reporting using PROGRESS-Plus which stands for Place of residence, Race/ethnicity, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital, and other factors such as age and disability.

Results

Two (3.2%) trials incorporated equity considerations into their objectives; nine (14.5%) engaged with communities; 4 (6.5%) described steps to increase enrollment from equity-relevant groups. Almost all trials (59, 95.2%) assessed baseline balance for at least one PROGRESS-Plus characteristic, but only 10 (16.1%) presented subgroup analyses across such characteristics. Differential recruitment, attrition, implementation, adherence, and applicability across PROGRESS-Plus were seldom discussed.

Discussion

Ongoing and future pragmatic trials should more rigorously integrate equity considerations in their design, conduct, and reporting.

Highlights

Few pragmatic trials are explicitly designed to inform equity-relevant objectives.Few pragmatic trials take steps to increase enrollment from equity-relevant groups.Disaggregated results across equity-relevant groups are seldom reported.Adherence to existing tools (e.g., IMPACT Best Practices, CONSORT-Equity) is key.

Key considerations for choosing a statistical method to deal with incomplete treatment adherence in pragmatic trials

Pragmatic trials offer practical means of obtaining real-world evidence to help improve decision-making in comparative effectiveness settings. Unfortunately, incomplete adherence is a common problem in pragmatic trials. The commonly used methods in randomized control trials often cannot handle the added complexity imposed by incomplete adherence, resulting in biased estimates. Several naive methods and advanced causal inference methods (e.g., inverse probability weighting and instrumental variable-based approaches) have been used in the literature to deal with incomplete adherence. Practitioners and applied researchers are often confused about which method to consider under a given setting. This current work is aimed to review commonly used statistical methods to deal with non-adherence along with their key assumptions, advantages, and limitations, with a particular focus on pragmatic trials. We have listed the applicable settings for these methods and provided a summary of available software. All methods were applied to two hypothetical datasets to demonstrate how these methods perform in a given scenario, along with the R codes. The key considerations include the type of intervention strategy (point treatment settings, where treatment is administered only once versus sustained treatment settings, where treatment has to be continued over time) and availability of data (e.g., the extent of measured or unmeasured covariates that are associated with adherence, dependent confounding impacted by past treatment, and potential violation of assumptions). This study will guide practitioners and applied researchers to use the appropriate statistical method to address incomplete adherence in pragmatic trial settings for both the point and sustained treatment strategies.

Pragmatic platform trials to improve the outcome of patients with acute kidney injury

PURPOSE OF REVIEW

There is an important need for improved diagnostic strategies and treatment among patients with acute kidney injury (AKI). Classical randomized clinical trials have generated relevant results in AKI but are associated with shortcomings, such as high costs and sometimes lack of generalizability. In this minireview, we discuss the value and limits of pragmatic trials and platform trials for AKI research.

RECENT FINDINGS

The implementation of pragmatic and platform trials in critical care settings has generated relevant clinical evidence impacting clinical practice. Pragmatic and platform designs have recently been applied to patients at risk of AKI and represent a crucial opportunity to advance our understanding of optimized treatment and strategies in patients at risk of AKI or presenting with AKI. Trials embedded in electronic health records can facilitate patient enrollment and data collection. Platform trials have allowed for a more efficient study design. Although both pragmatic and platform trials have several advantages, they also come with the challenges and shortcomings discussed in this review.

SUMMARY

Pragmatic and platform trials can provide clinical answers in 'real-life' settings, facilitate a significant sample size enrollment at a limited cost, and provide results that can have a faster implementation in clinical practice.

The Long-Term Care Data Cooperative: The Next Generation of Data Integration

Despite important advances in the linkage of residents' Medicare claims and Minimum Data Set (MDS) information, the data infrastructure for long-term care remains inadequate for public health surveillance and clinical research. It is widely known that the evidence base supporting treatment decisions for older nursing home residents is scant as residents are systematically excluded from clinical trials. Electronic health records (EHRs) hold the promise to improve this population's representation in clinical research, especially with the more timely and detailed clinical information available in EHRs that are lacking in claims and MDS. The COVID-19 pandemic shined a spotlight on the data gap in nursing homes. To address this need, the National Institute on Aging funded the Long-Term Care (LTC) Data Cooperative, a collaboration among providers and stakeholders in academia, government, and the private sector. The LTC Data Cooperative assembles residents' EHRs from major specialty vendors and facilitates linkage of these data with Medicare claims to create a comprehensive, longitudinal patient record. These data serve 4 key purposes: (1) health care operations and population health analytics; (2) public health surveillance; (3) observational, comparative effectiveness research; and (4) clinical research studies, including provider and patient recruitment into Phase 3 and Phase 4 randomized trials. Federally funded researchers wanting to conduct pragmatic trials can now enroll their partnering sites in this Cooperative to more easily access the clinical data needed to close the evidence gaps in LTC. Linkage to Medicare data facilitates tracking patients' long-term outcomes after being discharged back to the community. As of August 2022, nearly 1000 nursing homes have joined, feedback reports to facilities are being piloted, algorithms for identifying infections are being tested, and proposals for use of the data have been reviewed and approved. This emerging EHR system is a substantial innovation in the richness and timeliness of the data infrastructure of the nursing home population.

Effect of the STRIDE fall injury prevention intervention on falls, fall injuries, and health-related quality of life

BACKGROUND

Falls are common in older adults and can lead to severe injuries. The Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) trial cluster-randomized 86 primary care practices across 10 health systems to a multifactorial intervention to prevent fall injuries, delivered by registered nurses trained as falls care managers, or enhanced usual care. STRIDE enrolled 5451 community-dwelling older adults age ≥70 at increased fall injury risk.

METHODS

We assessed fall-related outcomes via telephone interviews of participants (or proxies) every 4 months. At baseline, 12 and 24 months, we assessed health-related quality of life (HRQOL) using the EQ-5D-5L and EQ-VAS. We used Poisson models to assess intervention effects on falls, fall-related fractures, fall injuries leading to hospital admission, and fall injuries leading to medical attention. We used hierarchical longitudinal linear models to assess HRQOL.

RESULTS

For recurrent event models, intervention versus control incidence rate ratios were 0.97 (95% confidence interval [CI], 0.93-1.00; p = 0.048) for falls, 0.93 (95% CI, 0.80-1.08; p = 0.337) for self-reported fractures, 0.89 (95% CI, 0.73-1.07; p = 0.205) for adjudicated fractures, 0.91 (95% CI, 0.77-1.07; p = 0.263) for falls leading to hospital admission, and 0.97 (95% CI, 0.89-1.06; p = 0.477) for falls leading to medical attention. Similar effect sizes (non-significant) were obtained for dichotomous outcomes (e.g., participants with ≥1 events). The difference in least square mean change over time in EQ-5D-5L (intervention minus control) was 0.009 (95% CI, -0.002 to 0.019; p = 0.106) at 12 months and 0.005 (95% CI, -0.006 to 0.015; p = 0.384) at 24 months.

CONCLUSIONS

Across a standard set of outcomes typically reported in fall prevention studies, we observed modest improvements, one of which was statistically significant. Future work should focus on patient-, practice-, and organization-level operational strategies to increase the real-world effectiveness of interventions, and improving the ability to detect small but potentially meaningful clinical effects.

CLINICALTRIALS

gov identifier: NCT02475850.

Pragmatic Trials in Long-Term Care: Implementation and Dissemination Challenges and Opportunities

Randomized controlled trials are considered the most rigorous research design in efficacy and effectiveness research; however, such trials present numerous challenges that limit their applicability in real-world settings. As a consequence, pragmatic trials are increasingly viewed as a research design that overcomes some of these barriers with the potential to produce findings that are more reproducible. Although pragmatic methodology in long-term care is receiving increasing attention as an approach to improve successful dissemination and implementation, pragmatic trials present complexities of their own. To address these complexities and related issues, experts with experience conducting pragmatic trials, developing nursing home policy, participating in advocacy efforts, and providing clinical care in long-term care settings participated in a virtual consensus conference funded by the National Institute on Aging in Spring 2021. Participants identified 4 cross-cutting principles key to dissemination and implementation of pragmatic trial interventions: (1) stakeholder engagement, (2) diversity and inclusion, (3) organizational strain and readiness, and (4) learn from adaptations. Participants emphasized that implementation processes must be grounded in the perspectives of the people who will ultimately be responsible for implementing the intervention once it is proven to be effective. In addition, messaging must speak to long-term care staff and all others who have a stake in its outcomes. Although our understanding of dissemination and implementation strategies remains underdeveloped, this article is designed to guide long-term care researchers and community providers who are increasingly aware of the need for pragmatism in disseminating and implementing evidence-based care interventions.

Pragmatic Trials in Long-Term Care: Research Challenges and Potential Solutions in Relation to Key Areas of Care

As a method of research, pragmatic trials are recommended so as to generate results that are applicable to real-world care. This intent is especially important for the millions of older adults who receive long-term care in thousands of nursing homes and assisted living communities across the country-and many millions more around the globe. This article presents key points raised by experts participating in a conference funded by the National Institute of Aging held at the 2021 conference of the Society for Post-Acute and Long-term Care Medicine. The purpose of the conference was to convene leading clinicians, researchers, and industry partners to address special considerations of pragmatic trials in long-term care. Cross-cutting and unique challenges and solutions to conducting pragmatic trials were discussed focusing on 3 areas of clinical relevance to long-term care: (1) functional care and outcomes, (2) psychosocial care and quality of life, and (3) medical care and outcomes, with a special focus on persons with dementia. Challenges and innovative solutions were organized across the 9 domains of the revised Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) Tool, and future research recommendations for pragmatic trials in long-term care were identified.

Blood Pressure Visit Intensification in Treatment (BP-Visit) Findings: a Pragmatic Stepped Wedge Cluster Randomized Trial

BACKGROUND

Shortening time between office visits for patients with uncontrolled hypertension represents a potential strategy for improving blood pressure (BP).

OBJECTIVE

We evaluated the impact of multimodal strategies on time between visits and on improvement in systolic BP (SBP) among patients with uncontrolled hypertension.

DESIGN

We used a stepped-wedge cluster randomized controlled trial with three wedges involving 12 federally qualified health centers with three study periods: pre-intervention, intervention, and post-intervention.

PARTICIPANTS

Adult patients with diagnosed hypertension and two BPs ≥ 140/90 pre-randomization and at least one visit during post-randomization control period (N = 4277).

INTERVENTION

The core intervention included three, clinician hypertension group-based trainings, monthly clinician feedback reports, and monthly meetings with practice champions to facilitate implementation.

MAIN MEASURES

The main measures were change in time between visits when BP was not controlled and change in SBP. A secondary planned outcome was changed in BP control among all hypertension patients in the practices.

KEY RESULTS

Median follow-up times were 34, 32, and 32 days and the mean SBPs were 142.0, 139.5, and 139.8 mmHg, respectively. In adjusted analyses, the intervention did not improve time to the next visit compared with control periods, HR = 1.01 (95% CI: 0.98, 1.04). SBP was reduced by 1.13 mmHg (95% CI: -2.10, -0.16), but was not maintained during follow-up. Hypertension control (< 140/90) in the practices improved by 5% during intervention (95% CI: 2.6%, 7.3%) and was sustained post-intervention 5.4% (95% CI: 2.6%, 8.2%).

CONCLUSIONS

The intervention failed to shorten follow-up time for patients with uncontrolled BP and showed very small, statistically significant improvements in SBP that were not sustained. However, the intervention showed statistically and clinically relevant improvement in hypertension control suggesting that the intervention affected clinician decision-making regarding BP control apart from visit frequency. Future practice initiatives should consider hypertension control as a primary outcome.

CLINICAL TRIAL

www.ClinicalTrials.gov Identifier: NCT02164331.

Pragmatic trials of pain therapies: a systematic review of methods

ABSTRACT

Pragmatic randomised clinical trials aim to directly inform clinical or health policy decision making. Here, we systematically review methods and design of pragmatic trials of pain therapies to examine methods, identify common challenges, and areas for improvement. Seven databases were searched for pragmatic randomised controlled clinical trials that assessed pain treatment in a clinical population of adults reporting pain. All screening steps and data extractions were performed twice. Data were synthesised descriptively, and correlation analyses between prespecified trial features and PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) ratings and attrition were performed. Protocol registration: PROSPERO-ID CRD42020178954. Of 57 included trials, only 21% assessed pharmacological interventions, the remainder physical, surgical, psychological, or self-management pain therapies. Three-quarters of the trials were comparative effectiveness designs, often conducted in multiple centres (median: 5; Q1/3: 1, 9.25) and with a median sample size of 234 patients at randomization (Q1/3: 135.5; 363.5). Although most trials recruited patients with chronic pain, reporting of pain duration was poor and not well described. Reporting was comprehensive for most general items, while often deficient for specific pragmatic aspects. Average ratings for pragmatism were highest for treatment adherence flexibility and clinical relevance of outcome measures. They were lowest for patient recruitment methods and extent of follow-up measurements and appointments. Current practice in pragmatic trials of pain treatments can be improved in areas such as patient recruitment and reporting of methods, analysis, and interpretation of data. These improvements will facilitate translatability to other real-world settings-the purpose of pragmatic trials.

Can pragmatic research, real-world data and digital technologies aid the development of psychedelic medicine?

Favourable regulatory assessments, liberal policy changes, new research centres and substantial commercial investment signal that psychedelic therapy is making a major comeback. Positive findings from modern trials are catalysing developments, but it is questionable whether current confirmatory trials are sufficient for advancing our understanding of safety and best practice. Here we suggest supplementing traditional confirmatory trials with pragmatic trials, real-world data initiatives and digital health solutions to better support the discovery of optimal and personalised treatment protocols and parameters. These recommendations are intended to help support the development of safe, effective and cost-efficient psychedelic therapy, which, given its history, is vulnerable to excesses of hype and regulation.

Methodological challenges in pragmatic trials in Alzheimer's disease and related dementias: Opportunities for improvement

BACKGROUND AND AIMS

We need more pragmatic trials of interventions to improve care and outcomes for people living with Alzheimer's disease and related dementias. However, these trials present unique methodological challenges in their design, analysis, and reporting-often, due to the presence of one or more sources of clustering. Failure to account for clustering in the design and analysis can lead to increased risks of Type I and Type II errors. We conducted a review to describe key methodological characteristics and obtain a "baseline assessment" of methodological quality of pragmatic trials in dementia research, with a view to developing new methods and practical guidance to support investigators and methodologists conducting pragmatic trials in this field.

METHODS

We used a published search filter in MEDLINE to identify trials more likely to be pragmatic and identified a subset that focused on people living with Alzheimer's disease or other dementias or included them as a defined subgroup. Pairs of reviewers extracted descriptive information and key methodological quality indicators from each trial.

RESULTS

We identified N = 62 eligible primary trial reports published across 36 different journals. There were 15 (24%) individually randomized, 38 (61%) cluster randomized, and 9 (15%) individually randomized group treatment designs; 54 (87%) trials used repeated measures on the same individual and/or cluster over time and 17 (27%) had a multivariate primary outcome (e.g. due to measuring an outcome on both the patient and their caregiver). Of the 38 cluster randomized trials, 16 (42%) did not report sample size calculations accounting for the intracluster correlation and 13 (34%) did not account for intracluster correlation in the analysis. Of the 9 individually randomized group treatment trials, 6 (67%) did not report sample size calculations accounting for intracluster correlation and 8 (89%) did not account for it in the analysis. Of the 54 trials with repeated measurements, 45 (83%) did not report sample size calculations accounting for repeated measurements and 19 (35%) did not utilize at least some of the repeated measures in the analysis. No trials accounted for the multivariate nature of their primary outcomes in sample size calculation; only one did so in the analysis.

CONCLUSION

There is a need and opportunity to improve the design, analysis, and reporting of pragmatic trials in dementia research. Investigators should pay attention to the potential presence of one or more sources of clustering. While methods for longitudinal and cluster randomized trials are well developed, accessible resources and new methods for dealing with multiple sources of clustering are required. Involvement of a statistician with expertise in longitudinal and clustered designs is recommended.

Understanding the performance of a pan-African intervention to reduce postoperative mortality: a mixed-methods process evaluation of the ASOS-2 trial

BACKGROUND

The African Surgical OutcomeS-2 (ASOS-2) trial tested an enhanced postoperative surveillance intervention to reduce postoperative mortality in Africa. We undertook a concurrent evaluation to understand the process of intervention delivery.

METHODS

Mixed-methods process evaluation, including field notes, interviews, and post-trial questionnaire responses. Qualitative analysis used the framework method with subsequent creation of comparative case studies, grouping hospitals by intervention fidelity. A post-trial questionnaire was developed using initial qualitative analyses. Categorical variables were summarised as count (%) and continuous variables as median (inter-quartile range [IQR]). Odds ratios (OR) were used to rank influences by impact on fidelity.

RESULTS

The dataset included eight in-depth case studies, and 96 questionnaire responses (response rate 67%) plus intervention fidelity data for each trial site. Overall, 57% (n=55/96) of hospitals achieved intervention delivery using an inclusive definition of fidelity. Delivery of the ASOS-2 interventions and data collection presented a significant burden to the investigators, outstripping limited resources. The influences most associated with fidelity were: surgical staff enthusiasm for the trial (OR=3.0; 95% confidence interval [CI], 1.3-7.0); nursing management support of the trial (OR=2.6; 95% CI, 1.1-6.5); performance of a dummy run (OR=2.6; 95% CI, 1.1-6.1); nursing colleagues seeing the value of the intervention(s) (OR=2.1; 95% CI, 0.9-5.7); and site investigators' belief in the effectiveness of the intervention (OR=3.2; 95% CI, 1.2-9.4).

CONCLUSIONS

ASOS-2 has proved that coordinated interventional research across Africa is possible, but delivering the ASOS-2 interventions was a major challenge for many investigators. Future improvement science efforts must include better planning for intervention delivery, additional support to investigators, and promotion of strong inter-professional teamwork.

CLINICAL TRIAL REGISTRATION

ClinicalTrials gov NCT03853824.

Impact of complex, partially nested clustering in a three-arm individually randomized group treatment trial: A case study with the wHOPE trial

BACKGROUND/AIMS

When participants in individually randomized group treatment trials are treated by multiple clinicians or in multiple group treatment sessions throughout the trial, this induces partially nested clusters which can affect the power of a trial. We investigate this issue in the Whole Health Options and Pain Education trial, a three-arm pragmatic, individually randomized clinical trial. We evaluate whether partial clusters due to multiple visits delivered by different clinicians in the Whole Health Team arm and dynamic participant groups due to changing group leaders and/or participants across treatment sessions during treatment delivery in the Primary Care Group Education arm may impact the power of the trial. We also present a Bayesian approach to estimate the intraclass correlation coefficients.

METHODS

We present statistical models for each treatment arm of Whole Health Options and Pain Education trial in which power is estimated under different intraclass correlation coefficients and mapping matrices between participants and clinicians or treatment sessions. Power calculations are based on pairwise comparisons. In practice, sample size calculations depend on estimates of the intraclass correlation coefficients at the treatment sessions and clinician levels. To accommodate such complexities, we present a Bayesian framework for the estimation of intraclass correlation coefficients under different participant-to-session and participant-to-clinician mapping scenarios. We simulated continuous outcome data based on various clinical scenarios in Whole Health Options and Pain Education trial using a range of intraclass correlation coefficients and mapping matrices and used Gibbs samplers with conjugate priors to obtain posteriors of the intraclass correlation coefficients under those different scenarios. Posterior means and medians and their biases are calculated for the intraclass correlation coefficients to evaluate the operating characteristics of the Bayesian intraclass correlation coefficient estimators.

RESULTS

Power for Whole Health Team versus Primary Care Group Education is sensitive to the intraclass correlation coefficient in the Whole Health Team arm. In these two arms, an increased number of clinicians, more evenly distributed workload of clinicians, or more homogeneous treatment group sizes leads to increased power. Our simulation study for the intraclass correlation coefficient estimation indicates that the posterior mean intraclass correlation coefficient estimator has less bias when the true intraclass correlation coefficients are large (i.e. 0.10), but when the intraclass correlation coefficient is small (i.e. 0.01), the posterior median intraclass correlation coefficient estimator is less biased.

CONCLUSION

Knowledge of intraclass correlation coefficients and the structure of clustering are critical to the design of individually randomized group treatment trials with partially nested clusters. We demonstrate that the intraclass correlation coefficient of the Whole Health Team arm can affect power in the Whole Health Options and Pain Education trial. A Bayesian approach provides a flexible procedure for estimating the intraclass correlation coefficients under complex scenarios. More work is needed to educate the research community about the individually randomized group treatment design and encourage publication of intraclass correlation coefficients to help inform future trial designs.

Hospital-free Days: A Pragmatic and Patient-centered Outcome for Trials Among Critically and Seriously Ill Patients

Hospital-free days (HFDs), alternatively known as "days alive and outside the hospital," is increasingly used as a primary or secondary outcome in randomized trials among critically and seriously ill patients. This novel outcome measure addresses an existing gap in the availability of patient-centered, reliably obtained outcome measures among patients with acute respiratory failure, advanced lung diseases, lung transplantation, and other serious and critical illnesses. Traditional outcomes such as mortality, organ-failure-free days, and longitudinal patient-reported measures have distinct drawbacks that limit their suitability as endpoints in trials of patients with serious illness, particularly those trials with pragmatic designs. By contrast, HFDs provides a summary measure of important health events and is easily calculated from administrative or electronic health record data, thereby balancing the goals of patient-centeredness and pragmatic measurement. However, before HFDs can be widely adopted as an endpoint in trials of patients with respiratory and critical illnesses, several questions must be addressed regarding the optimal definition, measurement, and analysis of HFDs. In this perspective, we outline important considerations relevant to the use of HFDs as a trial endpoint and suggest directions for further development of the measure.

Building a National Program for Pilot Studies of Embedded Pragmatic Clinical Trials in Dementia Care

Sixteen million caregivers currently provide care to more than 5 million persons living with dementia (PLWD) in the United States. Although this population is growing and highly complex, evidence-based management remains poorly integrated within healthcare systems. Therefore, the National Institute on Aging IMPACT Collaboratory was formed to build the nation's ability to conduct embedded pragmatic clinical trials (ePCTs) for PLWD and their caregivers. The pilot core of the IMPACT Collaboratory seeks to provide funds for upward of 40 pilots for ePCTs to accelerate the testing of nonpharmacologic interventions with the goal that these pilots lead to full-scale ePCTs and eventually the embedding of evidence-based care into healthcare systems. The first two challenges for the pilot core in building the pilot study program were (1) to develop a transparent, ethical, and open nationwide process for soliciting, reviewing, and selecting pilot studies; and (2) to begin the process of describing the necessary components of a pilot study for an ePCT. During our initial funding cycle, we received 35 letters of intent, of which 17 were accepted for a full proposal and 14 were submitted. From this process we learned that investigators lack knowledge in ePCTs, many interventions lack readiness for an ePCT pilot study, and many proposed studies lack key pragmatic design elements. We therefore have set three key criteria that future pilot studies must meet at a minimum to be considered viable. We additionally discuss key design decisions investigators should consider in designing a pilot study for an ePCT. J Am Geriatr Soc 68:S14-S20, 2020.

Achieving Health Equity in Embedded Pragmatic Trials for People Living with Dementia and Their Family Caregivers

Embedded pragmatic clinical trials (ePCTs) advance research on Alzheimer's disease/Alzheimer's disease and related dementias (AD/ADRD) in real-world contexts; however, health equity issues have not yet been fully considered, assessed, or integrated into ePCT designs. Health disparity populations may not be well represented in ePCTs without special efforts to identify and successfully recruit sites of care that serve larger numbers of these populations. The National Institute on Aging (NIA) Imbedded Pragmatic Alzheimer's disease (AD) and AD-Related Dementias (AD/ADRD) Clinical Trials (IMPACT) Collaboratory's Health Equity Team will contribute to the overall mission of the collaboratory by developing and implementing strategies to address health equity in the conduct of ePCTs and ensure the collaboratory is a national resource for all Americans with dementia. As a first step toward meeting these goals, this article reviews what is currently known about the inclusion of health disparities populations of people living with dementia (PLWD) and their caregivers in ePCTs, highlights unique challenges related to health equity in the conduct of ePCTs, and suggests priority areas in the design and implementation of ePCTs to increase the awareness and avoidance of pitfalls that may perpetuate and magnify healthcare disparities. J Am Geriatr Soc 68:S8-S13, 2020.

Testing Meldonium: Assessing Soviet pragmatic alternatives to the randomized controlled trial

BACKGROUND/AIMS

Current research largely tends to ignore the drug-testing model that was developed in the "Second World" as an explicit alternative to the randomized controlled trial. This system can be described as "socialist pharmapolitics," accounting for the specific features of state socialism that influenced the development and testing of experimental drugs. The clinical trials model employed in the "Second World" was heavily influenced by the Soviet Union, which was by far the most influential player in the socialist bloc during the Cold War. Based on extensive archival research, this article presents an empirical case of a late Soviet clinical trial as a pragmatic alternative to the randomized controlled trial model. It accounts for the divergences between the official model prescribed by the Soviet authorities and the messy realities of healthcare practice. It further outlines different factors that ultimately shaped how clinical trials were organized in Soviet institutions "on the ground." Accordingly, this article presents a "real-life" history of "socialist pharmapolitics" and outlines the problems that this system faced in practice.

METHODS

Archival research was conducted at the Russian State Archive of Scientific and Technical Documentation in Moscow. Archival files include scientific, technical, and registration documentation such as biochemical, pharmacological, and clinical descriptions of the experimental drug Meldonium, letters between various hospitals, research institutes and the Soviet regulatory body, as well as 26 reports of completed clinical trials. Manual content analysis was used for the interpretation of results.

RESULTS

This article presents an empirical case of a late Soviet clinical trial as a pragmatic alternative to the randomized controlled trial model. It demonstrates some key differences from the randomized controlled trial model. This article also highlights some of the discrepancies between the model that was officially prescribed by the Soviet authorities and the realities of experimental drug testing in the Soviet Union in the late 1980s and early 1990s. In particular, it notes some elements of randomization, double-blinding, and the use of placebo that were present in Meldonium trials despite being formally denounced by Soviet bioethics.

CONCLUSION

The Soviet model for testing experimental drugs differed from the Western one substantially in a number of respects. This difference was not only proclaimed officially by the Soviet authorities, but was for the most part enforced in clinical trials in practice. At the same time, our research demonstrates that there were important differences between the official model and the clinical realities on the ground.

Can Electronic Health Records Validly Estimate the Effects of Health System Interventions Aimed at Controlling Body Weight?

OBJECTIVE

This study aimed to compare weight collected at clinics and recorded in the electronic health record (EHR) with primary study-collected trial weights to assess the validity of using EHR data in future pragmatic weight loss or weight gain prevention trials.

METHODS

For both the Track and Shape obesity intervention randomized trials, clinic EHR weight data were compared with primary trial weight data over the same time period. In analyzing the EHR weights, intervention effects were estimated on the primary outcome of weight (in kilograms) with EHR data, using linear mixed effects models.

RESULTS

EHR weight measurements were higher on average and more variable than trial weight measurements. The mean difference and 95% CI were similar at all time points between the estimates using EHR and study-collected weights.

CONCLUSIONS

The results of this study can be used to help guide the planning of future pragmatic weight-related trials. This study provides evidence that body weight measurements abstracted from the EHR can provide valid, efficient, and cost-effective data to estimate treatment effects from randomized clinical weight loss and weight management trials. However, care should be taken to properly understand the data-generating process and any mechanisms that may affect the validity of these estimates.

Effect of a Multifactorial Fall Injury Prevention Intervention on Patient Well-Being: The STRIDE Study

BACKGROUND/OBJECTIVES

In the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) study, a multifactorial intervention was associated with a nonsignificant 8% reduction in time to first serious fall injury but a significant 10% reduction in time to first self-reported fall injury relative to enhanced usual care. The effect of the intervention on other outcomes important to patients has not yet been reported. We aimed to evaluate the effect of the intervention on patient well-being including concern about falling, anxiety, depression, physical function, and disability.

DESIGN

Pragmatic cluster-randomized trial of 5,451 community-living persons at high risk for serious fall injuries.

SETTING

A total of 86 primary care practices within 10 U.S. healthcare systems.

PARTICIPANTS

A random subsample of 743 persons aged 75 and older.

MEASUREMENTS

The well-being measures, assessed at baseline, 12 months, and 24 months, included a modified version of the Fall Efficacy Scale, Patient-Reported Outcomes Measurement Information System (PROMIS) anxiety and depression scales, and Late-Life Function and Disability Instrument.

RESULTS

Participants in the intervention (n = 384) and control groups (n = 359) were comparable in age: mean (standard deviation) of 81.9 (4.7) versus 81.8 (5.0) years. Mean scores were similar between groups at 12 and 24 months for concern about falling, physical function, and disability, whereas the intervention group's mean scores on anxiety and depression were .7 points lower (i.e., better) at 12 months and .6 to .8 points lower at 24 months. For each of these outcomes, differences between the groups' adjusted least square mean changes from baseline to 12 and 24 months, respectively, were quantitatively small. The overall difference in means between groups over 2 years was statistically significant only for depression, favoring the intervention: -1.19 (99% confidence interval, -2.36 to -.02), with 3.5 points representing a minimally important difference.

CONCLUSIONS

STRIDE's multifactorial intervention to reduce fall injuries was not associated with clinically meaningful improvements in patient well-being.

D-CARE: The Dementia Care Study: Design of a Pragmatic Trial of the Effectiveness and Cost Effectiveness of Health System-Based Versus Community-Based Dementia Care Versus Usual Dementia Care

BACKGROUND/OBJECTIVES

Although several approaches have been developed to provide comprehensive care for persons living with dementia (PWD) and their family or friend caregivers, the relative effectiveness and cost effectiveness of community-based dementia care (CBDC) versus health system-based dementia care (CBDC) and the effectiveness of both approaches compared with usual care (UC) are unknown.

DESIGN

Pragmatic randomized three-arm superiority trial. The unit of randomization is the PWD/caregiver dyad.

SETTING

Four clinical trial sites (CTSs) based in academic and clinical health systems.

PARTICIPANTS

A total of 2,150 English- or Spanish-speaking PWD who are not receiving hospice or residing in a nursing home and their caregivers.

INTERVENTIONS

Eighteen months of (1) HSDC provided by a nurse practitioner or physician's assistant dementia care specialist who works within the health system, or (2) CBDC provided by a social worker or nurse care consultant who works at a community-based organization, or (3) UC with as needed referral to the Alzheimer's Association Helpline.

MEASUREMENTS

Primary outcomes: PWD behavioral symptoms and caregiver distress as measured by the Neuropsychiatric Inventory Questionnaire (NPI-Q) Severity and Modified Caregiver Strain Index scales.

SECONDARY OUTCOMES

NPI-Q Distress, caregiver unmet needs and confidence, and caregiver depressive symptoms. Tertiary outcomes: PWD long-term nursing home placement rates, caregiver-reported PWD functional status, cognition, goal attainment, "time spent at home," Dementia Burden Scale-Caregiver, a composite measure of clinical benefit, Quality of Life of persons with dementia, Positive Aspects of Caregiving, and cost effectiveness using intervention costs and Medicare claims.

RESULTS

The results will be reported in the spring of 2024.

CONCLUSION

D-CARE will address whether emphasis on clinical support and tighter integration with other medical services has greater benefit than emphasis on social support that is tied more closely to community resources. It will also assess the effectiveness of both interventions compared with UC and will evaluate the cost effectiveness of each intervention.

When and how to include vulnerable subjects in clinical trials

There has been a good deal of discussion in the literature regarding which subjects are vulnerable in the context of clinical trials. There has been significantly less discussion regarding when and how to include vulnerable subjects in clinical trials. This lack of guidance is a particular problem for trials covered by the US regulations, which mandate strict requirements on the inclusion of three groups: pregnant women/fetuses, prisoners, and children. For the past 30 years, funders, investigators, and institutional review boards have frequently responded to these regulations by excluding pregnant women/fetuses, prisoners, and children from clinical trials. More recent work has emphasized the extent to which a default of exclusion can undermine the value of clinical trials, especially pragmatic trials. A default of exclusion also has the potential to undermine the interests of vulnerable groups, in both the short and the long term. These concerns raise the need for guidance on how to satisfy existing US regulations, while minimizing their negative impact on the value of clinical trials and the interests of vulnerable groups. The present manuscript thus describes a six-step decision procedure that institutional review boards can use to determine when and how to include vulnerable subjects in clinical trials, including pragmatic trials, that are covered by US regulations.

Objecting to experiments even while approving of the policies or treatments they compare

We resolve a controversy over two competing hypotheses about why people object to randomized experiments: 1) People unsurprisingly object to experiments only when they object to a policy or treatment the experiment contains, or 2) people can paradoxically object to experiments even when they approve of implementing either condition for everyone. Using multiple measures of preference and test criteria in five preregistered within-subjects studies with 1,955 participants, we find that people often disapprove of experiments involving randomization despite approving of the policies or treatments to be tested.

A pragmatic study of mid-mixture insulin and basal insulin treatment in patients with type 2 diabetes uncontrolled with oral antihyperglycaemic medications: A lesson from real-world experience

BACKGROUND

Chinese guidelines for the treatment of type 2 diabetes (T2D) recommend basal or premixed insulins as insulin starters after failed oral antihyperglycaemic medication (OAM). This pragmatic study compared effectiveness and safety of add-on basal insulin analog (BI) and mid-mixture insulin analog (MMI; 50:50 premixed insulin) as starter insulin regimens in Chinese patients with T2D in a real-world setting.

MATERIALS AND METHODS

This was a multicentre, open-label, randomized, parallel, pragmatic trial. Patients receiving OAMs were randomized 1:1 to BI (n = 410) or MMI (n = 404) for 24 weeks. Insulin titration and OAM adjustment were determined by investigators following usual standard-of-care. The primary outcome was change in glycated haemoglobin (HbA1c) from baseline.

RESULTS

Least-squares mean changes in HbA1c from baseline to week 24 were -2.00% and -2.15% for BI and MMI groups, respectively (P = .13). The MMI group demonstrated a greater reduction in concomitant OAM therapies used than BI group (53.8% vs. 35.3%, respectively; P < .001). Very limited daily insulin dose increments were observed from baseline to week 24 in both BI and MMI groups (2.5 U/day and 1.8 U/day, respectively). Although both insulin analogs were well-tolerated without severe hypoglycaemia, small weight gains were seen with both treatments. Higher total hypoglycaemia rates were noticed with the MMI group, while nocturnal hypoglycaemia events were comparable.

CONCLUSIONS

In real-world settings, BI and MMI provided similar improvement in glucose control without conceding hypoglycaemia. The BI group received a greater number of OAMs in real-world settings. Limited insulin dose titration was observed, while more adjustments occurred with OAM usage.

Training the Workforce to Conduct Embedded Pragmatic Clinical Trials to Improve Care for People Living with Dementia and Their Caregivers

The National Institute on Aging IMbedded Pragmatic Alzheimer's Disease and Alzheimer's Disease-Related Dementias Clinical Trials (IMPACT) Collaboratory serves as a national resource for the conduct of embedded pragmatic clinical trials to improve the care of people living with dementia (PLWD) in partnership with the healthcare systems that serve them. Inherent in this objective is the need to train and support a cadre of investigators prepared to conduct this work now and in the future. The Training Core of the IMPACT Collaboratory supports the training of investigators to become experts in this field through three objectives: (1) curricula development and dissemination; (2) network generation and navigation; and (3) a career development award program. The innovative approach of the Training Core will require developing content and providing training experiences that recognize the unique challenges of research at the intersection of health systems, pragmatic trials, and PLWD and their caregivers. Ultimately, we seek to build the nation's capacity to conduct research that bridges the gaps between efficacy studies to effectiveness research to implementation science. Although foundational resources in the methods of each of these areas are already available, few actually focus on pragmatic trials embedded within healthcare systems that focus on PLWD. To bring new interventions for PLWD from efficacy to widespread implementation, researchers must build diffusability, adaptability, heterogeneity, and scalability into the design of the intervention. In achieving these objectives, the Training Core will utilize the network of investigators, institutions, and stakeholders represented in the IMPACT Collaboratory. J Am Geriatr Soc 68:S21-S27, 2020.

Sociodemographic characteristics of youth in a trauma focused-cognitive behavioral therapy effectiveness trial in the city of Philadelphia

While randomized controlled trials of trauma-focused cognitive behavioral therapy (TF-CBT) have demonstrated efficacy for youth with posttraumatic stress disorder, TF-CBT effectiveness trials typically show attenuated outcomes. This decrease in effectiveness may be due to the differences in sociodemographic characteristics of youth in these trials; youth in efficacy trials are more often white and middle-income, whereas youth in effectiveness trials are more often racial/ethnic minorities, of low socioeconomic status (SES) and live in high crime neighborhoods. In this study-drawn from an effectiveness trial of TF-CBT in community mental health clinics across Philadelphia-we describe the sociodemographic characteristics of enrolled youth. We measured neighborhood SES by matching participants' addresses to American Community Survey data from their Census tracts, housing stability using the National Outcomes Measurement System, and neighborhood violence using police department crime statistics. Our results suggest that the majority of youth presenting for TF-CBT in mental health clinics in the City of Philadelphia live in poor and high-crime neighborhoods, experience substantial housing instability, and are predominantly ethnic and racial minorities. Thus, youth presenting for treatment experience significant racial and socioeconomic adversity. We also explored the association between these characteristics and youth symptom severity upon presenting for treatment. These factors were not associated with youth symptom severity or overall mental health functioning in our sample (with small effect sizes and p > .05 for all). Implications for future research, such as the need for efficacy and effectiveness trials to more fully characterize their samples and the need for pragmatic trials are discussed.

Optimizing Retention in a Pragmatic Trial of Community-Living Older Persons: The STRIDE Study

OBJECTIVES

The Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) study is testing the effectiveness of a multifactorial intervention to prevent serious fall injuries. Our aim was to describe procedures that were implemented to optimize participant retention; report retention yields by age, sex, clinical site, and follow-up time; provide reasons for study withdrawals; and highlight the successes and lessons learned from the STRIDE retention efforts.

DESIGN

Pragmatic cluster randomized trial.

SETTING

A total of 86 primary care practices within 10 US healthcare systems.

PARTICIPANTS

A total of 5451 community-living persons, 70 years of age or older, at high risk for serious fall injuries.

MEASUREMENTS

Study outcomes were collected every 4 months by a central call center. Reconsent was required to extend follow-up beyond the originally planned 36 months.

RESULTS

Over a median follow-up of 3.2 years (interquartile range = 2.8-3.7 y), 439 (8.1%) participants died and 600 (11.0%) withdrew their consent or did not reconsent to extend follow-up beyond 36 months, yielding rates (per 100 person-years) of deaths and withdrawals of 2.6 and 3.6, respectively. The withdrawal rate increased with advancing age, was comparable for men and women, and did not differ much by clinical site. The most common reasons for withdrawal were illness and unable to contact for reconsent at 36 months. Completion of the follow-up interviews was greater than 93% at each time point. Most participants completed all (71.8%) or all but one (9.2%) of the follow-up interviews. The most common reason for not completing a follow-up interview was unable to contact, with rates ranging from 2.8% at 40 months to 4.6% at 20 months.

CONCLUSION

Completion of the thrice-yearly follow-up interviews in STRIDE was high, and retention of participants over 44 months exceeded the original projections. The procedures used in STRIDE, together with lessons learned, should assist other investigators who are planning or conducting large pragmatic trials of vulnerable older persons. J Am Geriatr Soc 68:1242-1249, 2020.

Using predictive analytics to improve pragmatic trial design

Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.

Closing the cycle of innovation in healthcare in Europe

Pragmatic or practice-oriented comparative effectiveness trials may be conducted to fill the evidence gaps that are revealed after the private sector has performed the trials needed for bringing their product to the market. A tool of increasing importance to identify such evidence gaps is resulting from health technology assessments (HTA) whereby the data derived from clinical research are examined in a systematic manner with reference to effect, safety, as well as additional parameters. Practice-oriented trials are informative for healthcare decision makers, practice-changing and may even be cost-saving for the healthcare payers. There are however only a limited number of funding sources for such trials. Public and private healthcare payers should stimulate the conduct of practice-oriented trials in their effort to maximize patient benefit within the limitation of the available resources. Pragmatic randomized trials can be performed at low cost when based on existing coded electronic health records and as well health registries. Public health decision makers are increasingly taking advantage of results from health technology assessments to support priority setting. In accordance with this it would appear reasonable that decision makers should get more involved in priority setting and funding also in the field of clinical research in order to provide further evidence needed for assessments, reassessments, and subsequent qualified decisions and resource allocations in health care. A closer dialogue and collaboration between the clinical research and HTA communities would facilitate a more efficient utilization of such opportunities.

Masking in Pragmatic Trials: Who, What, and When to Blind

Masking (or blinding) of treatment assignment is routinely implemented in classical randomized clinical trials (RCTs) to isolate the effect of the intervention itself and to minimize the potential for bias that could occur with traditional trials. Such biases could be introduced with the conduct, assessment of endpoints, management of conditions, analysis, and reporting when the treatment assignments are known. However, masking of treatments is not only complex but it hinders how generalizable the findings are to the "real world" clinical setting. Pragmatic RCTs (pRCTs) are intended to evaluate the effects of interventions within routine medical care, and as such, do not typically mask treatment groups; moreover, pRCTs assess comparators that are available in routine medical practice, not masked placebos. Whether pRCTs should be masked if intended for regulatory or other purposes has recently been questioned. The literature on pRCTs, while extensive, does not address how much actual benefit is gained from masking outcomes and how masking may affect the "real world" nature of a study. Here, we propose an approach to evaluate sources of bias, describe stakeholders in the conduct of pRCTs who are most likely affected, and offer a framework for considering how masking may be implemented effectively while maintaining generalizability.

Research-grade data in the real world: challenges and opportunities in data quality from a pragmatic trial in community-based practices

Randomized controlled trials face cost, logistic, and generalizability limitations, including difficulty engaging racial/ethnic minorities. Real-world data (RWD) from pragmatic trials, including electronic health record (EHR) data, may produce intervention evaluation findings generalizable to diverse populations. This case study of Project IMPACT describes unique barriers and facilitators of optimizing RWD to improve health outcomes and advance health equity in small immigrant-serving community-based practices. Project IMPACT tested the effect of an EHR-based health information technology intervention on hypertension control among small urban practices serving South Asian patients. Challenges in acquiring accurate RWD included EHR field availability and registry capabilities, cross-sector communication, and financial, personnel, and space resources. Although using RWD from community-based practices can inform health equity initiatives, it requires multidisciplinary collaborations, clinic support, procedures for data input (including social determinants), and standardized field logic/rules across EHR platforms.

Objecting to experiments that compare two unobjectionable policies or treatments

Randomized experiments have enormous potential to improve human welfare in many domains, including healthcare, education, finance, and public policy. However, such "A/B tests" are often criticized on ethical grounds even as similar, untested interventions are implemented without objection. We find robust evidence across 16 studies of 5,873 participants from three diverse populations spanning nine domains-from healthcare to autonomous vehicle design to poverty reduction-that people frequently rate A/B tests designed to establish the comparative effectiveness of two policies or treatments as inappropriate even when universally implementing either A or B, untested, is seen as appropriate. This "A/B effect" is as strong among those with higher educational attainment and science literacy and among relevant professionals. It persists even when there is no reason to prefer A to B and even when recipients are treated unequally and randomly in all conditions (A, B, and A/B). Several remaining explanations for the effect-a belief that consent is required to impose a policy on half of a population but not on the entire population; an aversion to controlled but not to uncontrolled experiments; and a proxy form of the illusion of knowledge (according to which randomized evaluations are unnecessary because experts already do or should know "what works")-appear to contribute to the effect, but none dominates or fully accounts for it. We conclude that rigorously evaluating policies or treatments via pragmatic randomized trials may provoke greater objection than simply implementing those same policies or treatments untested.

Whole Systems Research Methods in Health Care: A Scoping Review

Objectives: This scoping review evaluates two decades of methodological advances made by “whole systems research” (WSR) pioneers in the fields of traditional, complementary, and integrative medicine (TCIM). Rooted in critiques of the classical randomized controlled trial (RCT)'s suitability for evaluating holistic, complex TCIM interventions, WSR centralizes the principle of “model validity,” representing a “fit” between research design and therapeutic paradigm.

Design: In consultation with field experts, 41 clinical research exemplars were selected for review from across 13 TCIM disciplines, with the aim of mapping the range and methodological characteristics of WSR studies. Using an analytic charting approach, these studies' primary and secondary features are characterized with reference to three focal areas: research method, intervention design, and outcome assessment.

Results: The reviewed WSR exemplars investigate a wide range of multimodal and multicomponent TCIM interventions, typified by wellness-geared, multitarget, and multimorbid therapeutic aims. Most studies include a behavioral focus, at times in multidisciplinary or team-based contexts. Treatments are variously individualized, often with reference to “dual” (biomedical and paradigm-specific) diagnoses. Prospective and retrospective study designs substantially reflect established biomedical research methods. Pragmatic, randomized, open label comparative effectiveness designs with “usual care” comparators are most widely used, at times with factorial treatment arms. Only two studies adopt a double-blind, placebo-controlled RCT format. Some cohort-based controlled trials engage nonrandomized allocation strategies (e.g., matched controls, preference-based assignment, and minimization); other key designs include single-cohort pre–post studies, modified n-of-1 series, case series, case report, and ethnography. Mixed methods designs (i.e., qualitative research and economic evaluations) are evident in about one-third of exemplars. Primary and secondary outcomes are predominantly assessed, at multiple intervals, through patient-reported measures for symptom severity, quality of life/wellness, and/or treatment satisfaction; some studies concurrently evaluate objective outcomes.

Conclusions: Aligned with trends emphasizing “fit-for-purpose” research designs to study the “real-world” effectiveness of complex, personalized clinical interventions, WSR has emerged as a maturing scholarly discipline. The field is distinguished by its patient-centered salutogenic focus and engagement with nonbiomedical diagnostic and treatment frameworks. The rigorous pursuit of model validity may be further advanced by emphasizing complex analytic models, paradigm-specific outcome assessment, inter-rater reliability, and ethnographically informed designs. Policy makers and funders seeking to support best practices in TCIM research may refer to this review as a key resource.

Lessons from Outside and Within: Exploring Advancements in Methodology for Naturopathic Medicine Clinical Research

Introduction: Naturopathy is a mixture of both traditional and complementary medicine. It incorporates a broad set of health care practices that may or may not be traditional to that country or conventional medicine and are not fully integrated into the dominant health care system. Research required to evaluate or substantiate naturopathic medicine may not fall under the testing of randomized clinical trials, which opens up discussions on what is the best practice for research in naturopathic medicine.

Discussion: Not only do advances in health research methodology offer important opportunities to progress naturopathic research, there are also areas where the unique characteristics of naturopathic philosophy and practice can impact other areas of health research. Some of the new advances in health research methodology involve whole-system research, pragmatic trials, template for intervention description and replication protocols for complex interventions, patient-centered care models, and the pragmatic-explanatory continuum indicator summary tool for designing pragmatic trials. Discussion and critique of these health-related methodologies shows that these research methods are more suited for the philosophy and treatment options that naturopathy is based on.

Conclusions: Successful implementation of naturopathic research methodologies, and translation and dissemination of research will require a substantial paradigm shift in which naturopathic practitioners adopt a greater level of responsibility for developing an evidence base for naturopathic medicine.

Effectiveness trials: critical data to help understand how respiratory medicines really work?

Most of the information about the benefits, safety aspects, and cost effectiveness of pharmacological treatment in the respiratory field has been obtained from traditional efficacy studies, such as randomised controlled trials (RCT). The highly controlled environment of an RCT does not always reflect everyday practice. The collection, analysis, and application of effectiveness data to generate Real World Evidence (RWE) through pragmatic trials or observational studies therefore has the potential to improve decision making by regulators, payers, and clinicians.

Despite calls for more RWE, effectiveness data are not widely used in decision making in the respiratory field. Recent advances in data capture, curation, and storage combined with new analytical tools have now made it feasible for effectiveness data to become routine sources of evidence to supplement traditional efficacy data. In this paper, we will examine some of the current data gaps, diverse types of effectiveness data, look at proposed frameworks for the positioning of effectiveness data, as well as provide examples from therapeutic areas. We will give examples of both previous effectiveness studies and studies that are ongoing within the respiratory field. Effectiveness data hold the potential to address several evidentiary gaps related to the effectiveness, safety, and value of treatments in patients with respiratory diseases.

Semi-Individualized Homeopathy Add-On Versus Usual Care Only for Premenstrual Disorders: A Randomized, Controlled Feasibility Study

OBJECTIVES

Premenstrual syndrome and premenstrual dysphoric disorder (PMS/PMDD) bother a substantial number of women. Homeopathy seems a promising treatment, but it needs investigation using reliable study designs. The feasibility of organizing an international randomized pragmatic trial on a homeopathic add-on treatment (usual care [UC] + HT) compared with UC alone was evaluated.

DESIGN

A multicenter, randomized, controlled pragmatic trial with parallel groups.

SETTINGS/LOCATION

The study was organized in general and private homeopathic practices in the Netherlands and Sweden and in an outpatient university clinic in Germany.

SUBJECTS

Women diagnosed as having PMS/PMDD, based on prospective daily rating by the daily record of severity of problems (DRSP) during a period of 2 months, were included and randomized.

INTERVENTIONS

Women were to receive UC + HT or UC for 4 months. Homeopathic medicine selection was according to a previously tested prognostic questionnaire and electronic algorithm. Usual care was as provided by the women's general practitioner according to their preferences.

OUTCOME MEASURES

Before and after treatment, the women completed diaries (DRSP), the measure yourself concerns and well-being, and other questionnaires. Intention-to-treat (ITT) and per protocol (PP) analyses were performed.

RESULTS

In Germany, the study could not proceed because of legal limitations. In Sweden, recruitment proved extremely difficult. In the Netherlands and Sweden, 60 women were randomized (UC + HT: 28; UC: 32), data of 47/46 women were analyzed (ITT/PP). After 4 months, relative mean change of DRSP scores in the UC + HT group was significantly better than in the UC group (p = 0.03).

CONCLUSIONS

With respect to recruitment and different legal status, it does not seem feasible to perform a larger, international, pragmatic randomized trial on (semi-)individualized homeopathy for PMS/PMDD. Since the added value of HT compared with UC was demonstrated by significant differences in symptom score changes, further studies are warranted.

Evaluating the design and reporting of pragmatic trials in osteoarthritis research

Objectives

Among the challenges in health research is translating interventions from controlled experimental settings to clinical and community settings where chronic disease is managed daily. Pragmatic trials offer a method for testing interventions in real-world settings but are seldom used in OA research. The aim of this study was to evaluate the literature on pragmatic trials in OA research up to August 2016 in order to identify strengths and weaknesses in the design and reporting of these trials.

Methods

We used established guidelines to assess the degree to which 61 OA studies complied with pragmatic trial design and reporting. We assessed design according to the pragmatic-explanatory continuum indicator summary and reporting according to the pragmatic trials extension of the CONsolidated Standards of Reporting Trials guidelines.

Results

None of the pragmatic trials met all 11 criteria evaluated and most of the trials met between 5 and 8 of the criteria. Criteria most often unmet pertained to practitioner expertise (by requiring specialists) and criteria most often met pertained to primary outcome analysis (by using intention-to-treat analysis).

Conclusion

Our results suggest a lack of highly pragmatic trials in OA research. We identify this as a point of opportunity to improve research translation, since optimizing the design and reporting of pragmatic trials can facilitate implementation of evidence-based interventions for OA care.