Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory

The Integration of Clinical Trials With the Practice of Medicine: Repairing a House Divided

Importance

Optimal health care delivery, both now and in the future, requires a continuous loop of knowledge generation, dissemination, and uptake on how best to provide care, not just determining what interventions work but also how best to ensure they are provided to those who need them. The randomized clinical trial (RCT) is the most rigorous instrument to determine what works in health care. However, major issues with both the clinical trials enterprise and the lack of integration of clinical trials with health care delivery compromise medicine's ability to best serve society.

Observations

In most resource-rich countries, the clinical trials and health care delivery enterprises function as separate entities, with siloed goals, infrastructure, and incentives. Consequently, RCTs are often poorly relevant and responsive to the needs of patients and those responsible for care delivery. At the same time, health care delivery systems are often disengaged from clinical trials and fail to rapidly incorporate knowledge generated from RCTs into practice. Though longstanding, these issues are more pressing given the lessons learned from the COVID-19 pandemic, heightened awareness of the disproportionate impact of poor access to optimal care on vulnerable populations, and the unprecedented opportunity for improvement offered by the digital revolution in health care. Four major areas must be improved. First, especially in the US, greater clarity is required to ensure appropriate regulation and oversight of implementation science, quality improvement, embedded clinical trials, and learning health systems. Second, greater adoption is required of study designs that improve statistical and logistical efficiency and lower the burden on participants and clinicians, allowing trials to be smarter, safer, and faster. Third, RCTs could be considerably more responsive and efficient if they were better integrated with electronic health records. However, this advance first requires greater adoption of standards and processes designed to ensure health data are adequately reliable and accurate and capable of being transferred responsibly and efficiently across platforms and organizations. Fourth, tackling the problems described above requires alignment of stakeholders in the clinical trials and health care delivery enterprises through financial and nonfinancial incentives, which could be enabled by new legislation. Solutions exist for each of these problems, and there are examples of success for each, but there is a failure to implement at adequate scale.

Conclusions and Relevance

The gulf between current care and that which could be delivered has arguably never been wider. A key contributor is that the 2 limbs of knowledge generation and implementation-the clinical trials and health care delivery enterprises-operate as a house divided. Better integration of these 2 worlds is key to accelerated improvement in health care delivery.

Fidelity to the acupuncture intervention protocol in the ACUpuncture In The EmergencY department for pain management (ACUITY) trial: Expanding the gold standard of STRICTA and CONSORT guidelines

Background

Acupuncture shows promise as an effective nonpharmacologic option for reduction of acute pain in the emergency department (ED). Following CONSORT and STRICTA guidelines, randomized controlled trials (RCTs) generally report intervention details and acupoint options, but fidelity to acupuncture interventions, critical to reliability in intervention research, is rarely reported.

Methods

ACUITY is an NCCIH-funded, multi-site feasibility RCT of acupuncture in 3 EDs (Cleveland, Nashville, and San Diego). ACUITY acupuncturists were trained in study design, responsive acupuncture manualization protocol, logistics and real-time recording of session details via REDCap forms created to track fidelity.

Results

Across 3 recruiting sites, 79 participants received acupuncture: 51 % women, 43 % Black/African American, with heterogeneous acute pain sites at baseline: 32 % low back, 22 % extremity, 20 % abdominal, 10 % head. Pragmatically, participants were treated in ED common areas (52 %), private rooms (39 %), and semi-private rooms (9 %). Objective tracking found 98 % adherence to the six components of the acupuncture manualization protocol: staging, number of insertion points (M = 13.2, range 2-22), needle retention time (M = 23.5 min, range 4-52), session length (M = 40.3 min, range 20-66), whether general recommendations were provided and completion of the session form.

Conclusion

To the best of our knowledge, this is the first RCT to assess and report fidelity to an acupuncture protocol. Fidelity monitoring will be fundamental for ACUITY2, which would be a future definitive, multi-site RCT. Furthermore, we recommend that fidelity to acupuncture interventions be added to CONSORT and STRICTA reporting guidelines in future RCTs.

Protocol registration

The protocol of this study is registered at clinicaltrials.gov: NCT04880733.

Framework of the Strengths and Challenges of Clinically Integrated Trials: An Expert Panel Report

The limitations of the explanatory clinical trial framework include the high expense of implementing explanatory trials, restrictive entry criteria for participants, and redundant logistical processes. These limitations can result in slow evidence generation that is not responsive to population health needs, yielding evidence that is not generalizable. Clinically integrated trials, which integrate clinical research into routine care, represent a potential solution to this challenge and an opportunity to support learning health systems. The operational and design features of clinically integrated trials include a focused scope, simplicity in design and requirements, the leveraging of existing data structures, and patient participation in the entire trial process. These features are designed to minimize barriers to participation and trial execution and reduce additional research burdens for participants and clinicians alike. Broad adoption and scalability of clinically integrated trials are dependent, in part, on continuing regulatory, healthcare system, and payer support. This analysis presents a framework of the strengths and challenges of clinically integrated trials and is based on a multidisciplinary expert "Think Tank" panel discussion that included representatives from patient populations, academia, non-profit funding agencies, the U.S. Food and Drug Administration, and industry.

Regulatory Issues in Electronic Health Records for Adolescent HIV Research: Strategies and Lessons Learned

BACKGROUND

Electronic health records (EHRs) are a cost-effective approach to provide the necessary foundations for clinical trial research. The ability to use EHRs in real-world clinical settings allows for pragmatic approaches to intervention studies with the emerging adult HIV population within these settings; however, the regulatory components related to the use of EHR data in multisite clinical trials poses unique challenges that researchers may find themselves unprepared to address, which may result in delays in study implementation and adversely impact study timelines, and risk noncompliance with established guidance.

OBJECTIVE

As part of the larger Adolescent Trials Network (ATN) for HIV/AIDS Interventions Protocol 162b (ATN 162b) study that evaluated clinical-level outcomes of an intervention including HIV treatment and pre-exposure prophylaxis services to improve retention within the emerging adult HIV population, the objective of this study is to highlight the regulatory process and challenges in the implementation of a multisite pragmatic trial using EHRs to assist future researchers conducting similar studies in navigating the often time-consuming regulatory process and ensure compliance with adherence to study timelines and compliance with institutional and sponsor guidelines.

METHODS

Eight sites were engaged in research activities, with 4 sites selected from participant recruitment venues as part of the ATN, who participated in the intervention and data extraction activities, and an additional 4 sites were engaged in data management and analysis. The ATN 162b protocol team worked with site personnel to establish the necessary regulatory infrastructure to collect EHR data to evaluate retention in care and viral suppression, as well as para-data on the intervention component to assess the feasibility and acceptability of the mobile health intervention. Methods to develop this infrastructure included site-specific training activities and the development of both institutional reliance and data use agreements.

RESULTS

Due to variations in site-specific activities, and the associated regulatory implications, the study team used a phased approach with the data extraction sites as phase 1 and intervention sites as phase 2. This phased approach was intended to address the unique regulatory needs of all participating sites to ensure that all sites were properly onboarded and all regulatory components were in place. Across all sites, the regulatory process spanned 6 months for the 4 data extraction and intervention sites, and up to 10 months for the data management and analysis sites.

CONCLUSIONS

The process for engaging in multisite clinical trial studies using EHR data is a multistep, collaborative effort that requires proper advanced planning from the proposal stage to adequately implement the necessary training and infrastructure. Planning, training, and understanding the various regulatory aspects, including the necessity of data use agreements, reliance agreements, external institutional review board review, and engagement with clinical sites, are foremost considerations to ensure successful implementation and adherence to pragmatic trial timelines and outcomes.

Sample size and power calculation for testing treatment effect heterogeneity in cluster randomized crossover designs

The cluster randomized crossover design has been proposed to improve efficiency over the traditional parallel-arm cluster randomized design. While statistical methods have been developed for designing cluster randomized crossover trials, they have exclusively focused on testing the overall average treatment effect, with little attention to differential treatment effects across subpopulations. Recently, interest has grown in understanding whether treatment effects may vary across pre-specified patient subpopulations, such as those defined by demographic or clinical characteristics. In this article, we consider the two-treatment two-period cluster randomized crossover design under either a cross-sectional or closed-cohort sampling scheme, where it is of interest to detect the heterogeneity of treatment effect via an interaction test. Assuming a patterned correlation structure for both the covariate and the outcome, we derive new sample size formulas for testing the heterogeneity of treatment effect with continuous outcomes based on linear mixed models. Our formulas also address unequal cluster sizes and therefore allow us to analytically assess the impact of unequal cluster sizes on the power of the interaction test in cluster randomized crossover designs. We conduct simulations to confirm the accuracy of the proposed methods, and illustrate their application in two real cluster randomized crossover trials.

The dementia care study (D-CARE): Recruitment strategies and demographic characteristics of participants in a pragmatic randomized trial of dementia care

INTRODUCTION

Pragmatic research studies that include diverse dyads of persons living with dementia (PLWD) and their family caregivers are rare.

METHODS

Community-dwelling dyads were recruited for a pragmatic clinical trial evaluating three approaches to dementia care. Four clinical trial sites used shared and site-specific recruitment strategies to enroll health system patients.

RESULTS

Electronic health record (EHR) queries of patients with a diagnosis of dementia and engagement of their clinicians were the main recruitment strategies. A total of 2176 dyads were enrolled, with 80% recruited after the onset of the pandemic. PLWD had a mean age of 80.6 years (SD 8.5), 58.4% were women, and 8.8% were Hispanic/Latino, and 11.9% were Black/African American. Caregivers were mostly children of the PLWD (46.5%) or spouses/partners (45.2%), 75.8% were women, 9.4% were Hispanic/Latino, and 11.6% were Black/African American.

DISCUSSION

Health systems can successfully enroll diverse dyads in a pragmatic clinical trial.

If You Build It, Will They Come? Patient and Provider Use of a Novel Hybrid Telehealth Care Pathway for Low Back Pain

OBJECTIVE

The purpose of this study was to describe the referrals and use of a hybrid care model for low back pain that includes on-site care by physical therapists, physical activity training, and psychologically informed practice (PiP) delivered by telehealth in the Improving Veteran Access to Integrated Management of Low Back Pain (AIM-Back) trial.

METHODS

Data were collected from November 2020 through February 2023 from 5 Veteran Health Administration clinics participating in AIM-Back, a multisite, cluster-randomized embedded pragmatic trial. The authors extracted data from the Veteran Health Administration Corporate Data Warehouse to describe referral and enrollment metrics, telehealth use (eg, distribution of physical activity and PiP calls), and treatments used by physical therapists and telehealth providers.

RESULTS

Seven hundred one veterans were referred to the AIM-Back trial with 422 enrolling in the program (consult-to-enrollment rate = 60.2%). After travel restrictions were lifted, site visits resulted in a significant increase in referrals and a number of new referring providers. At initial evaluation by on-site physical therapists, 92.2% of veterans received pain modulation (eg, transcutaneous electrical nerve stimulation, manual therapy). Over 81% of enrollees completed at least 1 telehealth physical activity call, with a mean of 2.8 (SD = 2.0) calls out of 6. Of the 167 veterans who screened as medium to high risk of persistent disability, 74.9% completed at least 1 PiP call, with a mean of 2.5 (SD = 2.0) calls out of 6. Of those who completed at least 1 PiP call (n = 125), 100% received communication strategies, 97.6% received pain coping skills training, 89.6% received activity-based treatments, and 99.2% received education in a home program.

CONCLUSION

In implementing a hybrid care pathway for low back pain, the authors observed consistency in the delivery of core components (ie, pain modulation, use of physical activity training, and risk stratification to PiP), notable variability in telehealth calls, high use of PiP components, and increased referrals with tailored provider engagement.

IMPACT

These findings describe variability occurring within a hybrid care pathway and can inform future implementation efforts.

Algorithmic identification of persons with dementia for research recruitment: ethical considerations

Underdiagnosis, misdiagnosis, and patterns of social inequality that translate into unequal access to health systems all pose barriers to identifying and recruiting diverse and representative populations into research on Alzheimer's disease and Alzheimer's disease related dementias. In response, some have turned to algorithms to identify patients living with dementia using information that is associated with this condition but that is not as specific as a diagnosis. This paper explains six ethical issues associated with the use of such algorithms including the generation of new, sensitive, identifiable medical information for research purposes without participant consent, issues of justice and equity, risk, and ethical communication. It concludes with a discussion of strategies for addressing these issues and prompting valuable research.

Implementing the learning health system paradigm within academic health centers

Introduction

The learning health system (LHS) concept represents a bold innovation that combines organizational learning, strategic analysis of patient data, stakeholder engagement and the systematic translation of research into practice - all in service of improving the quality of health care delivered across the organization. This innovation has been diffused and widely adopted by healthcare organizations over the past 15 years, but academic health centers (AHCs) have been slower on the uptake. The irony is that AHCs have the resources (e.g., trained researchers, sophisticated clinical data systems, informatics infrastructure) that are necessary to do the highest-quality and most impactful LHS work.

Methods

Based on a review of publications describing how AHCs have implemented LHS work, as well as the authors' direct experience promoting the adoption of the LHS paradigm at Atrium Health Wake Forest Baptist (AHWFB), we:identify a set of factors that have inhibited broader adoption of the LHS paradigm among AHCs; distinguish between the forms of LHS work that are consistent and inconsistent with the mission of AHCs; and offer recommendations for broader adoption and fuller implementation of the LHS paradigm.

Results

The LHS paradigm represents an expansion of the scientific paradigm which serves as the foundation of research enterprise within AHCs. Both paradigms value rigorous studies of new treatments and practices, including pragmatic clinical trials. The LHS paradigm also places a high value on quality improvement studies, organizational learning, and the translation of research findings into improved patient care and operations within the local health system. The two paradigms differ on the origin of the research question, i.e., a pressing patient-care issue facing the health system versus the investigator's own research interests. Academic researchers have been disincentivized from pursuing at least some forms of LHS research. However, a growing number of AHCs are finding ways to integrate the LHS paradigm into their research enterprise, either by providing research faculty with institutional funding to cover their effort on studies that address the health system's priority issues, or by establishing an institute dedicated to LHS research.

Conclusions

The LHS paradigm is a disruptive intervention for AHCs, one that was initially resisted but is increasingly being embraced. AHCs are developing strategies for conducting LHS research, typically in parallel to the more traditional biomedical science that is core to academic medicine. Full implementation of the LHS paradigm will require further alignment between LHS and science, including a shift in the criteria for promotion and tenure to support those researchers who choose to focus on the pressing issues facing the health system.

Optimizing clinical nutrition research: the role of adaptive and pragmatic trials

Evidence-based nutritional recommendations address the health impact of suboptimal nutritional status. Efficacy randomized controlled trials (RCTs) have traditionally been the preferred method for determining the effects of nutritional interventions on health outcomes. Nevertheless, obtaining a holistic understanding of intervention efficacy and effectiveness in real-world settings is stymied by inherent constraints of efficacy RCTs. These limitations are further compounded by the complexity of nutritional interventions and the intricacies of the clinical context. Herein, we explore the advantages and limitations of alternative study designs (e.g., adaptive and pragmatic trials), which can be incorporated into RCTs to optimize the efficacy or effectiveness of interventions in clinical nutrition research. Efficacy RCTs often lack external validity due to their fixed design and restrictive eligibility criteria, leading to efficacy-effectiveness and evidence-practice gaps. Adaptive trials improve the evaluation of nutritional intervention efficacy through planned study modifications, such as recalculating sample sizes or discontinuing a study arm. Pragmatic trials are embedded within clinical practice or conducted in settings that resemble standard of care, enabling a more comprehensive assessment of intervention effectiveness. Pragmatic trials often rely on patient-oriented primary outcomes, acquire outcome data from electronic health records, and employ broader eligibility criteria. Consequently, adaptive and pragmatic trials facilitate the prompt implementation of evidence-based nutritional recommendations into clinical practice. Recognizing the limitations of efficacy RCTs and the potential advantages of alternative trial designs is essential for bridging efficacy-effectiveness and evidence-practice gaps. Ultimately, this awareness will lead to a greater number of patients benefiting from evidence-based nutritional recommendations.

The Case for Conducting Pragmatic Dementia Care Trials in Medicaid Home and Community-Based Service Settings

Medicaid-funded home and community-based services (HCBSs) reach large numbers of individuals living with dementia who would otherwise reside in nursing homes with Medicaid funding. Medicaid HCBSs also often augment care provided by family and other informal caregivers to individuals living with dementia. Although Medicaid-funded HCBSs are offered in most states in lieu of nursing home care, they have been largely overlooked as health care system partners for implementation and testing of evidence-based dementia care interventions using embedded pragmatic clinical trial (ePCT) designs. In this article, we make the case for the importance of Medicaid-funded HCBSs as dementia care ePCT partners because of the volume of vulnerable clients with dementia served and the potential positive impacts that evidence-based dementia care programs can have on clients and their informal caregivers. This article first characterizes the Medicaid HCBS setting in terms of populations served and organizational arrangements across states. We then characterize strengths and potential limitations presented by Medicaid HCBSs as settings within which to implement dementia care ePCTs, using as a conceptual framework the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) tool and its domains. We draw on our experiences implementing the Care of Persons with Dementia in their Environments (COPE) program in a statewide Medicaid HCBS setting to highlight how these potential ePCT partners can help optimize pragmatic approaches to several PRECIS-2 domains. We found that partners are especially effective in implementing pragmatic ways to determine eligibility for evidence-based dementia care programs; assist with recruitment of eligible individuals; incorporate dementia care interventions into the range of existing HCBSs; and track outcomes relevant to persons living with dementia, caregivers, HCBS providers, and Medicaid insurance stakeholders. We conclude with recommendations for researchers, potential ePCT partners, and policymakers to help facilitate the growth of dementia care ePCTs in Medicaid HCBS settings across the United States.

Partnering with social service staff to implement pragmatic clinical trials: an interim analysis of implementation strategies

BACKGROUND

With recent growth in the conduct of pragmatic clinical trials, the reliance on frontline staff to contribute to trial-related activities has grown as well. Active partnerships with staff members are often critical to pragmatic trial implementation, but rarely do research teams track and evaluate the specific "implementation strategies" used to support staff's involvement in trial procedures (e.g., participant recruitment). Accordingly, we adapted implementation science methodologies and conducted an interim analysis of the strategies deployed with social service staff involved in one multi-site pragmatic clinical trial.

METHODS

We used a naturalistic, observational study design to characterize strategies our research team deployed with staff during monthly, virtual meetings. Data were drawn from meeting notes and recordings from the trial's 4-month Preparation phase and 8-month Implementation phase. Strategies were mapped to the Expert Recommendations for Implementing Change taxonomy and categorized into nine implementation clusters. Survey data were also collected from staff to identify the most useful strategies the research team should deploy when onboarding new staff members in the trial's second year.

RESULTS

A total of 287 strategies were deployed. Strategies in the develop stakeholder interrelationships cluster predominated in both the Preparation (35%) and Implementation (31%) phases, followed by strategies in the use iterative and evaluative approaches cluster, though these were more prevalent during trial Preparation (24%) as compared to trial Implementation (18%). When surveyed on strategy usefulness, strategies in the provide interactive assistance, use financial approaches, and support staff clusters were most useful, per staff responses.

CONCLUSIONS

While strategies to develop stakeholder interrelationships were used most frequently during trial Preparation and Implementation, program staff perceived strategies that provided technical assistance, supported clinicians, and used financial approaches to be most useful and should be deployed when onboarding new staff members. Research teams are encouraged to adapt and apply implementation strategy tracking methods when partnering with social service staff and deploy practical strategies that support pragmatic trial success given staff needs and preferences.

TRIAL REGISTRATION

NCT05357261. May 2, 2022.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.

Career development in pragmatic clinical trials to improve care for people living with dementia

The growing number of people living with dementia (PLWD) requires a coordinated clinical response to deliver pragmatic, evidence-based interventions in frontline care settings. However, infrastructure to support such a response is lacking. Moreover, there are too few researchers conducting rigorous embedded pragmatic clinical trials (ePCTs) to make the vision of high quality, widely accessible dementia care a reality. National Institute on Aging (NIA) Imbedded Pragmatic Alzheimer's disease and Related Dementias Clinical Trials (IMPACT) Collaboratory seeks to improve the pipeline of early career researchers qualified to lead ePCTs by funding career development awards. Even with support from the Collaboratory, awardees face practical and methodological challenges to success, recently exacerbated by the COVID-19 pandemic. We first describe the training opportunities and support network for the IMPACT CDA recipients. This report then describes the unique career development challenges faced by early-career researchers involved in ePCTs for dementia care. Topics addressed include challenges in establishing a laboratory, academic promotion, mentoring and professional development, and work-life balance. Concrete suggestions to address these challenges are offered for early-career investigators, their mentors, and their supporting institutions. While some of these challenges are faced by researchers in other fields, this report seeks to provide a roadmap for expanding the work of the IMPACT Collaboratory and initiating future efforts to recruit, train, and retain talented early-career researchers involved in ePCTs for dementia care.

Let's talk early labour: The L-TEL randomised controlled trial

BACKGROUND

Women without complications have lower obstetric intervention if they remain at home in early labour but many women report dissatisfaction in doing this. Using self-efficacy theory as an underpinning framework, a web-based intervention was co-created with women who had previously used maternity services. The intervention provides early labour advice, alongside the videoed, real experiences of women.

METHOD

The pragmatic, randomised control trial aimed to evaluate the impact of the web-based intervention on women's self-reported experiences of early labour. Low-risk, nulliparous, pregnant women (140) were randomised. The intervention group (69) received the web-based intervention antenatally to use at their own convenience and the control group (71) received usual care. Data were collected at 7-28 days postnatally using an online version of the Early Labour Experience Questionnaire (ELEQ). The primary outcome was the ELEQ score. Secondary, clinical outcomes such as labour onset, augmentation and mode of birth were collected from the existing hospital system.

RESULTS

There were no statistically significant differences in the ELEQ scores between trial arms. Women in the intervention group were significantly more likely to progress spontaneously in labour without the need for labour augmentation (39.1 %) compared to the control group (21.1 %) (OR 2.41, CI 95 %; 1.14-5.11).

CONCLUSION

Although the L-TEL Trial found no statistically significant differences in the primary outcome, the innovative intervention to support women during latent phase labour was positively received by women. Web-based resources are a cost effective, user-friendly and accessible way to provide women with education. A larger trial is needed to detect differences in clinical outcomes.

Partner engagement for planning and development of non-pharmacological care pathways in the AIM-Back trial

BACKGROUND/AIMS

Embedded pragmatic clinical trials are increasingly recommended for non-pharmacological pain care research due to their focus on examining intervention effectiveness within real-world settings. Engagement with patients, health care providers, and other partners is essential, yet there is limited guidance for how to use engagement to meaningfully inform the design of interventions to be tested in pain-related pragmatic clinical trials. This manuscript aims to describe the process and impacts of partner input on the design of two interventions (care pathways) for low back pain currently being tested in an embedded pragmatic trial in the Veterans Affairs health care system.

METHODS

Sequential cohort design for intervention development was followed. Engagement activities were conducted with 25 participants between November 2017 and June 2018. Participants included representatives from multiple groups: clinicians, administrative leadership, patients, and caregivers.

RESULTS

Partner feedback led to several changes in each of the care pathways to improve patient experience and usability. Major changes to the sequenced care pathway included transitioning from telephone-based delivery to a flexible telehealth model, increased specificity about pain modulation activities, and reduction of physical therapy visits. Major changes to the pain navigator pathway included transitioning from a traditional stepped care model to one that offers care in a feedback loop, increased flexibility regarding pain navigator provider type, and increased specificity for patient discharge criteria. Centering patient experience emerged as a key consideration from all partner groups.

CONCLUSION

Diverse input is important to consider before implementing new interventions in embedded pragmatic trials. Partner engagement can increase acceptability of new care pathways to patients and providers and enhance uptake of effective interventions by health systems.

TRIAL REGISTRATION

NCT#04411420. Registered on 2 June 2020.

Extending the vision of adaptive point-of-care platform trials to improve targeted use of drug therapy regimens: An agile approach in the learning healthcare system toolkit

OBJECTIVES

Improving the targeted use of drug regimens requires robust real-world evidence (RWE) to address the uncertainties that remain regarding their real-world performance following market entry. However, challenges in the current state of RWE production limit its impact on clinical decisions, as well as its operational scalability and sustainability. We propose an adaptive point-of-care (APoC) platform trial as an approach to RWE production that improves both clinical and operational efficiencies.

METHODS AND FINDINGS

We explored design innovations, operational challenges, and infrastructure needs within a multi-stakeholder consortium to evaluate the potential of an APoC platform trial for studying chronic disease treatment regimens using rheumatoid arthritis as a case study. The concept integrates elements from adaptive clinical trials (dynamic treatment regimen strategies) and point-of-care trials (research embedded into routine clinical care) under a perpetual platform infrastructure. The necessary components to implement an APoC platform trial within outpatient settings exist, and present an opportunity for a cross-disciplinary, multi-stakeholder approach. Effective engagement of key stakeholders involved in and impacted by the platform is critical to success. Our collaborative design process identified three high-impact stakeholder-engagement areas: (1) focus on research question(s), (2) design and implementation planning such that it is feasible and fit-for-purpose, and (3) measurement, or meaningful metrics for both clinical (patient outcomes) and system (operational efficiencies) impact.

CONCLUSIONS

An APoC platform trial for rheumatoid arthritis integrating innovative design elements in a scalable infrastructure has the potential to reduce important uncertainties about the real-world performance of biomedical innovations and improve clinical decisions.

Clinical Nurses' Identification of a Wearable Universal Serial Bus Used for Pediatric Oncology Clinical Trial Participant Safety Management

The expanded access to clinical trials has provided more patients the opportunity to participate in novel therapeutics research. There is an increased likelihood of a patient, as a pediatric oncology clinical trial participant, to present for clinical care outside the research site, such as at an emergency room or urgent care center. A novel wearable universal serial bus device is a proposed technology to bridge potential communication gaps, pertaining to critical information such as side effects and permitted therapies, between research teams and clinical teams where investigational agents may be contraindicated to standard treatments. Fifty-five emergency and urgent care nurses across the United States were presented, via online survey without priming to the context of clinical trials or the device, a picture of a pediatric patient wearing the novel wearable device prompted to identify significant, environmental cues important for patient care. Of the 40 nurses observing the patient photo, three identified the wearable device within Situational Awareness Global Assessment Tool formatted narrative response fields. Analysis of the narrative nurse-participant responses of significant clinical findings upon initial assessment of the pediatric patient photo is described, as well as the implications for subsequent prototyping of the novel universal serial bus prototype.

Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory

Embedded pragmatic clinical trials (ePCTs) play a vital role in addressing current population health problems, and their use of electronic health record (EHR) systems promises efficiencies that will increase the speed and volume of relevant and generalizable research. However, as the number of ePCTs using EHR-derived data grows, so does the risk that research will become more vulnerable to biases due to differences in data capture and access to care for different subsets of the population, thereby propagating inequities in health and the healthcare system. We identify 3 challenges-incomplete and variable capture of data on social determinants of health, lack of representation of vulnerable populations that do not access or receive treatment, and data loss due to variable use of technology-that exacerbate bias when working with EHR data and offer recommendations and examples of ways to actively mitigate bias.

Think Pragmatically: Investigators' Obligations to Patient-Subjects When Research is Embedded in Care

Growing interest in embedded research approaches-where research is incorporated into clinical care-has spurred numerous studies to generate knowledge relevant to the real-world needs of patients and other stakeholders. However, it also has presented ethical challenges. An emerging challenge is how to understand the nature and extent of investigators' obligations to patient-subjects. Prior scholarship on investigator duties has generally been grounded upon the premise that research and clinical care are distinct activities, bearing distinct duties. Yet this premise-and its corresponding implications-are challenged when research and clinical care are deliberately integrated. After presenting three case studies from recent pragmatic clinical trials, we identify six differences between explanatory trials and embedded research that limit the application of existing scholarship for ascertaining investigator duties. We suggest that these limitations indicate a need to account for the implications of usual care and to move beyond a narrow focus on the investigator-subject dyad, one that better reflects the team- and institution-based nature of contemporary health systems.

Clinical reasoning in pragmatic trial randomization: a qualitative interview study

BACKGROUND

Pragmatic trials, because they study widely used treatments in settings of routine practice, require intensive participation from clinicians who determine whether patients can be enrolled. Clinicians are often conflicted between their therapeutic obligation to patients and their willingness to enroll them in trials in which treatments are randomly determined and thus potentially suboptimal. Refusal to enroll eligible patients can hinder trial completion and damage generalizability. In order to help evaluate and mitigate clinician refusal, this qualitative study examined how clinicians reason about whether to randomize eligible patients.

METHODS

We performed interviews with 29 anesthesiologists who participated in REGAIN, a multicenter pragmatic randomized trial comparing spinal and general anesthesia in hip fracture. Interviews included a chart-stimulated section in which physicians described their reasoning pertaining to specific eligible patients as well as a general semi-structured section about their views on clinical research. Guided by a constructivist grounded theory approach, we analyzed data via coding, synthesized thematic patterns using focused coding, and developed an explanation using abduction.

RESULTS

Anesthesiologists perceived their main clinical function as preventing peri- and intraoperative complications. In some cases, they used prototype-based reasoning to determine whether patients with contraindications should be randomized; in others, they used probabilistic reasoning. These modes of reasoning involved different types of uncertainty. In contrast, anesthesiologists expressed confidence about anesthetic options when they accepted patients for randomization. Anesthesiologists saw themselves as having a fiduciary responsibility to patients and thus did not hesitate to communicate their inclinations, even when this complicated trial recruitment. Nevertheless, they voiced strong support for clinical research, stating that their involvement was mainly hindered by production pressure and workflow disruptions.

CONCLUSIONS

Our findings suggest that prominent ways of assessing clinician decisions about trial randomization are based on questionable assumptions about clinical reasoning. Close examination of routine clinical practice, attuned to the features of clinical reasoning we reveal here, will help both in evaluating clinicians' enrollment determinations in specific trials and in anticipating and responding to them.

TRIAL REGISTRATION

Regional Versus General Anesthesia for Promoting Independence After Hip Fracture (REGAIN).

CLINICALTRIALS

gov NCT02507505. Prospectively registered on July 24, 2015.

Analysis of the characteristics and the degree of pragmatism exhibited by pragmatic-labelled trials of antineoplastic treatments

BACKGROUND

Pragmatic clinical trials (PCTs) are designed to reflect how an investigational treatment would be applied in clinical practice. As such, unlike their explanatory counterparts, they measure therapeutic effectiveness and are capable of generating high-quality real-world evidence. However, the conduct of PCTs remains extremely rare. The scarcity of such studies has contributed to the emergence of the efficacy-effectiveness gap and has led to calls for launching more of them, including in the field of oncology. This analysis aimed to identify self-labelled pragmatic trials of antineoplastic interventions and to evaluate whether their use of this label was justified.

METHODS

We searched PubMed® and Embase® for publications corresponding with studies that investigated antitumor therapies and that were tagged as pragmatic in their titles, abstracts and/or index terms. Subsequently, we consulted all available source documents for the included trials and extracted relevant information from them. The data collected were then used to appraise the degree of pragmatism displayed by the PCTs with the help of the validated PRECIS-2 tool.

RESULTS

The literature search returned 803 unique records, of which 46 were retained upon conclusion of the screening process. This ultimately resulted in the identification of 42 distinct trials that carried the 'pragmatic' label. These studies examined eight different categories of neoplasms and were mostly randomized, open-label, multicentric, single-country trials sponsored by non-commercial parties. On a scale of one (very explanatory) to five (very pragmatic), the median PCT had a PRECIS-2 score per domain of 3.13 (interquartile range: 2.57-3.53). The most and least pragmatic studies in the sample had a score of 4.44 and 1.57, respectively. Only a minority of trials were described in sufficient detail to allow them to be graded across all domains of the PRECIS-2 instrument. Many of the studies examined also had features that arguably precluded them from being pragmatic altogether, such as being monocentric or placebo-controlled in nature.

CONCLUSION

PCTs of antineoplastic treatments are generally no more pragmatic than they are explanatory.

Mayo-PACC: A parsimonious preclinical Alzheimer's disease cognitive composite comprised of public-domain measures to facilitate clinical translation

INTRODUCTION

We aimed to define a Mayo Preclinical Alzheimer's disease Cognitive Composite (Mayo-PACC) that prioritizes parsimony and use of public domain measures to facilitate clinical translation.

METHODS

Cognitively unimpaired participants aged 65 to 85 at baseline with amyloid PET imaging were included, yielding 428 amyloid negative (A-) and 186 amyloid positive (A+) individuals with 7 years mean follow-up. Sensitivity to amyloid-related cognitive decline was examined using slope estimates derived from linear mixed models (difference in annualized change across A+ and A- groups). We compared differences in rates of change between Mayo-PACC and other composites (A+ > A- indicating more significant decline in A+).

RESULTS

All composites showed sensitivity to amyloid-related longitudinal cognitive decline (A+ > A- annualized change p < 0.05). Comparisons revealed that Mayo-PACC (AVLT sum of trials 1-5+6+delay, Trails B, animal fluency) showed comparable longitudinal sensitivity to other composites.

DISCUSSION

Mayo-PACC performs similarly to other composites and can be directly translated to the clinic.

"We bleed for our community:" A qualitative exploration of the implementation of a pragmatic weight gain prevention trial from the perspectives of community health center professionals

BACKGROUND

Clinical trial implementation continues to shift toward pragmatic design, with the goal of increasing future adoption in clinical practice. Yet, few pragmatic trials within clinical settings have qualitatively assessed stakeholder input, especially from those most impacted by research implementation and outcomes, i.e., providers and staff. Within this context, we conducted a qualitative study of the implementation of a pragmatic digital health obesity trial with employees at a Federally qualified health center (FQHC) network in central North Carolina.

METHODS

Participant recruitment was conducted through purposive sampling of FQHC employees from a variety of backgrounds. Two researchers conducted semi-structured qualitative interviews and collected demographic data. Interviews were digitally recorded, professionally transcribed and double-coded by two independent researchers using NVivo 12. Coding discrepancies were reviewed by a third researcher until intercoder consensus was reached. Responses were compared within and across participants to elucidate emergent themes.

RESULTS

Eighteen qualitative interviews were conducted, of whom 39% provided direct medical care to patients and 44% worked at the FQHC for at least seven years. Results illuminated the challenges and successes of a pragmatically designed obesity treatment intervention within the community that serves medically vulnerable patients. Although limited time and staffing shortages may have challenged recruitment processes, respondents described early buy-in from leadership; an alignment of organizational and research goals; and consideration of patient needs as facilitators to implementation. Respondents also described the need for personnel power to sustain novel research interventions and considerations of health center resource constraints.

CONCLUSIONS

Results from this study contribute to the limited literature on pragmatic trials utilizing qualitative methods, particularly in community-based obesity treatment. To continue to merge the gaps between research implementation and clinical care, qualitative assessments that solicit stakeholder input are needed within pragmatic trial design. For maximum impact, researchers may wish to solicit input from a variety of professionals at trial onset and ensure that shared common goals and open collaboration between all partners is maintained throughout the trial.

TRIAL REGISTRATION

This trial was registered with ClinicalTrials.gov (NCT03003403) on December 28, 2016.

Experiment aversion among clinicians and the public - an obstacle to evidence-based medicine and public health

Background

Randomized controlled trials (RCTs) are essential for determining the safety and efficacy of healthcare interventions. However, both laypeople and clinicians often demonstrate experiment aversion: preferring to implement either of two interventions for everyone rather than comparing them to determine which is best. We studied whether clinician and layperson views of pragmatic RCTs for Covid-19 or other interventions became more positive early in the pandemic, which increased both the urgency and public discussion of RCTs.

Methods

We conducted several survey studies with laypeople (total n=2,909) and two with clinicians (n=895; n=1,254) in 2020 and 2021. Participants read vignettes in which a hypothetical decision-maker who sought to improve health could choose to implement intervention A for all, implement intervention B for all, or experimentally compare A and B and implement the superior intervention. Participants rated and ranked the appropriateness of each decision.

Results

Compared to our pre-pandemic results, we found no decrease in laypeople's aversion to non-Covid-19 experiments involving catheterization checklists and hypertension drugs. Nor were either laypeople or clinicians less averse to Covid-19 RCTs (concerning corticosteroid drugs, vaccines, intubation checklists, proning, school reopening, and mask protocols), on average. Across all vignettes and samples, levels of experiment aversion ranged from 28% to 57%, while levels of experiment appreciation (in which the RCT is rated higher than the participant's highest-rated intervention) ranged from only 6% to 35%.

Conclusions

Advancing evidence-based medicine through pragmatic RCTs will require anticipating and addressing experiment aversion among both patients and healthcare professionals.

Intervention delivery for embedded pragmatic clinical trials: Development of a tool to measure complexity

BACKGROUND

Conducting an embedded pragmatic clinical trial in the workflow of a healthcare system is a complex endeavor. The complexity of the intervention delivery can have implications for study planning, ability to maintain fidelity to the intervention during the trial, and/or ability to detect meaningful differences in outcomes.

METHODS

We conducted a literature review, developed a tool, and conducted two rounds of phone calls with NIH Pragmatic Trials Collaboratory Demonstration Project principal investigators to develop the Intervention Delivery Complexity Tool. After refining the tool, we piloted it with Collaboratory demonstration projects and developed an online version of the tool using the R Shiny application (https://duke-som.shinyapps.io/ICT-ePCT/).

RESULTS

The 6-item tool consists of internal and external factors. Internal factors pertain to the intervention itself and include workflow, training, and the number of intervention components. External factors are related to intervention delivery at the system level including differences in healthcare systems, the dependency on setting for implementation, and the number of steps between the intervention and the outcome.

CONCLUSION

The Intervention Delivery Complexity Tool was developed as a standard way to overcome communication challenges of intervention delivery within an embedded pragmatic trial. This version of the tool is most likely to be useful to the trial team and its health system partners during trial planning and conduct. We expect further evolution of the tool as more pragmatic trials are conducted and feedback is received on its performance outside of the NIH Pragmatic Trials Collaboratory.

Concordance Between Patient-Reported Health Data and Electronic Health Data in the ADAPTABLE Trial

Importance

Patient-reported health data can facilitate clinical event capture in pragmatic clinical trials. However, few data are available on the fitness for use of patient-reported data in large-scale health research.

Objective

To evaluate the concordance of a set of variables reported by patients and available in the electronic health record as part of a pragmatic clinical trial.

Design, Setting, and Participants

Data from ADAPTABLE (Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-term Effectiveness), a pragmatic clinical trial, were used in a concordance substudy of a comparative effectiveness research trial. The trial randomized 15 076 patients with existing atherosclerotic cardiovascular disease in a 1:1 ratio to low- or high-dose aspirin from April 2016 through June 30, 2019.

Main Outcomes and Measures

Concordance of data was evaluated from 4 domains (demographic characteristics, encounters, diagnoses, and procedures) present in 2 data sources: patient-reported data captured through an online portal and data from electronic sources (electronic health record data). Overall agreement, sensitivity, specificity, positive predictive value, negative predictive value, and κ statistics with 95% CIs were calculated using patient report as the criterion standard for demographic characteristics and the electronic health record as the criterion standard for clinical outcomes.

Results

Of 15 076 patients with complete information, the median age was 67.6 years (range, 21-99 years), and 68.7% were male. With the use of patient-reported data as the criterion standard, agreement (κ) was high for Black and White race and ethnicity but only moderate for current smoking status. Electronic health record data were highly specific (99.6%) but less sensitive (82.5%) for Hispanic ethnicity. Compared with electronic health record data, patient report of clinical end points had low sensitivity for myocardial infarction (33.0%), stroke (34.2%), and major bleeding (36.6%). Positive predictive value was similarly low for myocardial infarction (40.7%), stroke (38.8%), and major bleeding (21.9%). Coronary revascularization was the most concordant event by data source, with only moderate agreement (κ = 0.54) and positive predictive value. Agreement metrics varied by site for all demographic characteristics and several clinical events.

Conclusions and Relevance

In a concordance substudy of a large, pragmatic comparative effectiveness research trial, sensitivity and chance-corrected agreement of patient-reported data captured through an online portal for cardiovascular events were low to moderate. Findings suggest that additional work is needed to optimize integration of patient-reported health data into pragmatic research studies.

Trial Registration

ClinicalTrials.gov Identifier: NCT02697916.

Fitness of real-world data for clinical trial data collection: Results and lessons from a HARMONY Outcomes ancillary study

BACKGROUND

Despite the extensive use of real-world data for retrospective, observational clinical research, our understanding of how real-world data might increase the efficiency of data collection in patient-level randomized clinical trials is largely unknown. The structure of real-world data is inherently heterogeneous, with each source electronic health record and claims database different from the next. Their fitness-for-use as data sources for multisite trials in the United States has not been established.

METHODS

For a subset of participants in the HARMONY Outcomes Trial, we obtained electronic health record data from recruiting sites or Medicare claims data from the Centers for Medicare & Medicaid Services. For baseline characteristics and follow-up events, we assessed the level of agreement between these real-world data and data documented in the trial database.

RESULTS

Real-world data-derived demographic information tended to agree with trial-reported demographic information, although real-world data were less accurate in identifying medical history. The ability of real-world data to identify baseline medication usage differed by real-world data source, with claims data demonstrating substantially better performance than electronic health record data. The limited number of lab results in the collected electronic health record data matched closely with values in the trial database. There were not enough follow-up events in the ancillary study population to draw meaningful conclusions about the performance of real-world data for identification of events. Based on the conduct of this ancillary study, the challenges and opportunities of using real-world data within clinical trials are discussed.

CONCLUSION

Based on a subset of participants from the HARMONY Outcomes Trial, our results suggest that electronic health record or claims data, as currently available, are unlikely to be a complete substitute for trial data collection of medical history or baseline lab results, but that Medicare claims were able to identify most medications. The limited size of the study population prevents us from drawing strong conclusions based on these results, and other studies are clearly needed to confirm or refute these findings.

Ethics challenges in sharing data from pragmatic clinical trials

Numerous arguments have been advanced for broadly sharing de-identified, participant-level clinical trials data, and trial sponsors and journals are increasingly requiring it. However, data sharing in pragmatic clinical trials presents ethical challenges related to the use of waivers or alterations of informed consent for some pragmatic clinical trials and corresponding limitations of informed consent to guide sharing decisions; the potential for data sharing in pragmatic clinical trials to present risks not only for individual patient-subjects, but also for health systems and the clinicians within them; sharing of data from electronic health records instead of data newly collected for research purposes; and researchers' limited capacity to control sensitive data within an electronic health record and potential implications of such limits for meeting obligations inherent to Certificates of Confidentiality. These challenges raise questions about the extent to which traditional research ethics governance structures are capable of guiding decisions about pragmatic clinical trial data sharing. This article identifies and examines these ethical challenges for pragmatic clinical trial data sharing. We suggest several areas for future empirical scholarship, including the need to identify patient and public attitudes regarding pragmatic clinical trial data sharing as well as to assess the demand for pragmatic clinical trial data and the correspondingly likely benefit of such sharing. Further conceptual work is also needed to explore how requirements to respect patient-subjects about whom data are shared in the context of pragmatic clinical trials should be understood, particularly in the absence of informed consent for initial research activities, and the appropriate balance between promoting the generation of socially valuable knowledge and respecting autonomy.

Opportunities and challenges in heart rhythm research: Rationale and development of an electrophysiology collaboratory

There are many challenges in the current landscape of electrophysiology (EP) clinical and translational research, including increasing costs and complexity, competing demands, regulatory requirements, and challenges with study implementation. This review seeks to broadly discuss the state of EP research, including challenges and opportunities. Included here are results from a Heart Rhythm Society (HRS) Research Committee member survey detailing HRS members' perspectives regarding both barriers to clinical and translational research and opportunities to address these challenges. We also provide stakeholder perspectives on barriers and opportunities for future EP research, including input from representatives of the U.S. Food and Drug Administration, industry, and research funding institutions that participated in a Research Collaboratory Summit convened by HRS. This review further summarizes the experiences of the heart failure and heart valve communities and how they have approached similar challenges in their own fields. We then explore potential solutions, including various models of research ecosystems designed to identify research challenges and to coordinate ways to address them in a collaborative fashion in order to optimize innovation, increase efficiency of evidence generation, and advance the development of new therapeutic products. The objectives of the proposed collaborative cardiac EP research community are to encourage and support scientific discourse, research efficiency, and evidence generation by exploring collaborative and equitable solutions in which stakeholders within the EP community can interact to address knowledge gaps, innovate, and advance new therapies.

An electronic pillbox intervention designed to improve medication safety during care transitions: challenges and lessons learned regarding implementation and evaluation

Background

Adverse drug events are common during transitions of care. As part of the Smart Pillbox study, a cluster-randomized controlled trial of an electronic pillbox designed to reduce medication discrepancies and improve medication adherence after hospital discharge, we explored barriers to successful implementation and evaluation of this intervention.

Methods

Eligible patients were those admitted to a medicine service of a large teaching hospital with a plan to be discharged home on five or more chronic medications. The intervention consisted of an electronic pillbox with pre-filled weekly blister pack medication trays given to patients prior to discharge. Pillbox features included alarms to take medications, detection of pill removal from each well, alerts to patients or caregivers by phone, email, or text if medications were not taken, and adherence reports accessible by providers. Greater than 20% missed doses for three days in a row triggered outreach from a pharmacist. To identify barriers to implementation and evaluation of the intervention, we reviewed patient exit surveys, including quantitative data on satisfaction and free-text responses regarding their experiences; technical issue logs; and team meeting minutes. Themes were derived by consensus among the study authors and organized using the Consolidated Framework for Implementation Research.

Results

Barriers to implementation included intervention characteristics such as perceived portability issues with the pillbox and time required by pharmacists to enter medication information into the software; external policies such as lack of insurance coverage for early refills and regulatory prohibitions on repackaging medications; implementation climate issues such as the incompatibility between the rushed nature of hospital discharge with the time required to deploy the intervention; and patient issues such as denial of previous problems with medication adherence. We founds several obstacles to conducting the study, including patients declining study enrollment and limited attempts by the hospital to streamline logistics by building the intervention into usual care. Several solutions to address many of these challenges were implemented or planned. Despite these challenges, many patients with the pillbox were pleased with the service and believed the intervention worked well for them.

Conclusions

In this evaluation, several barriers to implementing and conducting a study of the effectiveness of the intervention were identified. Our findings provide lessons learned for others wishing to implement and evaluate HIT-related interventions designed to improve medication safety during care transitions.

Trial registration

Clinicaltrials.gov NCT03475030

Update on trials examining effects of night-time blood pressure lowering drug treatment on prevention of cardiovascular disease

Current evidence on benefits of night-time blood pressure (BP) lowering drug treatment on cardiovascular disease (CVD) prevention attributable to the Ambulatory Blood Pressure Monitoring in the Prediction of Cardiovascular Events and Effects of Chronotherapy (MAPEC) trial and Bedtime hypertension treatment improves cardiovascular risk reduction (Hygia) trials has raised concern on their validity and methodology. In this commentary, the authors have updated the progress of the ongoing trials that were planned to examine the effect of night-time BP lowering drug treatment on CVD prevention. As compared to MAPEC and Hygia trials, three pragmatic trials the Blood Pressure Medication Timing (BPMedtime) trial (US), the Treatment In Morning versus Evening (TIME) trial (UK), Bedmed and Bedmed-frail (Canada) were planned without ambulatory BP monitoring. The BPMedtime trial was stopped after the pilot phase due to underestimated sample size and insufficient funds. TIME trial (UK) had a similar issue when changing the sample size from 10,269 to more than 20,000 participants. The TIME trial was completed and the initial results showing that protection against heart attack, stroke and vascular death is not affected by whether antihypertensive medications are taken in the morning or evening. The full study of the TIME trial is published in December 2022. Bedmed and Bedmed-frail trials are ongoing and will be completed in 2023. Time of taking BP lowering drug should be determined by patients at their convenience to improve the adherence. There was no difference in adverse effects of taking BP lowering drugs at night or morning. Evidence on the effect of night-time treatment on CVD events is inconsistent. The results from ongoing trials in Canada will contribute evidence to the use of BP lowering drug treatment for the prevention of CVD.

Data capture and sharing in the COVID-19 pandemic: a cause for concern

Routine health care and research have been profoundly influenced by digital-health technologies. These technologies range from primary data collection in electronic health records (EHRs) and administrative claims to web-based artificial-intelligence-driven analyses. There has been increased use of such health technologies during the COVID-19 pandemic, driven in part by the availability of these data. In some cases, this has resulted in profound and potentially long-lasting positive effects on medical research and routine health-care delivery. In other cases, high profile shortcomings have been evident, potentially attenuating the effect of-or representing a decreased appetite for-digital-health transformation. In this Series paper, we provide an overview of how facets of health technologies in routinely collected medical data (including EHRs and digital data sharing) have been used for COVID-19 research and tracking, and how these technologies might influence future pandemics and health-care research. We explore the strengths and weaknesses of digital-health research during the COVID-19 pandemic and discuss how learnings from COVID-19 might translate into new approaches in a post-pandemic era.

Co-creation using crowdsourcing to promote PrEP adherence in China: study protocol for a stepped-wedge randomized controlled trial

Background

Adherent pre-exposure prophylaxis (PrEP) uptake can prevent HIV infections. Despite the high HIV incidence, Chinese key populations have low PrEP uptake and adherence. New interventions are needed to increase PrEP adherence among key populations in China. Co-creation methods are helpful to solicit ideas from the community to solve public health problems. The study protocol aims to describe the design of a stepped-wedge trial and to evaluate the efficacy of co-created interventions to facilitate PrEP adherence among key populations in China.

Methods

The study will develop intervention packages to facilitate PrEP adherence among Chinese key populations using co-creation methods. The study will then evaluate the efficacy of the co-created intervention packages using a stepped-wedge randomized controlled trial. This four-phased closed cohort stepped-wedge design will have four clusters. Each cluster will start intervention at three-month intervals. Seven hundred participants who initiated PrEP will be recruited. Participants will be randomized to the clusters using block randomization. The intervention condition includes receiving co-created interventions in addition to standard of care. The control condition is the standard of care that includes routine clinical assessment every 3 months. All participants will also receive an online follow-up survey every 3 months to record medication adherence and will be encouraged to use a WeChat mini-app for sexual and mental health education throughout the study. The primary outcomes are PrEP adherence and retention in PrEP care throughout the study period. We will examine a hypothesis that a co-created intervention can facilitate PrEP adherence. Generalized linear mixed models will be used for the primary outcome analysis.

Discussion

Developing PrEP adherence interventions in China faces barriers including suboptimal PrEP uptake among key populations, the lack of effective PrEP service delivery models, and insufficient community engagement in PrEP initiatives. Our study design addresses these obstacles by using co-creation to generate social media-based intervention materials and embedding the study design in the local healthcare system. The study outcomes may have implications for policy and intervention practices among CBOs and the medical system to facilitate PrEP adherence among key populations.

Trial registration

The study is registered in Clinical Trial databases in China (ChiCTR2100048981, July 19, 2021) and the US (NCT04754139, February 11, 2021).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-14117-5.

Ethics and regulatory complexities posed by a pragmatic clinical trial: a case study from Lilongwe, Malawi

Background

Pragmatic clinical trials generally rely on real world data and have the potential to generate real world evidence. This approach arose from concerns that many trial results did not adequately inform real world practice. However, maintaining the real world setting during the conduct of a trial and ensuring adequate protection for research participants can be challenging. Best practices in research oversight for pragmatic clinical trials are nascent and underdeveloped, especially in developing countries.

Methods

We use the PRECIS-2 tool to present a case study from Lilongwe in Malawi to describe ethical and regulatory challenges encountered during the conduct of a pragmatic trial and suggest possible solutions.

Results

In this article, we highlight the following six issues: (1) one public facility hosting several pragmatic trials within the same period; (2) research participants refusing financial incentives; (3) inadequate infrastructure and high workload to conduct research; (4) silos among partner organisations involved in delivery of health care; (5) individuals influencing the implementation of revised national guidelines; (6) difficulties with access to electronic medical records.

Conclusion

Multiple stakeholder engagement is critical to the conduct of pragmatic trials, and even with careful stakeholder engagement, continuous monitoring by gatekeepers is essential. In the Malawian context, active engagement of the district research committees can complement the work of the research ethics committees (RECs).

Implementing a behavior management approach in the hospital setting for individuals with challenging behaviors during acute traumatic brain injury

PRIMARY OBJECTIVE

This study aimed to determine if implementing a clinically pragmatic behavior management approach for challenging behaviors during acute TBI reduces use of restraints, security incidences, acute length of stay admission and cost, thereby improving progress to rehabilitation.

RESEARCH DESIGN

This pilot study involved a behavior management group (n = 27), compared with a historical control group (n = 74).

METHOD AND PROCEDURES

The behavior management group received care following the implementation of a behavior management approach for challenging behaviors during acute TBI at two hospitals in South Australia.

MAIN OUTCOMES AND RESULTS

Fidelity results demonstrated high level of adherence to the implemented behavior management approach. The behavior management group had significant lower use of mechanical restraints (p = 0.03), and significantly lower acute admission costs (p = 0.034). Trends in lower pharmacological restraint usage, lower acute hospital length of stay and time from admission to rehabilitation acceptance were recorded.

CONCLUSIONS

These pilot results contribute to improving quality of health care but methodological limitations make outcomes difficult to interpret as a direct result of the intervention. Future studies are required to investigate evidence-based behavior management interventions for acute TBI patients using more rigorous knowledge translation implementation designs.

Rationale and design of a randomized controlled clinical trial; Titration of Oxygen Levels (TOOL) during mechanical ventilation

BACKGROUND

Both hyperoxemia and hypoxemia are deleterious in critically ill patients. Targeted oxygenation is recommended to prevent both of these extremes, however this has not translated to the bedside. Hyperoxemia likely persists more than hypoxemia due to absence of immediate discernible adverse effects, cognitive biases and delay in prioritization of titration.

METHODS

We present the methodology for the Titration Of Oxygen Levels (TOOL) trial, an open label, randomized controlled trial of an algorithm-based FiO2 titration with electronic medical record-based automated alerts. We hypothesize that the study intervention will achieve targeted oxygenation by curbing episodes of hyperoxemia while preventing hypoxemia. In the intervention arm, electronic alerts will be used to titrate FiO2 if SpO2 is ≥94% with FiO2 levels ≥0.4 over 45 min. FiO2 will be titrated per standard practice in the control arm. This study is being carried out with deferred consent. The sample size to determine efficacy is 316 subjects, randomized in a 1:1 ratio to the intervention vs. control arm. The primary outcome is proportion of time during mechanical ventilation spent with FiO2 ≥ 0.4 and SpO2 ≥ 94%. We will also assess proportion of time during mechanical ventilation spent with SpO2 < 88%, duration of mechanical ventilation, length of ICU and hospital stay, hospital mortality, and adherence to electronic alerts as secondary outcomes.

CONCLUSION

This study is designed to evaluate the efficacy of a high fidelity, bioinformatics-based, electronic medical record derived electronic alert system to improve targeted oxygenation in mechanically ventilated patients by reducing excessive FiO2 exposure.

The impact of COVID-19 on pragmatic clinical trials: lessons learned from the NIH Health Care Systems Research Collaboratory

Background

The COVID-19 pandemic has considerably disrupted nearly all aspects of daily life, including healthcare delivery and clinical research. Because pragmatic clinical trials are often embedded within healthcare delivery systems, they may be at high risk of disruption due to the dual impacts on the conduct of both care and research.

Methods

We collected qualitative data using multiple methods to characterize the impact of COVID-19 on the research activities of 14 active pragmatic clinical trials in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. A COVID-19 impact questionnaire was administered electronically to principal investigators in June 2020. Text responses were analyzed thematically, and qualitative summaries were subsequently reviewed by five independent reviewers, who made iterative revisions. Additional COVID-19-related impacts were identified during virtual meetings with trial teams during April–July 2020 and combined with questionnaire responses for analysis.

Results

Impacts of the pandemic were broadly classified into two main types: healthcare operations and social distancing. In some instances, trial delays created statistical challenges, particularly with trials using stepped-wedge designs, and necessitated changing data collection strategies or modifying interventions. The majority of projects used existing stakeholder-driven approaches to adapt interventions. Several benefits of these adaptions were identified, including expanded outreach capabilities and ability to study virtual intervention delivery. All trial teams were able to adapt to pandemic-related modifications.

Conclusion

In a group of 14 ongoing pragmatic clinical trials, there was significant impact of COVID-19 on trial activities. Engaging appropriate stakeholders was critical to designing and implementing trial modifications and making continued safe progress toward meeting research objectives.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06385-8.

Stepped Wedge Cluster Randomized Trials: A Methodological Overview

BACKGROUND

Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials.

METHODS

We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs.

RESULTS

We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed.

CONCLUSIONS

Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.

Acupuncture in the emergency department for pain management: A BraveNet multi-center feasibility study

PURPOSE

Pain accounts for up to 78% of emergency department (ED) patient visits and opioids remain a primary method of treatment despite risks of addiction and adverse effects. While prior acupuncture studies are promising as an alternative opioid-sparing approach to pain reduction, successful conduct of a multi-center pilot study is needed to prepare for a future definitive randomized control trial (RCT).

METHODS

Acupuncture in the Emergency Department for Pain Management (ACUITY) is funded by the National Center for Complementary and Integrative Health. The objectives are to: conduct a multi-center feasibility RCT, examine feasibility of data collection, develop/deploy a manualized acupuncture intervention and assess feasibility/implementation (barrier/facilitators) in 3 EDs affiliated with the BraveNet Practice Based Research Network.Adults presenting to a recruiting ED with acute non-emergent pain (e.g., musculoskeletal, back, pelvic, noncardiac chest, abdominal, flank or head) of ≥4 on a 0-10-point Numeric Rating Scale will be eligible. ED participants (n = 165) will be equally randomized to Acupuncture or Usual Care.At pre-, post-, and discharge time-points, patients will self-assess pain and anxiety using the Numeric Rating Scale. Pain, anxiety, post-ED opioid use and adverse events will be assessed at 1 and 4 weeks. Opioid utilization in the ED and discharge prescriptions will be extracted from patients' electronic medical records.Acupuncture recipients will asked to participate in a brief qualitative interview about 3 weeks after their discharge. ED providers and staff will also be interviewed about their general perspectives/experiences related to acupuncture in the ED and implementation of acupuncture in ACUITY.

RESULTS

Recruitment began on 5/3/21. As of 12/7/21: 84 patients have enrolled, the responsive acupuncture intervention has been developed and deployed, and 26 qualitative interviews have been conducted.

CONCLUSION

Successful conduct of ACUITY will provide the necessary framework for conducting a future, multi-center, definitive RCT of acupuncture in the ED.

CLINICAL TRIALSGOV

NCT04880733 https://clinicaltrials.gov/ct2/show/NCT04880733.

Two weights make a wrong: Cluster randomized trials with variable cluster sizes and heterogeneous treatment effects

In cluster randomized trials (CRTs), the hierarchical nesting of participants (level 1) within clusters (level 2) leads to two conceptual populations: clusters and participants. When cluster sizes vary and the goal is to generalize to a hypothetical population of clusters, the unit average treatment effect (UATE), which averages equally at the cluster level rather than equally at the participant level, is a common estimand of interest. From an analytic perspective, when a generalized estimating equations (GEE) framework is used to obtain averaged treatment effect estimates for CRTs with variable cluster sizes, it is natural to specify an inverse cluster size weighted analysis so that each cluster contributes equally and to adopt an exchangeable working correlation matrix to account for within-cluster correlation. However, such an approach essentially uses two distinct weights in the analysis (i.e. both cluster size weights and covariance weights) and, in this article, we caution that it will lead to biased and/or inefficient treatment effect estimates for the UATE estimand. That is, two weights "make a wrong" or lead to poor estimation characteristics. These findings are based on theoretical derivations, corroborated via a simulation study, and illustrated using data from a CRT of a colorectal cancer screening program. We show that, an analysis with both an independence working correlation matrix and weighting by inverse cluster size is the only approach that always provides valid results for estimation of the UATE in CRTs with variable cluster sizes.

Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

BACKGROUND

In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).

METHODS

Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).

RESULTS

The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.

CONCLUSIONS

Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .

The Electronic Health Record as the Primary Data Source in a Pragmatic Trial: A Case Study

HIGHLIGHTS

Electronic health records are not a single system but a series of overlapping and legacy systems that require time and expertise to use efficiently.Commonly measured patient characteristics such as weight and body mass index are relatively easy to locate for most trial enrollees but less common characteristics, like ejection fraction, are not.Acquiring essential supplementary data-in this trial, state data on hospital admission-can be a lengthy and difficult process.

Power considerations for generalized estimating equations analyses of four-level cluster randomized trials

In this article, we develop methods for sample size and power calculations in four-level intervention studies when intervention assignment is carried out at any level, with a particular focus on cluster randomized trials (CRTs). CRTs involving four levels are becoming popular in healthcare research, where the effects are measured, for example, from evaluations (level 1) within participants (level 2) in divisions (level 3) that are nested in clusters (level 4). In such multilevel CRTs, we consider three types of intraclass correlations between different evaluations to account for such clustering: that of the same participant, that of different participants from the same division, and that of different participants from different divisions in the same cluster. Assuming arbitrary link and variance functions, with the proposed correlation structure as the true correlation structure, closed-form sample size formulas for randomization carried out at any level (including individually randomized trials within a four-level clustered structure) are derived based on the generalized estimating equations approach using the model-based variance and using the sandwich variance with an independence working correlation matrix. We demonstrate that empirical power corresponds well with that predicted by the proposed method for as few as eight clusters, when data are analyzed using the matrix-adjusted estimating equations for the correlation parameters with a bias-corrected sandwich variance estimator, under both balanced and unbalanced designs.

Nursing home leaders' perceptions of a research partnership

BACKGROUND/OBJECTIVE

Partnerships between healthcare providers and researchers may accelerate the translation of interventions into widespread practice by testing them under real-world conditions, but depend on provider's willingness to engage with researchers and ability to fully implement an intervention.

AIM

To understand nursing home leader's motivations for participating in a research study and perceptions of the process and value.

METHODS

After a feasibility study of tuned lighting in a nursing home, we conducted semi-structured telephone interviews with six facility leaders. Interviews were audio-recorded, transcribed, and independently coded by four investigators.

RESULTS

Three themes emerged: (1) The importance of the nursing home's culture and context: the facility had stable leadership, clear processes for prioritizing and implementing new initiatives, and an established interest in the study's topic. (2) The importance of leader's belief in the value of the intervention: leaders perceived research generally and the intervention specifically as positively impacting their facility and residents. (3) The importance of ongoing collaboration and flexibility throughout the study period: leaders served as champions to catalyze the project and overcome implementation barriers.

CONCLUSION

Nursing home leader's perspectives about their participation in a feasibility study underscore the importance of establishing strong researcher-provider partnerships, understanding the environment in which the intervention will be implemented, and employing pragmatic methods that allow for flexibility in response to real-world implementation barriers. We recommend eliciting qualitative information to understand the environment in which an intervention will be implemented and to engage opinion leaders who can inform the protocol and champion its success.