Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis

PRECIS-2 used as an implementation science tool for global environmental health: A cross-sectional evaluation of the Ecolectivos study protocol to reduce burning of household plastic waste in rural Guatemala

BACKGROUND

Randomized controlled trials (RCTs) that evaluate the efficacy of an intervention remain underutilized in community-based environmental health research. RCTs that use a pragmatic design emphasize the effectiveness of interventions in complex, real world settings. Pragmatic trials may be especially relevant when community-based interventions address social and environmental determinants that threaten health equity. The revised Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) is a validated tool developed in 2015 by trialists to ensure that clinical trials are designed to fit their intended purpose, with an assessment of applicability of the trial results to specific contexts. The purpose of this cross-sectional study was to ask Ecolectivos study investigators and external implementation scientists to evaluate the Ecolectivos study protocol using the PRECIS-2 tool prior to the launch of the trial. Ecolectivos is an implementation science study, using a village-level cluster randomized controlled trial design, to assess a behavioral intervention to reduce household plastic waste burning in rural Guatemala.

METHODS

We invited 60 researchers to participate in an online survey between February 2022 and January 2023. Respondents were asked to review the Ecolectivos study protocol and provide scores for the nine PRECIS-2 domains (eligibility, recruitment, setting, organization, flexibility-delivery, flexibility-adherence, follow-up, primary outcome, and primary analysis), with short responses explaining their score. The PRECIS-2 tool is used to assess the degree of pragmatism, ranked on a five-point Likert scale from very explanatory (Checkley W, 2022) to very pragmatic (Ashcraft LE, 2024). Descriptive statistics were used to compare responses between Ecolectivos investigators and external evaluators.

RESULTS

Twenty-five respondents provided data. Among the nine domains, four were rated as pragmatic-eligibility, setting, flexibility-delivery, and primary analysis. Four were evaluated to be equally pragmatic as explanatory-recruitment, organization, flexibility-adherence, and primary outcome. One domain was primarily explanatory in nature-follow-up. Only one domain, eligibility, was statistically significantly different between Ecolectivos investigators and external evaluators, demonstrating that the two groups were broadly consistent in their opinions in eight of the nine study domains. Using the PRECIS-2 tool, we found that our study protocol was viewed as more pragmatic than explanatory, providing evidence to support the pragmatic approach of the Ecolectivos study goals, which is to reduce burning of plastic waste and plastic use in community settings using a behavioral intervention.

CONCLUSIONS

By evaluating the degree of pragmatism within the nine domains, PRECIS-2 guides investigators to think about the applicability of potential results. Investigator assessment and communication regarding intervention protocols for community-level environmental interventions, their degree of pragmatism, and external validity are important for identifying strategies to address complex community problems. Our findings contribute to the growing body of literature that addresses greater research utility through pragmatic trial design, tying community environmental health interventions to the rigor of implementation science strategies.

Comments, suggestions, and criticisms of the Pragmatic Explanatory Continuum Indicator Summary-2 design tool: a citation analysis

INTRODUCTION

The pragmatic explanatory continuum indicator summary (PRECIS) tool, initially published in 2009 and revised in 2015, was created to assist trialists to align their design choices with the intended purpose of their randomised controlled trial (RCT): either to guide real-world decisions between alternative interventions (pragmatic) or to test hypotheses about intervention mechanisms by minimising sources of variation (explanatory). There have been many comments, suggestions, and criticisms of PRECIS-2. This summary will be used to facilitate the development of to the next revision, which is PRECIS-3.

METHODS

We used Web of Science to identify all publication types citing PRECIS-2, published between May 2015 and July 2023. Citations were eligible if they contained 'substantive' suggestions, comments, or criticism of the PRECIS-2 tool. We defined 'substantive' as comments explicitly referencing at least one PRECIS-2 domain or a concept directly linked to an existing or newly proposed domain. Two reviewers independently extracted comments, suggestions, and criticisms, noting their implications for the update. These were discussed among authors to achieve consensus on the interpretation of each comment and its implications for PRECIS-3.

RESULTS

The search yielded 885 publications, and after full-text review, 89 articles met the inclusion criteria. Comments pertained to new domains, changes in existing domains, or were relevant across several or all domains. Proposed new domains included assessment of the comparator arm and a domain to describe blinding. There were concerns about scoring eligibility and recruitment domains for cluster trials. Suggested areas for improvement across domains included the need for more scoring guidance for explanatory design choices.

DISCUSSION

Published comments recognise PRECIS-2's success in aiding trialists with pragmatic or explanatory design choices. Enhancing its implementation and widespread use will involve adding new domains, refining domain definitions, and addressing overall tool issues. This citation review offers valuable user feedback, pivotal for shaping the upcoming version of the PRECIS tool, PRECIS-3.

Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials

Pragmatism in clinical trials is focused on increasing the generalizability of research findings for routine clinical care settings. Hybridism in clinical trials (i.e., assessing both clinical effectiveness and implementation success) is focused on speeding up the process by which evidence-based practices are developed and adopted into routine clinical care. Even though pragmatic trial methodologies and implementation science evolved from very different disciplines, Pragmatic Trials and Hybrid Effectiveness-Implementation Trials share many similar design features. In fact, these types of trials can easily be conflated, creating the potential for investigators to mislabel their trial type or mistakenly use the wrong trial type to answer their research question. Blurred boundaries between trial types can hamper the evaluation of grant applications, the scientific interpretation of findings, and policy-making. Acknowledging that most trials are not pure Pragmatic Trials nor pure Hybrid Effectiveness-Implementation Trials, there are key differences in these trial types and they answer very different research questions. The purpose of this paper is to clarify the similarities and differences of these trial types for funders, researchers, and policy-makers. In addition, recommendations are offered to help investigators choose, label, and operationalize the most appropriate trial type to answer their research question. These recommendations complement existing reporting guidelines for clinical effectiveness trials (TIDieR) and implementation trials (StaRI).

Factors Affecting Post-trial Sustainment or De-implementation of Study Interventions: A Narrative Review

In contrast to traditional randomized controlled trials, embedded pragmatic clinical trials (ePCTs) are conducted within healthcare settings with real-world patient populations. ePCTs are intentionally designed to align with health system priorities leveraging existing healthcare system infrastructure and resources to ease intervention implementation and increase the likelihood that effective interventions translate into routine practice following the trial. The NIH Pragmatic Trials Collaboratory, funded by the National Institutes of Health (NIH), supports the conduct of large-scale ePCT Demonstration Projects that address major public health issues within healthcare systems. The Collaboratory has a unique opportunity to draw on the Demonstration Project experiences to generate lessons learned related to ePCTs and the dissemination and implementation of interventions tested in ePCTs. In this article, we use case studies from six completed Demonstration Projects to summarize the Collaboratory's experience with post-trial interpretation of results, and implications for sustainment (or de-implementation) of tested interventions. We highlight three key lessons learned. First, ineffective interventions (i.e., ePCT is null for the primary outcome) may be sustained if they have other measured benefits (e.g., secondary outcome or subgroup) or even perceived benefits (e.g., staff like the intervention). Second, effective interventions-even those solicited by the health system and/or designed with significant health system partner buy-in-may not be sustained if they require significant resources. Third, alignment with policy incentives is essential for achieving sustainment and scale-up of effective interventions. Our experiences point to several recommendations to aid in considering post-trial sustainment or de-implementation of interventions tested in ePCTs: (1) include secondary outcome measures that are salient to health system partners; (2) collect all appropriate data to allow for post hoc analysis of subgroups; (3) collect experience data from clinicians and staff; (4) engage policy-makers before starting the trial.

Implementation processes of social network interventions for physical activity and sedentary behavior among children and adolescents: a scoping review

BACKGROUND

The characteristics of the implementation process of interventions are essential for bridging the gap between research and practice. This scoping review aims to identify the implementation process of social network interventions (SNI) to address physical activity and sedentary behaviors in children and adolescents.

METHODS

The scoping review was conducted adhering to the established guidelines. The search was carried out in the ERIC, EBSCO, EMBASE, SCOPUS, and Lilacs databases in April 2023. Social network intervention studies in children and adolescents were included, addressing physical activity or sedentary behaviors. Replicability (TIDieR), applicability (PRECIS-2), and generalizability (RE-AIM) were the explored components of the implementation process. Each component was quantitatively and separately analyzed. Then, a qualitative integration was carried out using a narrative method.

RESULTS

Most SNI were theoretically framed on the self-determination theory, used social influence as a social mechanism, and used the individual typology of network intervention. Overall, SNI had strong replicability, tended to be pragmatic, and three RE-AIM domains (reach, adoption (staff), and implementation) showed an acceptable level of the generalizability of findings.

CONCLUSIONS

The analyzed SNI for physical activity and sedentary behaviors in adolescents tended to be reported with high replicability and were conducted pragmatically, i.e., with very similar conditions to real settings. The RE-AIM domains of reach, adoption (staff), and implementation support the generalizability of SNI. Some domains of the principles of implementation strategies of SNI had acceptable external validity (actor, action targets, temporality, dose, and theoretical justification).

Stepped collaborative care versus American College of Surgeons Committee on Trauma required screening and referral for posttraumatic stress disorder: Clinical trial protocol

BACKGROUND

Each year in the US, approximately 1.5-2.5 million individuals are so severely injured that they require inpatient hospital admissions. The American College of Surgeons Committee on Trauma (College) now requires that trauma centers have in place protocols to identify and refer hospitalized patients at risk injury psychological sequelae. Literature review revealed no investigations that have identified optimal screening, intervention, and referral procedures in the wake of the College requirement.

METHODS

The single-site pragmatic trial investigation will individually randomize 424 patients (212 intervention and 212 control) to a brief stepped care intervention versus College required mental health screening and referral control conditions. Blinded follow-up interviews at 1-, 3-, 6-, and 12-months post-injury will assess the symptoms of PTSD and related comorbidity for all patients. The emergency department information exchange (EDIE) will be used to capture population-level automated emergency department and inpatient utilization data for the intent-to-treat sample. The investigation aims to test the primary hypotheses that intervention patients will demonstrate significant reductions in PTSD symptoms and emergency department/inpatient utilization when compared to control patients. The study incorporates a Rapid Assessment Procedure-Informed Clinical Ethnography (RAPICE) implementation process assessment.

CONCLUSIONS

The overarching goal of the investigation is to advance the sustainable delivery of high-quality trauma center mental health screening, intervention, and referral procedures for diverse injury survivors. An end-of-study policy summit will harness pragmatic trial data to inform the capacity for US trauma centers to implement high-quality acute care mental health screening, intervention and referral services for diverse injured patient populations.

TRIAL REGISTRATION

Clinicaltrials.govNCT05632770.