The PRECIS-2 tool seems not to be useful to discriminate the degree of pragmatism of medicine masked trials from that of open-label trials

The need for more pragmatic trials in glaucoma research

AIM

There have been a number of clinical trials in glaucoma research published in the past two decades. Most of these trials were designed to evaluate very specific issues in selected populations placing them in the explanatory end of the pragmatic-explanatory continuum. The purpose of this study was to assess the level of pragmatism of published randomized controlled trials in glaucoma.

METHODS

A PubMed search using 'glaucoma' from 1995 to 2022 and randomized controlled trial (RCT) article type was done. Each study was assessed by three independent examiners using the Pragmatic-Explanatory Continuum Indicator Summary version 2 (PRECIS-2) toolkit. Scores were calculated for each study to determine the level of pragmatism. A summed score ≥36 was indicative of a very pragmatic study.

RESULTS

Thirty-two different articles were included in the analysis. These papers represented 13 different landmark trials. The median PRECIS-2 score was 32 (range, 25 for the Early Manifest Glaucoma Trial (EMGT) to 34 to the Collaborative Normal Tension Glaucoma Study (CNTGS) and the Ocular Hypertension Treatment Study). The Treatment of Advanced Glaucoma Study (TAGS), was considered very pragmatic and scored 33 points.

CONCLUSION

Despite the number of RCTs in glaucoma, there is still a need for more pragmatic studies.

Comments, suggestions, and criticisms of the Pragmatic Explanatory Continuum Indicator Summary-2 design tool: a citation analysis

INTRODUCTION

The pragmatic explanatory continuum indicator summary (PRECIS) tool, initially published in 2009 and revised in 2015, was created to assist trialists to align their design choices with the intended purpose of their randomised controlled trial (RCT): either to guide real-world decisions between alternative interventions (pragmatic) or to test hypotheses about intervention mechanisms by minimising sources of variation (explanatory). There have been many comments, suggestions, and criticisms of PRECIS-2. This summary will be used to facilitate the development of to the next revision, which is PRECIS-3.

METHODS

We used Web of Science to identify all publication types citing PRECIS-2, published between May 2015 and July 2023. Citations were eligible if they contained 'substantive' suggestions, comments, or criticism of the PRECIS-2 tool. We defined 'substantive' as comments explicitly referencing at least one PRECIS-2 domain or a concept directly linked to an existing or newly proposed domain. Two reviewers independently extracted comments, suggestions, and criticisms, noting their implications for the update. These were discussed among authors to achieve consensus on the interpretation of each comment and its implications for PRECIS-3.

RESULTS

The search yielded 885 publications, and after full-text review, 89 articles met the inclusion criteria. Comments pertained to new domains, changes in existing domains, or were relevant across several or all domains. Proposed new domains included assessment of the comparator arm and a domain to describe blinding. There were concerns about scoring eligibility and recruitment domains for cluster trials. Suggested areas for improvement across domains included the need for more scoring guidance for explanatory design choices.

DISCUSSION

Published comments recognise PRECIS-2's success in aiding trialists with pragmatic or explanatory design choices. Enhancing its implementation and widespread use will involve adding new domains, refining domain definitions, and addressing overall tool issues. This citation review offers valuable user feedback, pivotal for shaping the upcoming version of the PRECIS tool, PRECIS-3.

Modern trials are most useful when they are pragmatic and explanatory - there is no continuum

BACKGROUND AND OBJECTIVES

Over half a century ago, the terms "pragmatic" and "explanatory" were introduced to biomedicine by Schwartz and Lellouch, presenting two distinct conceptual approaches to trial design. Today, we frequently say that there are pragmatic trials and there are explanatory trials. Pragmatic trials inform decision-making in practice, and explanatory trials aim to understand the mechanism of an intervention. They are often perceived as diametral extremes of a continuum. In this commentary, we argue that with the digitalization of health care and clinical research, ways for modern trial designs were paved and new avenues opened, and that there is no such continuum.

METHODS, RESULTS AND CONCLUSIONS

Since the groundbreaking work of Schwartz and Lellouch, new approaches and methods have become available that allow researchers to address pragmatic and explanatory questions in parallel in the same trial. Emerging availability of routinely collected "real-world" data, development of decentralized trial techniques, and creation of digital biomarkers allow to observe health outcomes with minimal or no interference in real-world care. This overcomes previous limitations to studying mechanisms of interventions in routine care and makes the idea of a continuum obsolete. We argue that pragmatism and explanatorism need to be understood as two distinct but compatible conceptual dimensions to open new perspectives for using novel technologies to design the most informative clinical trials and make better clinical and regulatory decisions. We base our argument on an analysis of the concept of a continuum and highlight its limitations. We review key trial design features and introduce a new concept that sees explanatory design features as fundamental, invasive or noninvasive, or sufficient or insufficient. We describe their impact on pragmatism and explanatorism and show how multidimensional pragmatic explanatory trials that are most useful are possible today.

Lack of pragmatic attitude of self-labelled pragmatic trials on manual therapy: a methodological review

BACKGROUND

Pragmatic randomized controlled trials are getting more interest to improve trials' external validity. This study aimed to assess how pragmatic the design of the self-labelled pragmatic randomised controlled trials in the manual therapy field is.

METHODS

We searched MEDLINE and the Cochrane Central Register of Controlled Trials for self-labelled pragmatic randomised controlled trials in the manual therapy field until January 2024 were included. Two independent reviewers collected and extracted data related to the intention of the trial, the rationale for the intervention, and specific features of the trial and performed an assessment using the PRECIS-2 tool.

RESULTS

Of 39 self-labelled pragmatic trials, the mean PRECIS-2 score was 3.5 (SD: 0.6). Choice of outcome measures, how the interventions were performed, the follow-up of the participants and how all the available data were included in the statistical analysis were the domains rated as most 'pragmatic'. Participants' eligibility, recruitment, and setting obtained lower scores. Less than 25% of the trials claimed that the aim was to investigate an intervention under real-world conditions and to make clinical decisions about its effectiveness. In the 21% of the sample the authors described neither the proof-of-concept of the intervention nor the state of previous studies addressing related research questions.

CONCLUSIONS

Self-labelled pragmatic randomised controlled trials showed a moderately pragmatic attitude. Beyond the label 'pragmatic', the description of the intention of the trial and the context of every PRECIS-2 domain is crucial to understanding the real pragmatism of a trial.

Using PRECIS-2 in Chinese herbal medicine randomized controlled trials for irritable bowel syndrome: A methodological exploration based on literature

Background

The pragmatism levels of randomized controlled trials (RCTs) mean how similar the interventions delivered in the trial setting match those in the setting where the results will be applied. We aimed to investigate the association between the consistency of pragmatism among the characteristics of RCT design and its effect size of results in Chinese herbal medicine (CHM) for irritable bowel syndrome (IBS).

Methods

Eight English and Chinese language databases were searched for RCTs on CHM for IBS. Six reviewers independently assessed the pragmatism of trials using the pragmatic-explanatory continuum indicator summary 2 (PRECIS-2) tool. The consistency of pragmatism levels among the characteristics of RCT design was calculated using the coefficient of variation. Linear regression models were adopted to explore influence factors of the pragmatism of RCTs.

Results

78 RCTs were included. The level of consistency in the pragmatism for RCT's design was significantly correlated with the effect size of the results (binary outcome, r = -0.413; P = 0.005; continuous outcome, r = -0.779, P < 0.001). PRECIS-2 score was higher in trials with individualized interventions than fixed interventions (3.29 [0.32] vs 2.90 [0.32]; Cohen's d relative effect size, 0.52; P < 0.001) and in standard or usual-treatment-controlled trials than placebo-controlled (3.05 [0.37] vs 2.83 [0.28]; Cohen's d relative effect size, 0.32; P = 0.048).

Conclusion

The consistency of pragmatism level across the 9 domains of the PRECIS-2 tool in CHM IBS RCTs was positively correlated with the effect size of the results.

Generalizability of randomized controlled trials in primary health care: Applying the PRECIS-2 tool on published protocols

RATIONALE

Pragmatic design may facilitate the generalizability of effectiveness of randomized controlled trials (RCTs) in primary health care (PHC).

AIMS AND OBJECTIVES

The aim of this study was to investigate whether published protocols in PHC were designed pragmatically and to explore whether specific trial characteristics may be related to a pragmatic design.

METHODS

Using the Pragmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2), we assessed pragmatism for 123 published RCT protocols. For each domain, we calculated the mean score with the 95% confidence interval (95% CI). Interrater reliability was assessed by weighted κ-coefficient with 95% CI. We examined potential associations of published protocol characteristics with overall pragmatism by performing univariate and multivariate analyses.

RESULTS

We observed the highest score for primary analysis (4.66, 95% CI: 4.51, 4.82). The eligibility score was intermediate (3.16, 95% CI: 3.01, 3.32). Domains with scores towards the explanatory side included organization (2.50, 95% CI: 2.36, 2.63), flexibility of delivery (2.74, 95% CI: 2.60, 2.88) and flexibility of adherence (3.00, 95% CI: 2.83, 3.17). Interrater agreement was good (κ = 0.61; 95% CI: 0.34, 0.80; p < 0.001). Higher sample sizes were correlated to a pragmatic design (odds ratio: 6.86, 95% CI: 1.64, 28.75; p = 0.04).

CONCLUSION

Most RCT protocols were rated as intermediate in the pragmatic-explanatory continuum. Future research may guide all stakeholders on how best to incorporate the level of pragmatism in the interpretation of the results so that the trials are more likely to be applicable in real-world settings.

An analysis of published trials found that current use of pragmatic trial labels is uninformative

OBJECTIVES

Randomized trials labelled as "pragmatic" are attractive to funders, patients, and clinicians as the label implies that the results are directly applicable to clinical care. We examined how authors justify use of the label (e.g., by referring to one or more PRECIS [PRagmatic Explanatory Continuum Indicator Summary]-2 domains).

STUDY DESIGN AND SETTING

We reviewed primary trial reports published 2014-2019, registered in ClinicalTrials.gov and using the pragmatic label anywhere in the report.

RESULTS

Among 415 trials, the label was justified by reference to at least one design element in 282 (68.0%); of these, 240 (85.1%) referenced trial characteristics that can be mapped to one or more of the PRECIS-2 domains, most commonly eligibility (91, 32.3%), setting (90, 31.9%), flexibility delivery (89, 31.6%), and organization (75, 26.6%); 42 (14.9%) referenced characteristics that are not PRECIS-2 domains, most commonly type of intervention/comparator (48, 17%), recruitment without consent (22, 7.8%), routinely collected data (22, 7.8%), and cluster randomization (20, 7.1%). Most reports referenced only one or two design elements. Overall, 9/415 (2.2%) provided PRECIS wheels.

CONCLUSION

Current use of pragmatic labels is uninformative. Authors should clarify the decision the trial is intended to support and include a PRECIS-2 table to make the design transparent.