Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial

Development of a conceptual framework for defining trial efficiency

BACKGROUND

Globally, there is a growing focus on efficient trials, yet numerous interpretations have emerged, suggesting a significant heterogeneity in understanding "efficiency" within the trial context. Therefore in this study, we aimed to dissect the multifaceted nature of trial efficiency by establishing a comprehensive conceptual framework for its definition.

OBJECTIVES

To collate diverse perspectives regarding trial efficiency and to achieve consensus on a conceptual framework for defining trial efficiency.

METHODS

From July 2022 to July 2023, we undertook a literature review to identify various terms that have been used to define trial efficiency. We then conducted a modified e-Delphi study, comprising an exploratory open round and a subsequent scoring round to refine and validate the identified items. We recruited a wide range of experts in the global trial community including trialists, funders, sponsors, journal editors and members of the public. Consensus was defined as items rated "without disagreement", measured by the inter-percentile range adjusted for symmetry through the UCLA/RAND approach.

RESULTS

Seventy-eight studies were identified from a literature review, from which we extracted nine terms related to trial efficiency. We then used review findings as exemplars in the Delphi open round. Forty-nine international experts were recruited to the e-Delphi panel. Open round responses resulted in the refinement of the initial nine terms, which were consequently included in the scoring round. We obtained consensus on all nine items: 1) four constructs that collectively define trial efficiency containing scientific efficiency, operational efficiency, statistical efficiency and economic efficiency; and 2) five essential building blocks for efficient trial comprising trial design, trial process, infrastructure, superstructure, and stakeholders.

CONCLUSIONS

This is the first attempt to dissect the concept of trial efficiency into theoretical constructs. Having an agreed definition will allow better trial implementation and facilitate effective communication and decision-making across stakeholders. We also identified essential building blocks that are the cornerstones of an efficient trial. In this pursuit of understanding, we are not only unravelling the complexities of trial efficiency but also laying the groundwork for evaluating the efficiency of an individual trial or a trial system in the future.

The future of liver transplantation

Over the last 50 years, liver transplantation has evolved into a procedure routinely performed in many countries worldwide. Those able to access this therapy frequently experience a miraculous risk-benefit ratio, particularly if they face the imminently life-threatening disease. Over the decades, the success of liver transplantation, with dramatic improvements in early posttransplant survival, has aggressively driven demand. However, despite the emergence of living donors to augment deceased donors as a source of organs, supply has lagged far behind demand. As a result, rationing has been an unfortunate focus in recent decades. Recent shifts in the epidemiology of liver disease combined with transformative innovations in liver preservation suggest that the underlying premise of organ shortage may erode in the foreseeable future. The focus will sharpen on improving equitable access while mitigating constraints related to workforce training, infrastructure for organ recovery and rehabilitation, and their associated costs. Research efforts in liver preservation will undoubtedly blossom with the aim of optimizing both the timing and conditions of transplantation. Coupled with advances in genetic engineering, regenerative biology, and cellular therapies, the portfolio of innovation, both broad and deep, offers the promise that, in the future, liver transplantation will not only be broadly available to those in need but also represent a highly durable life-saving therapy.

Application of desirability of outcome ranking to the milking in non-vigorous infants trial

OBJECTIVES

Neonatal trials have traditionally used binary composite short-term (such as death or bronchopulmonary dysplasia) or longer-term (such as death or severe neurodevelopmental impairment) outcomes. We applied the Desirability Of Outcome Ranking (DOOR) method to rank the overall patient outcome by best (no morbidities) to worst (death).

STUDY DESIGN

Using a completed large multicenter trial (Milking In Non-Vigorous Infants [MINVI]) of umbilical cord milking (UCM) vs. early cord clamping (ECC), we applied the DOOR methodology to neonatal outcomes. Six outcomes were chosen and ranked: no interventions or NICU admission (most desirable); received initial cardiorespiratory support at birth; neonatal intensive care unit (NICU) admission for predefined criteria; mild hypoxic-ischemic encephalopathy (HIE); moderate to severe HIE; and death (least desirable).

RESULTS

1524 non-vigorous newborns born between 35 and 42 weeks' gestation had data for analysis. The DOOR distribution was different between the UCM and ECC arms, with a significantly greater probability (55.8 % [95 % CI 53.1-58.5 %; p < 0.0001]) of a randomly selected neonate having a more desirable outcome if they were in the UCM arm. DOOR probabilities of averting individual adverse outcomes such as NICU admission for predefined criteria (52.8 %; 95%CI 50.5-55.1 %) and cardiorespiratory support (54.0 %; 95%CI 51.6-56.4 %) were significantly higher among those in the UCM group.

CONCLUSION

DOOR provides an overall assessment of the benefits and harms with greater insight than typical binary composite measures to clinicians and parents when evaluating an intervention. Future neonatal trials should consider the a priori use of the DOOR methodology to evaluate trial outcomes.

Optimising the analysis of vascular prevention trials: Re-Assessment of the TARDIS trial, the first prevention trial to adopt an ordinal primary outcome measure

Background

Ordinalised vascular outcomes incorporating event severity are more informative than binary outcomes that just include event numbers. The TARDIS trial was the first vascular prevention study to use an ordinalised vascular outcome as its primary efficacy and safety measures and collected severity information for other vascular events.

Methods

TARDIS was an international prospective randomised open-label blinded-endpoint trial assessing one month of intensive versus guideline antiplatelet therapy in patients with acute non-cardioembolic stroke or TIA. Vascular events and their severity were recorded up to final follow-up at 90 days post randomisation. For each outcome, statistical techniques compared ordinal/continuous (10 models) and dichotomous (5 models) analyses; results were then ranked with the smallest p-value being given the smallest rank. Outcomes were also assessed within the pre-defined subgroup of participants with mild stroke (NIHSS≤3), or TIA recruited within 24 h.

Results

Ordinal versions of vascular event outcomes were created in 3096 participants for stroke, myocardial infarction, major cardiac events, bleeding events, serious adverse events and venous thromboembolism (VTE), with 32 outcomes being created overall (29 in the subgroup population due to the absence of VTE events). Overall, the tests run on ordinal outcomes tended to rank higher than tests performed on binary outcomes. 764 (24.7%) participants were recruited within 24 h of a mild stroke/TIA; again, tests run on ordinal outcomes ranked higher.

Conclusions

In TARDIS, tests performed on ordinal vascular outcomes tended to attain a higher rank than those performed on binary outcomes.

Trial registration

ISRCTN47823388.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Study protocol for a randomized clinical trial to assess 7 versus 14-days of treatment for Pseudomonas aeruginosa bloodstream infections (SHORTEN-2 trial)

BACKGROUND

Research priorities in Antimicrobial Stewardship (AMS) have rapidly evolved in the last decade. The need for a more efficient use of antimicrobials have fueled plenty of studies to define the optimal duration for antibiotic treatments, and yet, there still are large areas of uncertainty in common clinical scenarios. Pseudomonas aeruginosa has been pointed as a priority for clinical research, but it has been unattended by most randomized trials tackling the effectiveness of short treatments. The study protocol of the SHORTEN-2 trial is presented as a practical example of new ways to approach common obstacles for clinical research in AMS.

OBJECTIVE

To determine whether a 7-day course of antibiotics is superior to 14-day schemes for treating bloodstream infections by P. aeruginosa (BSI-PA).

METHODS

A superiority, open-label, randomized controlled trial will be performed across 30 Spanish hospitals. Adult patients with uncomplicated BSI-PA will be randomized to receive a 7 versus 14-day course of any active antibiotic. The primary endpoint will be the probability for the 7-day group of achieving better outcomes than the control group, assessing altogether clinical effectiveness, severe adverse events, and antibiotic exposure through a DOOR/RADAR analysis. Main secondary endpoints include treatment failure, BSI-PA relapses, and mortality. A superiority design was set for the primary endpoint and non-inferiority for treatment failure, resulting in a sample size of 304 patients.

CONCLUSIONS

SHORTEN-2 trial aligns with some of the priorities for clinical research in AMS. The implementation of several methodological innovations allowed overcoming common obstacles, like feasible sample sizes or measuring the clinical impact and unintended effects.

TRIAL REGISTRATION

EudraCt: 2021-003847-10; ClinicalTrials.gov: NCT05210439.

A desirability of outcome ranking (DOOR) for periprosthetic joint infection - a Delphi analysis

Background: Treatment outcomes in studies on prosthetic joint infection are generally assessed using a dichotomous outcome relating to treatment success or failure. These outcome measures neither include patient-centred outcome measures including joint function and quality of life, nor do they account for adverse effects of treatment. A desirability of outcome ranking (DOOR) measure can include these factors and has previously been proposed and validated for other serious infections. We aimed to develop a novel DOOR for prosthetic joint infections (PJIs). Methods: The Delphi method was used to develop a DOOR for PJI research. An international working group of 18 clinicians (orthopaedic surgeons and infectious disease specialists) completed the Delphi process. The final DOOR comprised the dimensions established to be most important by consensus with > 75  % of participant agreement. Results: The consensus DOOR comprised four main dimensions. The primary dimension was patient-reported joint function. The secondary dimensions were infection cure and mortality. The final dimension of quality of life was selected as a tie-breaker. Discussion: A desirability of outcome ranking for periprosthetic joint infection has been proposed. It focuses on patient-centric outcome measures of joint function, cure and quality of life. This DOOR provides a multidimensional assessment to comprehensively rank outcomes when comparing treatments for prosthetic joint infection.

Design and Analysis of Studies Based on Hierarchical Composite Endpoints: Insights from the DARE-19 Trial

Background/Aim

DARE-19 (NCT04350593) was a randomized trial studying the effects of dapagliflozin, an SGLT2 inhibitor, in hospitalized patients with COVID-19 pneumonia and cardiometabolic risk factors. The conduct of DARE-19 offered the opportunity to define an innovative and clinically meaningful endpoint in a new disease that would best reflect the known profile of dapagliflozin, accompanied by the statistical challenges of analysis and interpretation of such a novel endpoint.

Methods

Hierarchical composite endpoints (HCEs) are based on clinical outcomes which, unlike traditional composite endpoints incorporate ranking of components according to clinical importance. Design of an HCE requires the clinical considerations specific to the therapeutic area under study and the mechanism of action of the investigational treatment. Statistical aspects for the clinical endpoints include the proper definition of the estimand as suggested by ICH E9(R1) for the precise specification of the treatment effect measured by an HCE.

Results

We describe the estimand of the DARE-19 trial, where an HCE was constructed to capture the treatment effect of dapagliflozin in hospitalized patients with COVID-19, and was analyzed using a win odds. Practical aspects of designing new studies based on an HCE are described. These include sample size, power, and minimal detectable effect calculations for an HCE based on the win odds analysis, as well as handling of missing data and the clinical interpretability of the win odds in relation to the estimand.

Conclusions

HCEs are flexible endpoints that can be adapted for use in different therapeutic areas, with win odds as the analysis method. DARE-19 is an example of a COVID-19 trial with an HCE as one of the primary endpoints for estimating a clinically interpretable treatment effect in the COVID-19 setting.

Short- vs Standard-Course Outpatient Antibiotic Therapy for Community-Acquired Pneumonia in Children: The SCOUT-CAP Randomized Clinical Trial

IMPORTANCE

Childhood community-acquired pneumonia (CAP) is usually treated with 10 days of antibiotics. Shorter courses may be effective with fewer adverse effects and decreased potential for antibiotic resistance.

OBJECTIVE

To compare a short (5-day) vs standard (10-day) antibiotic treatment strategy for CAP in young children.

DESIGN, SETTING, AND PARTICIPANTS

Randomized double-blind placebo-controlled clinical trial in outpatient clinic, urgent care, or emergency settings in 8 US cities. A total of 380 healthy children aged 6 to 71 months with nonsevere CAP demonstrating early clinical improvement were enrolled from December 2, 2016, to December 16, 2019. Data were analyzed from January to September 2020.

INTERVENTION

On day 6 of their originally prescribed therapy, participants were randomized 1:1 to receive 5 days of matching placebo or 5 additional days of the same antibiotic.

MAIN OUTCOMES AND MEASURES

The primary end point was the end-of-treatment response adjusted for duration of antibiotic risk (RADAR), a composite end point that ranks each child's clinical response, resolution of symptoms, and antibiotic-associated adverse effects in an ordinal desirability of outcome ranking (DOOR). Within each DOOR rank, participants were further ranked by the number of antibiotic days, assuming that shorter antibiotic durations were more desirable. Using RADAR, the probability of a more desirable outcome was estimated for the short- vs standard-course strategy. In a subset of children, throat swabs were collected between study days 19 and 25 to quantify antibiotic resistance genes in oropharyngeal flora.

RESULTS

A total of 380 children (189 randomized to short course and 191 randomized to standard course) made up the study population. The mean (SD) age was 35.7 (17.2) months, and 194 participants (51%) were male. Of the included children, 8 were Asian, 99 were Black or African American, 234 were White, 32 were multiracial, and 7 were of unknown or unreported race; 33 were Hispanic or Latino, 344 were not Hispanic or Latino, and 3 were of unknown or unreported ethnicity. There were no differences between strategies in the DOOR or its individual components. Fewer than 10% of children in either strategy had an inadequate clinical response. The short-course strategy had a 69% (95% CI, 63-75) probability of a more desirable RADAR outcome compared with the standard-course strategy. A total of 171 children were included in the resistome analysis. The median (range) number of antibiotic resistance genes per prokaryotic cell (RGPC) was significantly lower in the short-course strategy compared with the standard-course strategy for total RGPC (1.17 [0.35-2.43] vs 1.33 [0.46-11.08]; P = .01) and β-lactamase RGPC (0.55 [0.18-1.24] vs 0.60 [0.21-2.45]; P = .03).

CONCLUSIONS AND RELEVANCE

In this study, among children responding to initial treatment for outpatient CAP, a 5-day antibiotic strategy was superior to a 10-day strategy. The shortened approach resulted in similar clinical response and antibiotic-associated adverse effects, while reducing antibiotic exposure and resistance.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02891915.

Using a Composite Maternal-Infant Outcome Measure in Tuberculosis-Prevention Studies Among Pregnant Women

BACKGROUND

Tuberculosis (TB-)-preventive therapy (TPT) among pregnant women reduces risk of TB in mothers and infants, but timing of initiation should consider potential adverse effects. We propose an analytical approach to evaluate the risk-benefit of interventions.

METHODS

A novel outcome measure that prioritizes maternal and infant events was developed with a 2-stage Delphi survey, where a panel of stakeholders assigned scores from 0 (best) to 100 (worst) based on perceived desirability. Using data from TB APPRISE, a trial among pregnant women living with human immunodeficiency virus (WLWH) that randomized the timing of initiation of isoniazid, antepartum versus postpartum, was evaluated.

RESULTS

The composite outcome scoring/ranking system categorized mother-infant paired outcomes into 8 groups assigned identical median scores by stakeholders. Maternal/infant TB and nonsevere adverse pregnancy outcomes were assigned similar scores. Mean (SD) composite outcome scores were 43.7 (33.0) and 41.2 (33.7) in the antepartum and postpartum TPT initiation arms, respectively. However, a modifying effect of baseline antiretroviral regimen was detected (P = .049). When women received nevirapine, composite scores were higher (worse outcomes) in the antepartum versus postpartum arms (adjusted difference, 14.3; 95% confidence interval [CI], 2.4-26.2; P = .02), whereas when women received efavirenz there was no difference by timing of TPT (adjusted difference, .62; 95% CI, -3.2-6.2; P = .53).

CONCLUSIONS

For TPT, when used by otherwise healthy persons, preventing adverse events is paramount from the perspective of stakeholders. Among pregnant WLWH in high-TB-burden regions, it is important to consider the antepartum antiretroviral regimen taken when deciding when to initiate TPT. Clinical Trials Registration. NCT01494038 (IMPAACT P1078).