Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study

OBJECTIVES

Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods.

STUDY DESIGN AND SETTING

We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions.

RESULTS

Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring.

CONCLUSION

Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.

Characteristics, Progression, and Output of Randomized Platform Trials: A Systematic Review

Importance

Platform trials have become increasingly common, and evidence is needed to determine how this trial design is actually applied in current research practice.

Objective

To determine the characteristics, progression, and output of randomized platform trials.

Evidence Review

In this systematic review of randomized platform trials, Medline, Embase, Scopus, trial registries, gray literature, and preprint servers were searched, and citation tracking was performed in July 2022. Investigators were contacted in February 2023 to confirm data accuracy and to provide updated information on the status of platform trial arms. Randomized platform trials were eligible if they explicitly planned to add or drop arms. Data were extracted in duplicate from protocols, publications, websites, and registry entries. For each platform trial, design features such as the use of a common control arm, use of nonconcurrent control data, statistical framework, adjustment for multiplicity, and use of additional adaptive design features were collected. Progression and output of each platform trial were determined by the recruitment status of individual arms, the number of arms added or dropped, and the availability of results for each intervention arm.

Findings

The search identified 127 randomized platform trials with a total of 823 arms; most trials were conducted in the field of oncology (57 [44.9%]) and COVID-19 (45 [35.4%]). After a more than twofold increase in the initiation of new platform trials at the beginning of the COVID-19 pandemic, the number of platform trials has since declined. Platform trial features were often not reported (not reported: nonconcurrent control, 61 of 127 [48.0%]; multiplicity adjustment for arms, 98 of 127 [77.2%]; statistical framework, 37 of 127 [29.1%]). Adaptive design features were only used by half the studies (63 of 127 [49.6%]). Results were available for 65.2% of closed arms (230 of 353). Premature closure of platform trial arms due to recruitment problems was infrequent (5 of 353 [1.4%]).

Conclusions and Relevance

This systematic review found that platform trials were initiated most frequently during the COVID-19 pandemic and declined thereafter. The reporting of platform features and the availability of results were insufficient. Premature arm closure for poor recruitment was rare.

Bridging the gap: identifying factors impacting mRNA severe acute respiratory syndrome coronavirus 2 vaccine booster response in people with HIV-1

OBJECTIVES

This study aimed to investigate the association of demographic and clinical characteristics, including HIV-specific parameters with the antibody response to a third dose of a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) mRNA vaccine in people with HIV-1 (PWH).

DESIGN

Post hoc analysis of data collected during the observational extension of the COrona VaccinE tRiAL pLatform trial (COVERALL-2) nested into the Swiss HIV Cohort Study (SHCS).

METHODS

Serological measurements were conducted on a total of 439 PWH who had received a third dose of either mRNA-1273 (Moderna) or BNT162b2 (Pfizer-BioNTech) SARS-CoV-2 vaccine. Antibody reactivity was assessed using the multifactorial ABCORA immunoassay that defines SARS-CoV-2 seroconversion and predicts neutralization activity. The association between log transformed antibody reactivity and various baseline factors, including vaccine type, demographics, immune and viral status, smoking status, comorbidities, infection history, and co-medication with chemotherapy and immunosuppressive drugs, was investigated using a multivariable linear regression model.

RESULTS

Antibody response to third SARS-CoV-2 vaccination was significantly lower among PWH with CD4 + cell count less than 350 cells/μl [ratio of means 0.79; 95% confidence interval (CI) 0.65-0.95]. Having a detectable HIV-1 viral load at least 50 copies/ml and being on concurrent chemotherapy was associated with an overall lower humoral immune response (ratio of means 0.75; 95% CI 0.57-1.00 and 0.34; 95% CI 0.22-0.52, respectively).

CONCLUSION

The study highlights the importance of optimal antiretroviral treatment for PWH, emphasizing the need for timely intervention to enhance the vaccine immunogenicity in this population. Moreover, it underscores the significance of sequential mRNA vaccination and provides important evidence for informing vaccine guidelines.

Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors

OBJECTIVES

To investigate the impact of potential risk of bias elements on effect estimates in randomized trials.

STUDY DESIGN AND SETTING

We conducted a systematic survey of meta-epidemiological studies examining the influence of potential risk of bias elements on effect estimates in randomized trials. We included only meta-epidemiological studies that either preserved the clustering of trials within meta-analyses (compared effect estimates between trials with and without the potential risk of bias element within each meta-analysis, then combined across meta-analyses; between-trial comparisons), or preserved the clustering of substudies within trials (compared effect estimates between substudies with and without the element, then combined across trials; within-trial comparisons). Separately for studies based on between- and within-trial comparisons, we extracted ratios of odds ratios (RORs) from each study and combined them using a random-effects model. We made overall inferences and assessed certainty of evidence based on Grading of Recommendations, Assessment, development, and Evaluation and Instrument to assess the Credibility of Effect Modification Analyses.

RESULTS

Forty-one meta-epidemiological studies (34 of between-, 7 of within-trial comparisons) proved eligible. Inadequate random sequence generation (ROR 0.94, 95% confidence interval [CI] 0.90-0.97) and allocation concealment (ROR 0.92, 95% CI 0.88-0.97) probably lead to effect overestimation (moderate certainty). Lack of patients blinding probably overestimates effects for patient-reported outcomes (ROR 0.36, 95% CI 0.28-0.48; moderate certainty). Lack of blinding of outcome assessors results in effect overestimation for subjective outcomes (ROR 0.69, 95% CI 0.51-0.93; high certainty). The impact of patients or outcome assessors blinding on other outcomes, and the impact of blinding of health-care providers, data collectors, or data analysts, remain uncertain. Trials stopped early for benefit probably overestimate effects (moderate certainty). Trials with imbalanced cointerventions may overestimate effects, while trials with missing outcome data may underestimate effects (low certainty). Influence of baseline imbalance, compliance, selective reporting, and intention-to-treat analysis remain uncertain.

CONCLUSION

Failure to ensure random sequence generation or adequate allocation concealment probably results in modest overestimates of effects. Lack of patients blinding probably leads to substantial overestimates of effects for patient-reported outcomes. Lack of blinding of outcome assessors results in substantial effect overestimation for subjective outcomes. For other elements, though evidence for consistent systematic overestimate of effect remains limited, failure to implement these safeguards may still introduce important bias.

Adjunct prednisone in community-acquired pneumonia: 180-day outcome of a multicentre, double-blind, randomized, placebo-controlled trial

BACKGROUND

Several trials and meta-analyses found a benefit of adjunct corticosteroids for community-acquired pneumonia with respect to short-term outcome, but there is uncertainty about longer-term health effects. Herein, we evaluated clinical outcomes at long term in patients participating in the STEP trial (Corticosteroid Treatment for Community-Acquired Pneumonia).

METHODS

This predefined secondary analysis investigated 180-day outcomes in 785 adult patients hospitalized with community-acquired pneumonia included in STEP, a randomised, placebo-controlled, double-blind trial. The primary endpoint was time to death from any cause at 180 days verified by telephone interview. Additional secondary endpoints included pneumonia-related death, readmission, recurrent pneumonia, secondary infections, new hypertension, and new insulin dependence.

RESULTS

From the originally included 785 patients, 727 were available for intention-to-treat analysis at day 180. There was no difference between groups with respect to time to death from any cause (HR for corticosteroid use 1.15, 95% CI 0.68 to 1.95, p = 0.601). Compared to placebo, corticosteroid-treated patients had significantly higher risks for recurrent pneumonia (OR 2.57, 95% CI 1.29 to 5.12, p = 0.007), secondary infections (OR 1.94, 95% CI 1.25 to 3.03, p = 0.003) and new insulin dependence (OR 8.73, 95% CI 1.10 to 69.62, p = 0.041). There was no difference regarding pneumonia-related death, readmission and new hypertension.

CONCLUSIONS

In patients with community-acquired pneumonia, corticosteroid use was associated with an increased risk for recurrent pneumonia, secondary infections and new insulin dependence at 180 days. Currently, it is uncertain whether these long-term adverse effects outweigh the short-term effects of corticosteroids in moderate CAP.

TRIAL REGISTRATION

This trial was registered with ClinicalTrials. gov, number NCT00973154 before the recruitment of the first patient. First posted: September 9, 2009. Last update posted: April 21, 2015.

Antibody Response After the Third SARS-CoV-2 Vaccine in Solid Organ Transplant Recipients and People Living With HIV (COVERALL-2)

Background

After basic immunization with 2 mRNA SARS-CoV-2 vaccine doses, only a small proportion of patients who are severely immunocompromised generate a sufficient antibody response. Hence, we assessed the additional benefit of a third SARS-CoV-2 vaccine in patients with different levels of immunosuppression.

Methods

In this observational extension of the COVERALL trial (Corona Vaccine Trial Platform), we recruited patients from the Swiss HIV Cohort Study and the Swiss Transplant Cohort Study (ie, lung and kidney transplant recipients). We collected blood samples before and 8 weeks after the third SARS-CoV-2 vaccination with either mRNA-1273 (Moderna) or BNT162b2 (Pfizer-BioNTech). The primary outcome was the proportion of participants showing an antibody response (Elecsys Anti-SARS-CoV-2 S test; threshold ≥100 U/mL) 8 weeks after the third SARS-CoV-2 vaccination. We also compared the proportion of patients who reached the primary outcome from basic immunization (the first and second vaccines) to the third vaccination.

Results

Nearly all participants (97.2% [95% CI, 95.9%-98.6%], 564/580) had an antibody response. This response was comparable between mRNA-1273 (96.1% [95% CI, 93.7%-98.6%], 245/255) and BNT162b2 (98.2% [95% CI, 96.7%-99.6%], 319/325). Stratification by cohort showed that 99.8% (502/503) of people living with HIV and 80.5% (62/77) of recipients of solid organ transplants achieved the primary endpoint. The proportion of patients with an antibody response in solid organ transplant recipients improved from the second vaccination (22.7%, 15/66) to the third (80.5%, 62/77).

Conclusions

People living with HIV had a high antibody response. The third vaccine increased the proportion of solid organ transplant recipients with an antibody response. Clinical Trials Registration. NCT04805125 (ClinicalTrials.gov).

Community-based type 2 diabetes care by lay village health workers in rural Lesotho: protocol for a cluster-randomized trial within the ComBaCaL cohort study (ComBaCaL T2D TwiC)

BACKGROUND

Type 2 diabetes (T2D) poses a growing public health burden, especially in low- and middle-income countries (LMICs). Task-shifting to lay village health workers (VHWs) and the use of digital clinical decision support systems (CDSS) are promising approaches to tackle the current T2D care gap in LMICs. However, evidence on the effectiveness of lay worker-led T2D care models, in which VHWs initiate and monitor drug treatment in addition to community-based screening and referral services, is lacking.

METHODS

We are conducting a cluster-randomized trial nested within the Community-Based Chronic Disease Care Lesotho (ComBaCaL) cohort study (NCT05596773) using the trial within cohort (TwiC) design to assess the effectiveness of a VHW-led, CDSS-assisted T2D care model in rural Lesotho. Participants are non-pregnant members of the ComBaCaL cohort study with T2D. The ComBaCaL cohort study is conducted in approximately 100 villages in two rural districts in Lesotho and is managed by trained and supervised VHWs. In intervention villages, VHWs offer a community-based T2D care package including lifestyle counselling, first-line oral antidiabetic, lipid-lowering, and antiplatelet treatment guided by a tablet-based CDSS to participants who are clinically eligible, as well as treatment support to participants who prefer or clinically require facility-based T2D care. In control clusters, all participants will be referred to a health facility for T2D management. The primary endpoint is the mean glycosylated haemoglobin (HbA1c) 12 months after enrolment. Secondary endpoints include the 10-year risk for cardiovascular events estimated using the World Health Organization risk prediction tool.

DISCUSSION

The trial was launched on May 13, 2023, and has enrolled 226 participants at the date of submission (October 6, 2023). To our knowledge, the trial is the first to assess task-shifting of T2D care to VHWs at the community level, including the prescription of first-line antidiabetic, lipid-lowering, and antiplatelet medication in sub-Saharan Africa, and will thus provide the missing evidence on the effectiveness of such a T2D care model in this setting. The study is operating within the established Lesotho VHW programme. Similar community health worker programmes which exist across sub-Saharan Africa may benefit from the findings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05743387. Registered on February 24 2023.

Evaluating the impact of including non-randomised studies of interventions in meta-analysis of randomised controlled trials: a protocol for a meta-epidemiological study

INTRODUCTION

Although interest in including non-randomised studies of interventions (NRSIs) in meta-analysis of randomised controlled trials (RCTs) is growing, estimates of effectiveness obtained from NRSIs are vulnerable to greater bias than RCTs. The objectives of this study are to: (1) explore how NRSIs can be integrated into a meta-analysis of RCTs; (2) assess concordance of the evidence from non-randomised and randomised trials and explore factors associated with agreement; and (3) investigate the impact on estimates of pooled bodies of evidence when NRSIs are included.

METHODS AND ANALYSIS

We will conduct a systematic survey of 210 systematic reviews that include both RCTs and NRSIs, published from 2017 to 2022. We will randomly select reviews, stratified in a 1:1 ratio by Core vs non-Core clinical journals, as defined by the National Library of Medicine. Teams of paired reviewers will independently determine eligibility and abstract data using standardised, pilot-tested forms. The concordance of the evidence will be assessed by exploring agreement in the relative effect reported by NRSIs and RCT addressing the same clinical question, defined as similarity of the population, intervention/exposure, control and outcomes. We will conduct univariable and multivariable logistic regression analyses to examine the association of prespecified study characteristics with agreement in the estimates between NRSIs and RCTs. We will calculate the ratio of the relative effect estimate from NRSIs over that from RCTs, along with the corresponding 95% CI. We will use a bias-corrected meta-analysis model to investigate the influence on pooled estimates when NRSIs are included in the evidence synthesis.

ETHICS AND DISSEMINATION

Ethics approval is not required. The findings of this study will be disseminated through peer-reviewed publications, conference presentations and condensed summaries for clinicians, health policymakers and guideline developers regarding the design, conduct, analysis, and interpretation of meta-analysis that integrate RCTs and NRSIs.

Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles: Primary Results of 2 Randomized Trials

Importance

Numerous studies have shown that adherence to reporting guidelines is suboptimal.

Objective

To evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles.

Design, Setting, and Participants

Two parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group.

Interventions

The first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded.

Main Outcomes and Measures

The difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles.

Results

In the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%).

Conclusions and Relevance

These 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future.

Trial Registration

ClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

Development of a risk-tailored approach and dashboard for efficient management and monitoring of investigator-initiated trials

BACKGROUND

Most randomized controlled trials (RCTs) in the academic setting have limited resources for clinical trial management and monitoring. Inefficient conduct of trials was identified as an important source of waste even in well-designed studies. Thoroughly identifying trial-specific risks to enable focussing of monitoring and management efforts on these critical areas during trial conduct may allow for the timely initiation of corrective action and to improve the efficiency of trial conduct. We developed a risk-tailored approach with an initial risk assessment of an individual trial that informs the compilation of monitoring and management procedures in a trial dashboard.

METHODS

We performed a literature review to identify risk indicators and trial monitoring approaches followed by a contextual analysis involving local, national and international stakeholders. Based on this work we developed a risk-tailored management approach with integrated monitoring for RCTs and including a visualizing trial dashboard. We piloted the approach and refined it in an iterative process based on feedback from stakeholders and performed formal user testing with investigators and staff of two clinical trials.

RESULTS

The developed risk assessment comprises four domains (patient safety and rights, overall trial management, intervention management, trial data). An accompanying manual provides rationales and detailed instructions for the risk assessment. We programmed two trial dashboards tailored to one medical and one surgical RCT to manage identified trial risks based on daily exports of accumulating trial data. We made the code for a generic dashboard available on GitHub that can be adapted to individual trials.

CONCLUSIONS

The presented trial management approach with integrated monitoring enables user-friendly, continuous checking of critical elements of trial conduct to support trial teams in the academic setting. Further work is needed in order to show effectiveness of the dashboard in terms of safe trial conduct and successful completion of clinical trials.

Antibody response after third vaccination with mRNA-1273 or BNT162b2: Extension of a randomised controlled SARS-CoV-2 non-inferiority vaccine trial in patients with different levels of immunosuppression (COVERALL-2)

Extension of the COVERALL randomised trial showed non-inferiority in antibody response of the third dose of Moderna mRNA-1273 vaccine (95.3%; 95% confidence interval [CI]: 91.9-98.7%) compared to Pfizer-BioNTech BNT162b2 vaccine (98.1%; 95% CI: 95.9-100.0%) in individuals with different levels of immunosuppression (difference: -2.8%; 95% CI: -6.8-1.3%).

GRADE Guidance 36: Updates to GRADE's approach to addressing inconsistency

OBJECTIVES

To update previous GRADE guidance addressing inconsistency and interpreting subgroup analyses.

STUDY DESIGN AND SETTING

Using an iterative process, we consulted with members of the GRADE working group through multiple rounds of written feedback and discussions at GRADE working group meetings.

RESULTS

The guidance complements previous guidance with clarification on two areas: (1) assessing inconsistency and (2) assessing the credibility of possible effect modifiers that might explain inconsistency. Specifically, the guidance clarifies that inconsistency refers to variability in results, not variability in study characteristics; that inconsistency assessment for binary outcomes requires consideration of both relative and absolute effects; how to decide between narrower and broader questions in systematic reviews and guidelines; that, with the same evidence, ratings of inconsistency may differ depending on the target of certainty rating; and how GRADE inconsistency ratings relate to a statistical measure of inconsistency I² depending on the context in which one views results. The second part of the guidance illustrates, based on a worked example, the use of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN). The guidance explains the stepwise process of moving from a subgroup analysis to assessing the credibility of effect modification and, if found credible, to subgroup-specific effect estimates and GRADE certainty ratings.

CONCLUSION

This updated guidance addresses specific conceptual and practical issues that systematic review authors frequently face when considering the degree of inconsistency in estimates of treatment effects across studies.

Effects of remdesivir in patients hospitalised with COVID-19: a systematic review and individual patient data meta-analysis of randomised controlled trials

BACKGROUND

Interpretation of the evidence from randomised controlled trials (RCTs) of remdesivir in patients treated in hospital for COVID-19 is conflicting. We aimed to assess the benefits and harms of remdesivir compared with placebo or usual care in these patients, and whether treatment effects differed between prespecified patient subgroups.

METHODS

For this systematic review and meta-analysis, we searched PubMed, Embase, the Cochrane COVID-19 trial registry, ClinicalTrials.gov, the International Clinical Trials Registry Platform, and preprint servers from Jan 1, 2020, until April 11, 2022, for RCTs of remdesivir in adult patients hospitalised with COVID-19, and contacted the authors of eligible trials to request individual patient data. The primary outcome was all-cause mortality at day 28 after randomisation. We used multivariable hierarchical regression-adjusting for respiratory support, age, and enrollment period-to investigate effect modifiers. This study was registered with PROSPERO, CRD42021257134.

FINDINGS

Our search identified 857 records, yielding nine RCTs eligible for inclusion. Of these nine eligible RCTs, individual data were provided for eight, covering 10 480 patients hospitalised with COVID-19 (99% of such patients included in such RCTs worldwide) recruited between Feb 6, 2020, and April 1, 2021. Within 28 days of randomisation, 662 (12·5%) of 5317 patients assigned to remdesivir and 706 (14·1%) of 5005 patients assigned to no remdesivir died (adjusted odds ratio [aOR] 0·88, 95% CI 0·78-1·00, p=0·045). We found evidence for a credible subgroup effect according to respiratory support at baseline (p_interaction=0·019). Of patients who were ventilated-including those who received high-flow oxygen-253 (30·0%) of 844 patients assigned to remdesivir died compared with 241 (28·5%) of 846 patients assigned to no remdesivir (aOR 1·10 [0·88-1·38]; low-certainty evidence). Of patients who received no oxygen or low-flow oxygen, 409 (9·1%) of 4473 patients assigned to remdesivir died compared with 465 (11·2%) of 4159 patients assigned to no remdesivir (0·80 [0·70-0·93]; high-certainty evidence). No credible subgroup effect was found for time to start of remdesivir after symptom onset, age, presence of comorbidities, enrolment period, or corticosteroid use. Remdesivir did not increase the frequency of severe or serious adverse events.

INTERPRETATION

This individual patient data meta-analysis showed that remdesivir reduced mortality in patients hospitalised with COVID-19 who required no or conventional oxygen support, but was underpowered to evaluate patients who were ventilated when receiving remdesivir. The effect size of remdesivir in patients with more respiratory support or acquired immunity and the cost-effectiveness of remdesivir remain to be further elucidated.

FUNDING

EU-RESPONSE.

Introducing the Library of Guidance for Health Scientists (LIGHTS): A Living Database for Methods Guidance

IMPORTANCE

Improving methodological quality is a priority in the health research community. Finding appropriate methods guidance can be challenging due to heterogeneous terminology, poor indexing in medical databases, and variation in formats. The Library of Guidance for Health Scientists (LIGHTS) is a new searchable database for methods guidance articles.

OBSERVATIONS

Journal articles that aim to provide guidance for performing (including planning, design, conduct, analysis, and interpretation), reporting, and assessing the quality of health-related research involving humans or human populations (ie, excluding basic and animal research) are eligible for LIGHTS. A team of health researchers, information specialists, and methodologists continuously identifies and manually indexes eligible guidance documents. The search strategy includes focused searches of specific journals, specialized databases, and suggestions from researchers. A current limitation is that a keyword-based search of MEDLINE (and other general databases) and manual screening of records were not feasible because of the large number of hits (n = 915 523). As of September 20, 2022, LIGHTS included 1246 articles (336 reporting guidelines, 80 quality assessment tools, and 830 other methods guidance articles). The LIGHTS website provides a user-oriented search interface including filters for study type, specific methodological topic, research context, guidance type, and development process of the guidance. Automated matching of alternative methodological expressions (eg, enter loss to follow-up and find articles indexed with missing data) enhances search queries.

CONCLUSIONS AND RELEVANCE

LIGHTS is a peer-supported initiative that is intended to increase access to and use of methods guidance relevant to health researchers, statisticians, methods consultants, methods developers, ethics boards, peer reviewers, journal editors, and funding bodies.

Towards full clinical trial registration and results publication: longitudinal meta-research study in Northwestern and Central Switzerland

OBJECTIVE

The registration of clinical trials is required by law in Switzerland. We investigated (1) the proportion of registered and prospectively registered clinical trials, (2) the availability of results for ethically approved trial protocols, (3) factors associated with increased registration, and (4) reasons for non-registration.

DESIGN AND SETTING

We included all clinical trials with mandatory prospective registration, which were approved by the ethics committee of Northwestern and Central Switzerland between January 1, 2016, and December 31, 2020.

METHODS

We extracted relevant trial characteristics from the Swiss Business Administration System for Ethics Committees and systematically searched the International Clinical Trials Registry Platform and primary trial registries for corresponding registry entries. We used multivariable logistic regression to examine the association between trial characteristics and registration. We qualitatively assessed reasons for non-registration of trials through an email questionnaire for trial investigators.

RESULTS

Of 473 included clinical trials, 432 (91%) were registered at all and 326 (69%) were prospectively registered. While the percentages of registration and prospective registration of investigator-sponsored trials increased from 85 to 93% and from 59 to 70% over 5 years, respectively, industry-sponsored trials consistently remained at a high level of prospective registration (92 to 100%). Trials with multiple centres, higher risk category, or methodological support from the local clinical trials unit were independently associated with increased registration rates. Of 103 clinical trials completed before August 2020, results were available for 70% of industry-sponsored trials and 45% of investigator-sponsored trials as peer-reviewed journal publications or in trial registries. Most common reasons for non-registration provided by investigators were lack of time or resources (53%), lack of knowledge (22%), and lack of reminders by the ethics committee (36%).

CONCLUSIONS

In Northwestern and Central Switzerland about 10% of clinical trials remained unregistered despite the obligation by law. More support for investigators and stricter enforcement by regulators are needed to improve the transparency of investigator-sponsored trials in particular.

Efficacy and safety of baricitinib in hospitalized adults with severe or critical COVID-19 (Bari-SolidAct): a randomised, double-blind, placebo-controlled phase 3 trial

BACKGROUND

Baricitinib has shown efficacy in hospitalized patients with COVID-19, but no placebo-controlled trials have focused specifically on severe/critical COVID, including vaccinated participants.

METHODS

Bari-SolidAct is a phase-3, multicentre, randomised, double-blind, placebo-controlled trial, enrolling participants from June 3, 2021 to March 7, 2022, stopped prematurely for external evidence. Patients with severe/critical COVID-19 were randomised to Baricitinib 4 mg once daily or placebo, added to standard of care. The primary endpoint was all-cause mortality within 60 days. Participants were remotely followed to day 90 for safety and patient related outcome measures.

RESULTS

Two hundred ninety-nine patients were screened, 284 randomised, and 275 received study drug or placebo and were included in the modified intent-to-treat analyses (139 receiving baricitinib and 136 placebo). Median age was 60 (IQR 49-69) years, 77% were male and 35% had received at least one dose of SARS-CoV2 vaccine. There were 21 deaths at day 60 in each group, 15.1% in the baricitinib group and 15.4% in the placebo group (adjusted absolute difference and 95% CI - 0.1% [- 8·3 to 8·0]). In sensitivity analysis censoring observations after drug discontinuation or rescue therapy (tocilizumab/increased steroid dose), proportions of death were 5.8% versus 8.8% (- 3.2% [- 9.0 to 2.7]), respectively. There were 148 serious adverse events in 46 participants (33.1%) receiving baricitinib and 155 in 51 participants (37.5%) receiving placebo. In subgroup analyses, there was a potential interaction between vaccination status and treatment allocation on 60-day mortality. In a subsequent post hoc analysis there was a significant interaction between vaccination status and treatment allocation on the occurrence of serious adverse events, with more respiratory complications and severe infections in vaccinated participants treated with baricitinib. Vaccinated participants were on average 11 years older, with more comorbidities.

CONCLUSION

This clinical trial was prematurely stopped for external evidence and therefore underpowered to conclude on a potential survival benefit of baricitinib in severe/critical COVID-19. We observed a possible safety signal in vaccinated participants, who were older with more comorbidities. Although based on a post-hoc analysis, these findings warrant further investigation in other trials and real-world studies. Trial registration Bari-SolidAct is registered at NCT04891133 (registered May 18, 2021) and EUClinicalTrials.eu ( 2022-500385-99-00 ).

Protocol for a mixed methods feasibility and implementation study of a community-based integrated care model for home-dwelling older adults: The INSPIRE project

BACKGROUND

Evaluations of integrated care models for home-dwelling frail older adults have shown inconclusive results on health and service outcomes. However, limited research has focused on the implementation of integrated care models. Applying implementation science methods may facilitate uptake of integrated care models, thus generating positive outcomes e.g., reduced hospital admissions. This paper describes the protocol to assess the feasibility of an integrated care model (featuring a four-step comprehensive geriatric assessment: screening, a multi-dimensional assessment, a coordinated individualized care plan and follow-up) designed for a new community-based center for home-dwelling older adults in Switzerland. The study includes the following objectives: 1) to assess implementation by a) monitoring respondents to the outreach strategies and describing the Center's visitors; b) assessing implementation outcomes related to the care model (i.e., adoption, acceptability, feasibility, fidelity) and implementation processes related to collaboration; and 2) assessing implementation costs.

METHODS

For objective 1a, we will use a descriptive design to assess respondents to the outreach strategies and describe the Center's visitors. We will use a parallel convergent mixed methods design for objective 1b. Implementation outcomes data will be collected from meetings with the Center's staff, interviews with older adults and their informal caregivers, and reviewing older adults' health records at the Center. Implementation processes related to collaboration will be assessed through a questionnaire to external collaborators (e.g., GPs) towards the end of the study. For objective 2, implementation costs will be calculated using time-driven activity-based costing methods. Data collection is anticipated to occur over approximately six months.

DISCUSSION

This study of a contextually adapted integrated care model will inform adaptations to the outreach strategies, care model and implementation strategies in one community center, prior to evaluating the care model effectiveness and potentially scaling out the intervention.

TRIAL REGISTRATION

Feasibility study registration ID with clinicaltrials.gov: NCT05302310; registration ID with BMC: ISRCTN12324618.

Factors associated with health-related quality of life among home-dwelling older adults aged 75 or older in Switzerland: a cross-sectional study

BACKGROUND

HRQoL is an indicator of individuals' perception of their overall health, including social and environmental aspects. As a multidimensional concept, HRQoL can be influenced by a multitude of factors. Studies of HRQoL and factors associated with it among home-dwelling older adults have often been limited to inpatient settings or to a sub-population with a chronic disease. Studying HRQoL and its correlating factors among this population, by providing an ecological lens on factors beyond the individual level, can provide a better understanding of the construct and the role of the environment on how they perceive their HRQoL. Thus, we aimed to assess the HRQoL and investigate the correlates of HRQOL among home-dwelling older adults, guided by the levels of the ecological model.

METHODS

This is a cross-sectional population survey conducted in 2019 in Canton Basel-Landschaft, in northwestern Switzerland, and includes a sample of 8786 home-dwelling older adults aged 75 and above. We assessed HRQoL by using the EQ-index and the EQ-VAS. The influence of independent variables at the macro, meso and micro level on HRQoL was tested using Tobit multiple linear regression modelling.

RESULTS

We found that having a better socio-economic status as denoted by higher income, having supplementary insurance and a higher level of education were all associated with a better HRQoL among home-dwelling older adults. Furthermore, being engaged in social activities was also related to an improved HRQoL. On the other hand, older age, female gender, presence of multimorbidity and polypharmacy as well as social isolation and loneliness were found to all have a negative impact on HRQoL.

CONCLUSIONS

Understanding factors related to HRQoL by using an ecological lens can help identify factors beyond the individual level that impact the HRQoL of home-dwelling older adults. Our study emphasises the importance of social determinants of health and potential disparities that exists, encouraging policymakers to focus on policies to reduce socio-economic disparities using a life-course approach, which consequently could also impact HRQoL in later stages of life.

Predicting neurological outcome in adult patients with cardiac arrest: systematic review and meta-analysis of prediction model performance

This work aims to assess the performance of two post-arrest (out-of-hospital cardiac arrest, OHCA, and cardiac arrest hospital prognosis, CAHP) and one pre-arrest (good outcome following attempted resuscitation, GO-FAR) prediction model for the prognostication of neurological outcome after cardiac arrest in a systematic review and meta-analysis. A systematic search was conducted in Embase, Medline, and Web of Science Core Collection from November 2006 to December 2021, and by forward citation tracking of key score publications. The search identified 1'021 records, of which 25 studies with a total of 124'168 patients were included in the review. A random-effects meta-analysis of C-statistics and overall calibration (total observed vs. expected [O:E] ratio) was conducted. Discriminatory performance was good for the OHCA (summary C-statistic: 0.83 [95% CI 0.81-0.85], 16 cohorts) and CAHP score (summary C-statistic: 0.84 [95% CI 0.82-0.87], 14 cohorts) and acceptable for the GO-FAR score (summary C-statistic: 0.78 [95% CI 0.72-0.84], five cohorts). Overall calibration was good for the OHCA (total O:E ratio: 0.78 [95% CI 0.67-0.92], nine cohorts) and the CAHP score (total O:E ratio: 0.78 [95% CI 0.72-0.84], nine cohorts) with an overestimation of poor outcome. Overall calibration of the GO-FAR score was poor with an underestimation of good outcome (total O:E ratio: 1.62 [95% CI 1.28-2.04], five cohorts). Two post-arrest scores showed good prognostic accuracy for predicting neurological outcome after cardiac arrest and may support early discussions about goals-of-care and therapeutic planning on the intensive care unit. A pre-arrest score showed acceptable prognostic accuracy and may support code status discussions.

Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues

OBJECTIVES

To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias.

STUDY DESIGN AND SETTING

We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above.

RESULTS

Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items.

CONCLUSION

Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.

Health and social care of home-dwelling frail older adults in Switzerland: a mixed methods study

Background

Home-dwelling frail older adults are often faced with multimorbidity and complex care needs, requiring health and social care systems that support frail older adults to age in place. The objective of this paper was to investigate the types of formal health and social care as well as informal care and social support used by home-dwelling frail older adults; whether they perceive their support as sufficient; and their experience with and preferences for care and support.

Methods

Using an explanatory sequential mixed methods design, we first conducted a secondary analysis of a subset of cross-sectional data from the ImplemeNtation of a community-baSed care Program for home dwelling senIoR citizEns (INSPIRE) population survey using descriptive analysis. Subsequently, we analyzed existing data from interviews in the parent study to help explain the survey results using applied thematic analysis. Results were organized according to adapted domains and concepts of the SELFIE framework and integrated via a joint display table.

Results

Of the parent population survey respondents, 2314 older adults indicating frailty were included in the quantitative arm of this study. Interview data was included from 7 older adults who indicated frailty. Support from health and social, formal and informal caregivers is diverse and anticipated to increase (e.g., for ‘care and assistance at home’ and ‘meal services’). Informal caregivers fulfilled various roles and while some older adults strongly relied on them for support, others feared burdening them. Most participants (93.5%) perceived their overall support to meet their needs; however, findings suggest areas (e.g., assessment of overall needs) which merit attention to optimize future care.

Conclusions

Given the anticipated demand for future care and support, we recommend efforts to prevent fragmentation between health and social as well as formal and informal care.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03552-z.

Determinants of antibody response to severe acute respiratory syndrome coronavirus 2 mRNA vaccines in people with HIV

We identified determinants of SARS-CoV-2 mRNA vaccine antibody response in people with HIV (PWH). Antibody response was higher among PWH less than 60 years, with CD4+ cell count superior to 350 cells/μl and vaccinated with mRNA-1273 by Moderna compared with BNT162b2 by Pfizer-BioNTech. Preinfection with SARS-CoV-2 boosted the antibody response and smokers had an overall lower antibody response. Elderly PWH and those with low CD4+ cell count should be prioritized for booster vaccinations.

Developing and testing a Corona VaccinE tRiAL pLatform (COVERALL) to study Covid-19 vaccine response in immunocompromised patients

BACKGROUND

The rapid course of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic calls for fast implementation of clinical trials to assess the effects of new treatment and prophylactic interventions. Building trial platforms embedded in existing data infrastructures is an ideal way to address such questions within well-defined subpopulations.

METHODS

We developed a trial platform building on the infrastructure of two established national cohort studies: the Swiss human immunodeficiency virus (HIV) Cohort Study (SHCS) and Swiss Transplant Cohort Study (STCS). In a pilot trial, termed Corona VaccinE tRiAL pLatform (COVERALL), we assessed the vaccine efficacy of the first two licensed SARS-CoV-2 vaccines in Switzerland and the functionality of the trial platform.

RESULTS

Using Research Electronic Data Capture (REDCap), we developed a trial platform integrating the infrastructure of the SHCS and STCS. An algorithm identifying eligible patients, as well as baseline data transfer ensured a fast inclusion procedure for eligible patients. We implemented convenient re-directions between the different data entry systems to ensure intuitive data entry for the participating study personnel. The trial platform, including a randomization algorithm ensuring balance among different subgroups, was continuously adapted to changing guidelines concerning vaccination policies. We were able to randomize and vaccinate the first trial participant the same day we received ethics approval. Time to enroll and randomize our target sample size of 380 patients was 22 days.

CONCLUSION

Taking the best of each system, we were able to flag eligible patients, transfer patient information automatically, randomize and enroll the patients in an easy workflow, decreasing the administrative burden usually associated with a trial of this size.

A longitudinal assessment of trial protocols approved by research ethics committees: The Adherance to SPIrit REcommendations in the UK (ASPIRE-UK) study

Background

To assess the quality of reporting of RCT protocols approved by UK research ethics committees before and after the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline.

Methods

We had access to RCT study protocols that received ethical approval in the UK in 2012 (n=103) and 2016 (n=108). From those, we assessed the adherence to the 33 SPIRIT items (i.e. a total of 64 components of the 33 SPIRIT items). We descriptively analysed the adherence to SPIRIT guidelines as proportion of adequately reported items (median and interquartile range [IQR]) and stratified the results by year of approval and sponsor.

Results

The proportion of reported SPIRIT items increased from a median of 64.9% (IQR, 57.6–69.2%) in 2012 to a median of 72.5% (IQR, 65.3–78.3%) in 2016. Industry-sponsored RCTs reported more SPIRIT items in 2012 (median 67.4%; IQR, 64.1–69.4%) compared to non-industry-sponsored trials (median 59.8%; IQR, 46.5–67.7%). This gap between industry- and non-industry-sponsored trials increased in 2016 (industry-sponsored: median 75.6%; IQR, 71.2–79.0% vs non-industry-sponsored: median 65.3%; IQR, 51.6–76.3%).

Conclusions

The adherence to SPIRIT guidelines has improved in the UK from 2012 to 2016 but remains on a modest level, especially for non-industry-sponsored RCTs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06516-1.

Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE)

OBJECTIVES

Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist.

DESIGN

Repeated cross sectional study.

SETTING

Swiss, German and Canadian research ethics committees (RECs).

PARTICIPANTS

RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292).

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reported RESULTS: The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%-79%) in 2012 to 77% (IQR, 68%-82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%-72%) in 2012 to 76% (IQR, 64%-83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%-80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship.

CONCLUSIONS

In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve.

Scoping review and characteristics of publicly available checklists for assessing clinical trial feasibility

Background

Whether there is sufficient capacity and capability for the successful conduct and delivery of a clinical trial should be assessed by several stakeholders according to transparent and evidence-based criteria during trial planning. For this openly shared, user-tested, and validated tools are necessary. Therefore, we systematically examined the public availability and content of checklists which assess the study-level feasibility in the planning phase of clinical trials.

Methods

In our scoping review we systematically searched Medline, EMBASE, and Google (last search, June 2021). We included all publicly available checklists or tools that assessed study level feasibility of clinical trials, examined their content, and checked whether they were user-tested or validated in any form. Data was analysed and synthesised using conventional content analysis.

Results

A total of 10 publicly available checklists from five countries were identified. The checklists included 48 distinct items that were classified according to the following seven different domains of clinical trial feasibility: regulation, review and oversight; participant recruitment; space, material and equipment; financial resources; trial team resources; trial management; and pilot or feasibility studies. None of the available checklists appeared to be user-tested or validated.

Conclusions

Although a number of publicly available checklists to assess the feasibility of clinical trials exist, their reliability and usefulness remain unclear. Openly shared, user-tested, and validated feasibility assessment tools for a better planning of clinical trials are lacking.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01617-6.

Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis

BACKGROUND

We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.

METHODS AND FINDINGS

We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.

CONCLUSIONS

We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Investigational medicinal products, related costs and hospital pharmacy services for investigator-initiated trials: A mixed-methods study

BACKGROUND

Conducting high quality investigator-initiated trials (IITs) is challenging and costly. The costs of investigational medicinal products (IMPs) in IITs and the role of hospital pharmacies in the planning of IITs are unclear. We conducted a mixed-methods study to compare planned and actual costs of IMPs in Swiss IITs, to examine potential reasons for differences, and to gather stakeholder views about hospital services for IITs.

METHODS

We included all IITs with IMP services from the Basel hospital pharmacy invoiced between January 2014 and June 2020 (n = 24). We documented trial and IMP characteristics including planned and actual IMP costs. Our working definition for a substantial cost difference was that the actual IMP costs were more than 10% higher than the planned IMP costs in a trial. We conducted semi-structured interviews with investigators, clinical trials unit and hospital pharmacy staff, and qualitatively analyzed transcribed interviews.

RESULTS

For 13 IITs we observed no differences between planned and actual costs of IMPs (median, 11'000 US$; interquartile range [IQR], 8'882-16'302 US$), but for 11 IITs we found cost increases from a median of 11'000 US$ (IQR, 8'922-36'166 US$) to a median over 28'000 US$ (IQR, 13'004-49'777 US$). All multicenter trials and 10 of 11 IITs with patients experienced substantial cost differences. From the interviews we identified four main themes: 1) Patient recruitment and organizational problems were identified as main reasons for cost differences, 2) higher actual IMP costs were bearable for most investigators, 3) IMP services for IITs were not a priority for the hospital pharmacy, and 4) closer collaboration between clinical trial unit and hospital pharmacy staff, and sufficient staff for IITs at the hospital pharmacy could improve IMP services.

CONCLUSIONS

Multicenter IITs enrolling patients are particularly at risk for higher IMP costs than planned. These trials are more difficult to plan and logistically challenging, which leads to delays and expiring IMP shelf-lives. IMP services of hospital pharmacies are important for IITs in Switzerland, but need to be further developed.

Antibody response in immunocompromised patients after the administration of SARS-CoV-2 vaccine BNT162b2 or mRNA-1273: A randomised controlled trial

BACKGROUND

BNT162b2 by Pfizer-BioNTech and mRNA-1273 by Moderna are the most commonly used vaccines to prevent SARS-CoV-2 infections. Head-to-head comparison of the efficacy of these vaccines in immunocompromised patients is lacking.

METHODS

Parallel, two-arm (allocation 1:1), open-label, non-inferiority randomised clinical trial nested into the Swiss HIV Cohort Study and the Swiss Transplant Cohort Study. Patients living with HIV (PLWH) or solid organ transplant recipients (SOTR; i.e. lung and kidney) from these cohorts were randomised to mRNA-1273 or BNT162b2. The primary endpoint was antibody response to SARS-CoV-2 spike (S1) protein receptor binding domain (Elecsys Anti-SARS-CoV-2 immunoassay, Roche; cut-off ≥0.8 units/ml) 8 weeks after second vaccination. In addition, antibody response was measured with the Antibody CORonavirus Assay 2 (ABCORA 2).

RESULTS

430 patients were randomised and 412 were included in the intention-to-treat analysis (341 PLWH and 71 SOTR). The percentage of patients showing an immune response was 92.1% (95% confidence interval [CI] 88.4-95.8%; 186/202) for mRNA-1273 and 94.3% (95% CI 91.2-97.4; 198/210) for BNT162b2 (difference: 2.2%; 95% CI -7.1 to 2.7), fulfilling non-inferiority of mRNA-1273. With the ABCORA 2 test 89.1% had an immune response to mRNA-1273 (95% CI 84.8-93.4%; 180/202) and 89.5% to BNT162b2 (95% CI 85.4-93.7%; 188/210). Based on the Elecsys test, all PLWH had an antibody response (100.0%; 341/341), while for SOTR only 60.6% (95% CI 49.2-71.9%; 43/71) had titres above the cut-off.

CONCLUSIONS

In immunocompromised patients the antibody response of mRNA-1273 was non-inferior to BNT162b2. PLWH had in general an antibody response, while a high proportion of SOTR had no antibody response.

Monitoring strategies for clinical intervention studies

BACKGROUND

Trial monitoring is an important component of good clinical practice to ensure the safety and rights of study participants, confidentiality of personal information, and quality of data. However, the effectiveness of various existing monitoring approaches is unclear. Information to guide the choice of monitoring methods in clinical intervention studies may help trialists, support units, and monitors to effectively adjust their approaches to current knowledge and evidence.

OBJECTIVES

To evaluate the advantages and disadvantages of different monitoring strategies (including risk-based strategies and others) for clinical intervention studies examined in prospective comparative studies of monitoring interventions.

SEARCH METHODS

We systematically searched CENTRAL, PubMed, and Embase via Ovid for relevant published literature up to March 2021. We searched the online 'Studies within A Trial' (SWAT) repository, grey literature, and trial registries for ongoing or unpublished studies.

SELECTION CRITERIA

We included randomized or non-randomized prospective, empirical evaluation studies of different monitoring strategies in one or more clinical intervention studies. We applied no restrictions for language or date of publication.

DATA COLLECTION AND ANALYSIS

We extracted data on the evaluated monitoring methods, countries involved, study population, study setting, randomization method, and numbers and proportions in each intervention group. Our primary outcome was critical and major monitoring findings in prospective intervention studies. Monitoring findings were classified according to different error domains (e.g. major eligibility violations) and the primary outcome measure was a composite of these domains. Secondary outcomes were individual error domains, participant recruitment and follow-up, and resource use. If we identified more than one study for a comparison and outcome definitions were similar across identified studies, we quantitatively summarized effects in a meta-analysis using a random-effects model. Otherwise, we qualitatively summarized the results of eligible studies stratified by different comparisons of monitoring strategies. We used the GRADE approach to assess the certainty of the evidence for different groups of comparisons.

MAIN RESULTS

We identified eight eligible studies, which we grouped into five comparisons. 1. Risk-based versus extensive on-site monitoring: based on two large studies, we found moderate certainty of evidence for the combined primary outcome of major or critical findings that risk-based monitoring is not inferior to extensive on-site monitoring. Although the risk ratio was close to 'no difference' (1.03 with a 95% confidence interval [CI] of 0.81 to 1.33, below 1.0 in favor of the risk-based strategy), the high imprecision in one study and the small number of eligible studies resulted in a wide CI of the summary estimate. Low certainty of evidence suggested that monitoring strategies with extensive on-site monitoring were associated with considerably higher resource use and costs (up to a factor of 3.4). Data on recruitment or retention of trial participants were not available. 2. Central monitoring with triggered on-site visits versus regular on-site visits: combining the results of two eligible studies yielded low certainty of evidence with a risk ratio of 1.83 (95% CI 0.51 to 6.55) in favor of triggered monitoring intervention. Data on recruitment, retention, and resource use were not available. 3. Central statistical monitoring and local monitoring performed by site staff with annual on-site visits versus central statistical monitoring and local monitoring only: based on one study, there was moderate certainty of evidence that a small number of major and critical findings were missed with the central monitoring approach without on-site visits: 3.8% of participants in the group without on-site visits and 6.4% in the group with on-site visits had a major or critical monitoring finding (odds ratio 1.7, 95% CI 1.1 to 2.7; P = 0.03). The absolute number of monitoring findings was very low, probably because defined major and critical findings were very study specific and central monitoring was present in both intervention groups. Very low certainty of evidence did not suggest a relevant effect on participant retention, and very low certainty evidence indicated an extra cost for on-site visits of USD 2,035,392. There were no data on recruitment. 4. Traditional 100% source data verification (SDV) versus targeted or remote SDV: the two studies assessing targeted and remote SDV reported findings only related to source documents. Compared to the final database obtained using the full SDV monitoring process, only a small proportion of remaining errors on overall data were identified using the targeted SDV process in the MONITORING study (absolute difference 1.47%, 95% CI 1.41% to 1.53%). Targeted SDV was effective in the verification of source documents, but increased the workload on data management. The other included study was a pilot study, which compared traditional on-site SDV versus remote SDV and found little difference in monitoring findings and the ability to locate data values despite marked differences in remote access in two clinical trial networks. There were no data on recruitment or retention. 5. Systematic on-site initiation visit versus on-site initiation visit upon request: very low certainty of evidence suggested no difference in retention and recruitment between the two approaches. There were no data on critical and major findings or on resource use.

AUTHORS' CONCLUSIONS

The evidence base is limited in terms of quantity and quality. Ideally, for each of the five identified comparisons, more prospective, comparative monitoring studies nested in clinical trials and measuring effects on all outcomes specified in this review are necessary to draw more reliable conclusions. However, the results suggesting risk-based, targeted, and mainly central monitoring as an efficient strategy are promising. The development of reliable triggers for on-site visits is ongoing; different triggers might be used in different settings. More evidence on risk indicators that identify sites with problems or the prognostic value of triggers is needed to further optimize central monitoring strategies. In particular, approaches with an initial assessment of trial-specific risks that need to be closely monitored centrally during trial conduct with triggered on-site visits should be evaluated in future research.

Exploring reasons for recruitment failure in clinical trials: a qualitative study with clinical trial stakeholders in Switzerland, Germany, and Canada

BACKGROUND

Poor participant recruitment is the most frequent reason for premature discontinuation of randomized clinical trials (RCTs), particularly if they are investigator-initiated. The aims of this qualitative study were to investigate (1) the views of clinical trial stakeholders from three different countries regarding reasons for recruitment failure in RCTs and (2) how these compare and contrast with the causes identified in a previous systematic review of RCT publications.

METHODS

From August 2015 to November 2016, we conducted 49 semi-structured interviews with a purposive sample of clinical trial stakeholders. This included investigators based in Germany (n = 9), Switzerland (n = 6) and Canada (n = 1) with personal experience of a discontinued RCT and 33 other stakeholders (e.g., representatives of ethics committees, clinical trial units, pharmaceutical industry) in Switzerland. Individual semi-structured qualitative interviews were conducted and analyzed using thematic analysis.

RESULTS

Interviewees identified a total of 29 different reasons for recruitment failure. Overoptimistic recruitment estimates, too narrow eligibility criteria, lack of engagement of recruiters/trial team, lack of competence/training/experience of recruiters, insufficient initial funding, and high burden for trial participants were mentioned most frequently. The interview findings largely confirm the previous systematic review on published reasons for recruitment failure. However, eight new reasons for recruitment failure were identified in the interviews, which led to the checklist of reasons for recruitment failure being revised and a new category describing research environment-related factors being added.

CONCLUSIONS

This study highlights the diversity of often interlinked reasons for recruitment failure in RCTs. Integrating the findings of this interview study with a previous systematic review of RCT publications led to a comprehensive, structured checklist of empirically-informed reasons for recruitment failure. The checklist may be useful to guide further research on interventions to improve participant recruitment in RCTs and helpful for trial investigators, research ethics committees, and funding agencies when assessing trial feasibility with respect to recruitment.

Treatment for gastrointestinal and pancreatic neuroendocrine tumours: a network meta-analysis

BACKGROUND

Several available therapies for neuroendocrine tumours (NETs) have demonstrated efficacy in randomised controlled trials. However, translation of these results into improved care faces several challenges, as a direct comparison of the most pertinent therapies is incomplete.

OBJECTIVES

To evaluate the safety and efficacy of therapies for NETs, to guide clinical decision-making, and to provide estimates of relative efficiency of the different treatment options (including placebo) and rank the treatments according to their efficiency based on a network meta-analysis.

SEARCH METHODS

We identified studies through systematic searches of the following bibliographic databases: the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library; MEDLINE (Ovid); and Embase from January 1947 to December 2020. In addition, we checked trial registries for ongoing or unpublished eligible trials and manually searched for abstracts from scientific and clinical meetings.

SELECTION CRITERIA

We evaluated randomised controlled trials (RCTs) comparing two or more therapies in people with NETs (primarily gastrointestinal and pancreatic).

DATA COLLECTION AND ANALYSIS

Two review authors independently selected studies and extracted data to a pre-designed data extraction form. Multi-arm studies were included in the network meta-analysis using the R-package netmeta. We separately analysed two different outcomes (disease control and progression-free survival) and two types of NET (gastrointestinal and pancreatic NET) in four network meta-analyses. A frequentist approach was used to compare the efficacy of therapies.

MAIN RESULTS

We identified 55 studies in 90 records in the qualitative analysis, reporting 39 primary RCTs and 16 subgroup analyses. We included 22 RCTs, with 4299 participants, that reported disease control and/or progression-free survival in the network meta-analysis. Precision-of-treatment estimates and estimated heterogeneity were limited, although the risk of bias was predominantly low. The network meta-analysis of progression-free survival found nine therapies for pancreatic NETs: everolimus (hazard ratio [HR], 0.36 [95% CI, 0.28 to 0.46]), interferon plus somatostatin analogue (HR, 0.34 [95% CI, 0.14 to 0.80]), everolimus plus somatostatin analogue (HR, 0.38 [95% CI, 0.26 to 0.57]), bevacizumab plus somatostatin analogue (HR, 0.36 [95% CI, 0.15 to 0.89]), interferon (HR, 0.41 [95% CI, 0.18 to 0.94]), sunitinib (HR, 0.42 [95% CI, 0.26 to 0.67]), everolimus plus bevacizumab plus somatostatin analogue (HR, 0.48 [95% CI, 0.28 to 0.83]), surufatinib (HR, 0.49 [95% CI, 0.32 to 0.76]), and somatostatin analogue (HR, 0.51 [95% CI, 0.34 to 0.77]); and six therapies for gastrointestinal NETs: 177-Lu-DOTATATE plus somatostatin analogue (HR, 0.07 [95% CI, 0.02 to 0.26]), everolimus plus somatostatin analogue (HR, 0.12 [95%CI, 0.03 to 0.54]), bevacizumab plus somatostatin analogue (HR, 0.18 [95% CI, 0.04 to 0.94]), interferon plus somatostatin analogue (HR, 0.23 [95% CI, 0.06 to 0.93]), surufatinib (HR, 0.33 [95%CI, 0.12 to 0.88]), and somatostatin analogue (HR, 0.34 [95% CI, 0.16 to 0.76]), with higher efficacy than placebo. Besides everolimus for pancreatic NETs, the results suggested an overall superiority of combination therapies, including somatostatin analogues. The results indicate that NET therapies have a broad range of risk for adverse events and effects on quality of life, but these were reported inconsistently. Evidence from this network meta-analysis (and underlying RCTs) does not support any particular therapy (or combinations of therapies) with respect to patient-centred outcomes (e.g. overall survival and quality of life).

AUTHORS' CONCLUSIONS

The findings from this study suggest that a range of efficient therapies with different safety profiles is available for people with NETs.

Reliability of Trial Information Across Registries for Trials With Multiple Registrations: A Systematic Review

IMPORTANCE

Clinical trial registries are important for gaining an overview of ongoing research efforts and for deterring and identifying publication bias and selective outcome reporting. The reliability of the information in trial registries is uncertain.

OBJECTIVE

To assess the reliability of information across registries for trials with multiple registrations.

EVIDENCE REVIEW

For this systematic review, 360 protocols of randomized clinical trials (RCTs) approved by research ethics committees in Switzerland, the UK, Canada, and Germany in 2012 were evaluated. Clinical trial registries were searched from March to September 2019 for corresponding registrations of these RCTs. For RCTS that were recorded in more than 1 clinical trial registry, key trial characteristics that should be identical among all trial registries (ie, sponsor, funding source, primary outcome, target sample size, trial status, date of first patient enrollment, results available, and main publication indexed) were extracted in duplicate. Agreement between the different trial registries for these key characteristics was analyzed descriptively. Data analyses were conducted from May 1 to November 30, 2020. Representatives from clinical trial registries were interviewed to discuss the study findings between February 1 and March 31, 2021.

FINDINGS

The analysis included 197 RCTs registered in more than 1 trial registry (151 in 2 registries and 46 in 3 registries), with 188 trials in ClinicalTrials.gov, 185 in the European Union Drug Regulating Authorities Clinical Trials Database (EudraCT), 20 in ISRCTN, and 47 in other registries. The agreement of key information across all registries was as follows: 178 of 197 RCTs (90%; 95% CI, 85%-94%) for sponsor, 18 of 20 (90%; 95% CI, 68%-99%) for funding source (funding was not reported on ClinicalTrials.gov), 154 of 197 (78%; 95% CI, 72%-84%) for primary outcome, 90 of 197 (46%; 95% CI, 39%-53%) for trial status, 122 of 194 (63%; 95% CI, 56%-70%) for target sample size, and 43 of 57 (75%; 95% CI, 62%-86%) for the date of first patient enrollment when the comparison time was increased to 30 days (date of first patient enrollment was not reported on EudraCT). For results availability in trial registries, agreement was 122 of 197 RCTs (62%; 95% CI, 55%-69%) for summary data reported in the registry and 91 of 197 (46%; 95% CI, 39%-53%) for whether a published article with the main results was indexed. Different legal requirements were stated as the main reason for inconsistencies by representatives of clinical trial registries.

CONCLUSIONS AND RELEVANCE

In this systematic review, for a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of the information provided in these registries. Further harmonization across clinical trial registries may be necessary to increase their usefulness.

Barriers and Facilitating Factors for Conducting Systematic Evidence Assessments in Academic Clinical Trials

IMPORTANCE

A systematic assessment of existing research should justify the conduct and inform the design of new clinical research but is often lacking. There is little research on the barriers to and factors facilitating systematic evidence assessments.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials.

DESIGN, SETTING, AND PARTICIPANTS

In this qualitative study, individual semistructured qualitative interviews were conducted between February and August 2020 with 48 Swiss stakeholder groups (27 primary investigators, 9 funders and sponsors, 6 clinical trial support organizations, and 6 ethics committee members) and between January and March 2021 with 9 international funders of clinical trials from North America and Europe with a reputation for requiring systematic evidence synthesis in applications for academic clinical trials.

MAIN OUTCOMES AND MEASURES

The main outcomes were practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. Interviews were analyzed using conventional content analysis.

RESULTS

Of the 57 participants, 40 (70.2%) were male. Participants universally acknowledged that a comprehensive understanding of the previous evidence is important but reported wide variation regarding how this should be achieved. Participants reported that the conduct of formal systematic reviews was currently not expected before most clinical trials, but most international funders reported expecting a systematic search for the existing evidence. Whereas time and resources were reported by all participants as barriers to conducting systematic reviews, the Swiss research ecosystem was reported not to be as supportive of a systematic approach compared with international settings.

CONCLUSIONS AND RELEVANCE

In this qualitative study, Swiss stakeholders and international funders generally agreed that new clinical trials should be justified by a systematic evidence assessment but that barriers on individual, organizational, and political levels kept them from implementing it. More explicit requirements from funders appear to be needed to clarify the required level of comprehensiveness in summarizing existing evidence for different types of clinical trials.

Evaluation of Planned Subgroup Analysis in Protocols of Randomized Clinical Trials

This cross-sectional study compares randomized clinical trial protocols to assess the prevalence and reporting quality of planned subgroup analyses over time.

Duration of antibiotic treatment using procalcitonin-guided treatment algorithms in older patients: a patient-level meta-analysis from randomized controlled trials

Background

Older patients have a less pronounced immune response to infection, which may also influence infection biomarkers. There is currently insufficient data regarding clinical effects of procalcitonin (PCT) to guide antibiotic treatment in older patients.

Objective and design

We performed an individual patient data meta-analysis to investigate the association of age on effects of PCT-guided antibiotic stewardship regarding antibiotic use and outcome.

Subjects and methods

We had access to 9,421 individual infection patients from 28 randomized controlled trials comparing PCT-guided antibiotic therapy (intervention group) or standard care. We stratified patients according to age in four groups (<75 years [n = 7,079], 75–80 years [n = 1,034], 81–85 years [n = 803] and >85 years [n = 505]). The primary endpoint was the duration of antibiotic treatment and the secondary endpoints were 30-day mortality and length of stay.

Results

Compared to control patients, mean duration of antibiotic therapy in PCT-guided patients was significantly reduced by 24, 22, 26 and 24% in the four age groups corresponding to adjusted differences in antibiotic days of −1.99 (95% confidence interval [CI] −2.36 to −1.62), −1.98 (95% CI −2.94 to −1.02), −2.20 (95% CI −3.15 to −1.25) and − 2.10 (95% CI −3.29 to −0.91) with no differences among age groups. There was no increase in the risk for mortality in any of the age groups. Effects were similar in subgroups by infection type, blood culture result and clinical setting (P interaction >0.05).

Conclusions

This large individual patient data meta-analysis confirms that, similar to younger patients, PCT-guided antibiotic treatment in older patients is associated with significantly reduced antibiotic exposures and no increase in mortality.

Clinical trial research on COVID-19 in Germany - a systematic analysis

Background: In 2020, the COVID-19 pandemic led to an unprecedented volume of almost 3,000 clinical trials registered worldwide. We aimed to describe the COVID-19 clinical trial research agenda in Germany during the first year of the pandemic. Methods: We identified randomized clinical trials assessing interventions to treat or prevent COVID-19 that were registered in 2020 and recruited or planned to recruit participants in Germany. We requested recruitment information from trial investigators as of April 2021. Results: In 2020, 65 trials were completely (n=27) or partially (n=38) conducted in Germany. Most trials investigated interventions to treat COVID-19 (86.2%; 56/65), in hospitalized patients (67.7%; 44/65), with industry funding (53.8%; 35/65). Few trials were completed (21.5%; 14/65). Overall, 187,179 participants were planned to be recruited (20,696 in Germany), with a median number of 106 German participants per trial (IQR 40 to 345). From the planned German participants, 13.4% were recruited (median 15 per trial (IQR 0 to 44). Conclusions: The overall German contribution to the worldwide COVID-19 clinical trial research agenda was modest. Few trials delivered urgently needed evidence. Most trials did not meet recruitment goals. Evaluation and international comparison of the challenges for conducting clinical trials in Germany is needed.

The INSPIRE Population Survey: development, dissemination and respondent characteristics

BACKGROUND

Most older adults prefer to continue living at home despite increasing care needs and demand for services. To aid in maintaining independence, integrated care models for community-dwelling older people are promoted as the most cost-effective approach. The implementation of such care models is challenging and often the end-users are not involved or their needs are not considered. We conducted a population survey in order to understand the needs and preferences of home-dwelling older adults living in Canton Basel-Landschaft, Switzerland. The aims of this paper are to chronicle the development of the INSPIRE Population Survey, outline its variables and measurements, describe the marketing strategy utilized for survey dissemination and report on the response rate and respondent characteristics.

METHODS

The INSPIRE Population Survey, conducted between March and August 2019, is a cross-sectional survey of older adults aged 75 and older living at home in Canton Basel-Landschaft. The questionnaire was developed by expert input and stakeholder involvement. Its readability and acceptability were pilot-tested with older people. To ensure the likelihood of a high and representative response rate, a meticulous step-by-step marketing strategy was developed prior to the dissemination of the questionnaire.

RESULTS

The overall response rate was 30.7% (n = 8,846), with variations between 20.6 and 34.5% across the different care regions in the canton. A generally higher response rate was found in the care regions with a higher density and which bordered the urban city of Basel. We received support from local stakeholders, policy makers and media through using a broad combination of marketing channels and targeting our community partners who have a strong relationship with our target audience.

CONCLUSIONS

Although recruiting older adults in research is challenging, our study shows that a high response rate can be achieved by developing the survey through expert input and by involving all important stakeholders, including older adults, throughout the entire process.

Practices and Attitudes of Swiss Stakeholders Regarding Investigator-Initiated Clinical Trial Funding Acquisition and Cost Management

IMPORTANCE

Randomized clinical trials (RCTs) are an essential method of evaluating health care interventions and a cornerstone for evidence-based health care. However, RCTs have become increasingly complex and costly, which is particularly challenging for independent investigator-initiated clinical trials (IICTs). IICTs have an essential role in clinical research, and it is important that efforts are made to ensure IICTs are adequately funded and are conducted cost-effectively.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders regarding IICT funding acquisition and cost management.

DESIGN, SETTING, AND PARTICIPANTS

For this qualitative study, interviews were conducted in Switzerland between February and August 2020. The purposive sample comprised 48 stakeholders from 4 different groups: primary investigators (n = 27), funders and sponsors (n = 9), clinical trial support organizations (n = 6), and ethics committee members (n = 6).

MAIN OUTCOMES AND MEASURES

Practices and attitudes of stakeholders regarding IICT funding acquisition and cost management were assessed using individual semistructured qualitative interviews. Interviews were analyzed using conventional content analysis.

RESULTS

After interviews with 48 IICT stakeholders (75% male presenting), these participants identified a systemic problem of IICTs being underfunded, which can lead to compromises being made regarding the quality and conduct of IICTs. Participants identified 2 overarching and interconnected groups of reasons why IICTs in Switzerland are regularly underfunded. First, it was reported that IICT budget estimations are often inaccurate because of poor planning and preparation, unforeseeable events, investigators intentionally underestimating budgets, and limited budget assessment and oversight. Second, with the exception of a specific IICT funding program by the Swiss National Science Foundation, it was reported that limited funding sources and unrealistic expectation of funders led to underlying challenges in getting IICTs fully funded. A number of measures that could help reduce the underfunding of IICTs were identified, including improving the support of investigators and IICTs, strengthening networking and guidance, harmonizing and simplifying bureaucracy, and increasing public funding of IICTs.

CONCLUSIONS AND RELEVANCE

This study highlights the inadequate expertise of Swiss stakeholders to correctly, systematically, and reproducibly calculate RCT budgets and the need for transparency on trial costs as well as training in budgeting practices. Limited financial resources for academic clinical research and issues regarding the professional planning and conduct of IICTs are persistent issues that many other countries also face.

Higher PEEP for acute respiratory distress syndrome: a Bayesian meta-analysis of randomised clinical trials

Objective: Benefit or harm of higher positive end expiratory pressure (PEEP) for acute respiratory distress syndrome (ARDS) is controversial. We aimed to assess the impact of higher levels of PEEP in patients with ARDS under a Bayesian framework. Design: Systematic review and Bayesian meta-analysis of randomised clinical trials comparing higher to lower PEEP in adult patients with ARDS. Data sources: MEDLINE, EMBASE and Cochrane Central Register of Controlled Trials from 1996 to 1 March 2020. Review methods: We extracted data from high quality randomised clinical trials comparing higher to lower levels of PEEP in adult patients, using low tidal volume in both arms, and conducted a Bayesian meta-analysis using aggregate data from these studies. Results: Eight clinical trials including 3703 patients (n = 1833 for higher PEEP, n = 1870 for lower PEEP) were included. Under a minimally informative prior, the posterior probability of benefit with higher PEEP was 65% (relative risk, 0.97 [95% credible interval, 0.78-1.14]). In patients with moderate-to- severe ARDS, the posterior probability of benefit with higher PEEP was 77% (relative risk, 0.94 [95% credible interval, 0.77-1.13]). Down-weighting studies that employed a maximum recruitment strategy by 100% increased the posterior probability of benefit to 92% under a minimally informative prior. Conclusions: The probability of benefit or harm from routine use of higher PEEP for patients with ARDS ranges from 27% to 86%, and from 14% to 73% depending on one's prior, suggesting continued uncertainty and equipoise regarding the benefit of PEEP If data from trials using a maximum recruitment strategy is discounted to some extent because of uncertainty over the appropriateness of this approach, the available evidence suggests that higher PEEP could be beneficial for moderate-to-severe ARDS. However, well powered randomised clinical trials are needed to confirm these findings.

Influence of Prednisone on Inflammatory Biomarkers in Community-Acquired Pneumonia: Secondary Analysis of a Randomized Trial

Glucocorticoids are frequently prescribed in inflammatory diseases and have recently experienced a boom in the treatment of COVID-19. Small studies have shown an effect of glucocorticoids on inflammatory marker levels, but definitive proof is lacking. We investigated the influence of prednisone on inflammatory biomarkers in a previous multicenter, randomized, placebo-controlled trial that compared a 7-day treatment course of 50-mg prednisone to placebo in patients hospitalized with community-acquired pneumonia. We compared levels of C-reactive protein (CRP), procalcitonin (PCT), leukocyte and neutrophil count between patients with and without glucocorticoid treatment at baseline and on days 3, 5, and 7 and at discharge by Wilcoxon tests and analysis of variance. A total of 356 patient data sets in the prednisone group and 355 in the placebo group were available for analysis. Compared to placebo, use of prednisone was associated with reductions in levels of CRP on days 3, 5, and 7 (mean difference of 46%, P < .001 for each time point). For PCT, no such difference was observed. Leukocyte and neutrophil count were higher in the prednisone group at all time points (mean difference of 27% for leukocytes and 33% for neutrophils, P <.001 for all time points). We conclude that after administration of glucocorticoids in community-acquired pneumonia, patients had lower CRP levels and increased leukocyte and neutrophil count as compared to the placebo group. PCT levels were not different between treatment groups. PCT levels thus may more appropriately mirror the resolution of infection compared to more traditional inflammatory markers.

Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses

BACKGROUND

Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN).

METHODS

To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification.

RESULTS

The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument.

INTERPRETATION

The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.

Association of Supporting Trial Evidence and Reimbursement for Off-Label Use of Cancer Drugs

IMPORTANCE

In many health systems, access to off-label drug use is controlled through reimbursement restrictions by health insurers, especially for expensive cancer drugs.

OBJECTIVE

To determine whether evidence from randomized clinical trials is associated with reimbursement decisions for requested off-label use of anticancer drugs in the Swiss health system.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used reimbursement requests from routinely collected health records of 5809 patients with drug treatment for cancer between January 2015 and July 2018 in 3 major cancer centers, covering cancer care of approximately 5% of the Swiss population, to identify off-label drug use. For each off-label use indication with 3 or more requests, randomized clinical trial evidence on treatment benefits was systematically identified for overall survival (OS) or progression-free survival (PFS). Data were analyzed from August 2018 to December 2020.

EXPOSURES

Available randomized clinical trial evidence on benefits for OS or PFS for requested off-label use indications.

MAIN OUTCOMES AND MEASURES

The main outcome was the association between evidence for treatment benefit (expressed as improved OS or PFS) and reimbursement in multivariable regression models.

RESULTS

Among 3046 patients with cancer, 695 off-label use reimbursement requests in 303 different indications were made for 598 patients (median [interquartile range] age, 64 [53-73] years; 420 [60%] men). Off-label use was intended as first-line treatment in 311 requests (45%). Reimbursement was accepted in 446 requests (64%). For 71 indications, including 431 requests for 376 patients, there were 3 or more requests. Of these, 246 requests (57%) had no supporting evidence for OS or PFS benefit. Reimbursement was granted in 162 of 246 requests without supporting evidence (66%). Of 117 requests supported by OS benefit, 79 (67%) were reimbursed, and of 68 requests supported by PFS benefit alone, 54 (79%) were reimbursed. Evidence of OS benefit from randomized clinical trials was not associated with a higher chance of reimbursement (odds ratio, 0.76, 95% CI, 0.45-1.27).

CONCLUSIONS AND RELEVANCE

These findings suggest that in a health care system enabling access to off-label use, it was frequently intended as a first-line treatment in cancer care. Availability of randomized clinical trial evidence showing survival benefit was not associated with reimbursement decisions for off-label anticancer drug treatment in Switzerland. A transparent process with criteria considering clinical evidence is needed for evidence-based reimbursement decisions to ensure fair access to cancer treatments.