Poor reporting quality of observational clinical studies comparing treatments of COVID-19 – a retrospective cross-sectional study

Point-of-Care Lung Ultrasound in Emergency Medicine: A Scoping Review With an Interactive Database

BACKGROUND

This scoping review was conducted to provide an overview of the evidence of point-of-care lung ultrasound (LUS) in emergency medicine. By emphasizing clinical topics, time trends, study designs, and the scope of the primary outcomes, a map is provided for physicians and researchers to guide their future initiatives.

RESEARCH QUESTION

Which study designs and primary outcomes are reported in published studies of LUS in emergency medicine?

STUDY DESIGN AND METHODS

We performed a systematic search in the PubMed/MEDLINE, Embase, Web of Science, Scopus, and Cochrane Library databases for LUS studies published prior to May 13, 2023. Study characteristics were synthesized quantitatively. The primary outcomes in all papers were categorized into the hierarchical Fryback and Thornbury levels.

RESULTS

A total of 4,076 papers were screened and, following selection and handsearching, 406 papers were included. The number of publications doubled from January 2020 to May 2023 (204 to 406 papers). The study designs were primarily observational (n = 375 [92%]), followed by randomized (n = 18 [4%]) and case series (n = 13 [3%]). The primary outcome measure concerned diagnostic accuracy in 319 papers (79%), diagnostic thinking in 32 (8%), therapeutic changes in 4 (1%), and patient outcomes in 14 (3%). No increase in the proportions of randomized controlled trials or the scope of primary outcome measures was observed with time. A freely available interactive database was created to enable readers to search for any given interest (https://public.tableau.com/app/profile/blinded/viz/LUSinEM_240216/INFO).

INTERPRETATION

Observational diagnostic studies have been produced in abundance, leaving a paucity of research exploring clinical utility. Notably, research exploring whether LUS causes changes to clinical decisions is imperative prior to any further research being made into patient benefits.

Visualizing the target estimand in comparative effectiveness studies with multiple treatments

Aim: Comparative effectiveness research using real-world data often involves pairwise propensity score matching to adjust for confounding bias. We show that corresponding treatment effect estimates may have limited external validity, and propose two visualization tools to clarify the target estimand. Materials & methods: We conduct a simulation study to demonstrate, with bivariate ellipses and joy plots, that differences in covariate distributions across treatment groups may affect the external validity of treatment effect estimates. We showcase how these visualization tools can facilitate the interpretation of target estimands in a case study comparing the effectiveness of teriflunomide (TERI), dimethyl fumarate (DMF) and natalizumab (NAT) on manual dexterity in patients with multiple sclerosis. Results: In the simulation study, estimates of the treatment effect greatly differed depending on the target population. For example, when comparing treatment B with C, the estimated treatment effect (and respective standard error) varied from -0.27 (0.03) to -0.37 (0.04) in the type of patients initially receiving treatment B and C, respectively. Visualization of the matched samples revealed that covariate distributions vary for each comparison and cannot be used to target one common treatment effect for the three treatment comparisons. In the case study, the bivariate distribution of age and disease duration varied across the population of patients receiving TERI, DMF or NAT. Although results suggest that DMF and NAT improve manual dexterity at 1 year compared with TERI, the effectiveness of DMF versus NAT differs depending on which target estimand is used. Conclusion: Visualization tools may help to clarify the target population in comparative effectiveness studies and resolve ambiguity about the interpretation of estimated treatment effects.

Transparency, openness, and reproducible research practices are frequently underused in health economic evaluations

OBJECTIVES

To investigate the extent to which articles of economic evaluations of healthcare interventions indexed in MEDLINE incorporate research practices that promote transparency, openness, and reproducibility.

STUDY DESIGN AND SETTING

We evaluated a random sample of health economic evaluations indexed in MEDLINE during 2019. We included articles written in English reporting an incremental cost-effectiveness ratio in terms of costs per life years gained, quality-adjusted life years, and/or disability-adjusted life years. Reproducible research practices, openness, and transparency in each article were extracted in duplicate. We explored whether reproducible research practices were associated with self-report use of a guideline.

RESULTS

We included 200 studies published in 147 journals. Almost half were published as open access articles (n = 93; 47%). Most studies (n = 150; 75%) were model-based economic evaluations. In 109 (55%) studies, authors self-reported use a guideline (e.g., for study conduct or reporting). Few studies (n = 31; 16%) reported working from a protocol. In 112 (56%) studies, authors reported the data needed to recreate the incremental cost-effectiveness ratio for the base case analysis. This percentage was higher in studies using a guideline than studies not using a guideline (72/109 [66%] with guideline vs. 40/91 [44%] without guideline; risk ratio 1.50, 95% confidence interval 1.15-1.97). Only 10 (5%) studies mentioned access to raw data and analytic code for reanalyses.

CONCLUSION

Transparency, openness, and reproducible research practices are frequently underused in health economic evaluations. This study provides baseline data to compare future progress in the field.

From randomised controlled trials to real-world data: Clinical evidence to guide management of diabetic macular oedema

Randomised clinical trials (RCTs) are generally considered the gold-standard for providing scientific evidence for treatments' effectiveness and safety but their findings may not always be generalisable to the broader population treated in routine clinical practice. RCTs include highly selected patient populations that fit specific inclusion and exclusion criteria. Although they may have a lower level of certainty than RCTs on the evidence hierarchy, real-world data (RWD), such as observational studies, registries and databases, provide real-world evidence (RWE) that can complement RCTs. For example, RWE may help satisfy requirements for a new indication of an already approved drug and help us better understand long-term treatment effectiveness, safety and patterns of use in clinical practice. Many countries have set up registries, observational studies and databases containing information on patients with retinal diseases, such as diabetic macular oedema (DMO). These DMO RWD have produced significant clinical evidence in the past decade that has changed the management of DMO. RWD and medico-administrative databases are a useful resource to identify low frequency safety signals. They often have long-term follow-up with a large number of patients and minimal exclusion criteria. We will discuss improvements in healthcare information exchange technologies, such as blockchain technology and FHIR (Fast Healthcare Interoperability Resources), which will connect and extend databases already available. These registries can be linked with existing or emerging retinal imaging modalities using artificial intelligence to aid diagnosis, treatment decisions and provide prognostic information. The results of RCTs and RWE are combined to provide evidence-based guidelines.

A Systematic Review to Inform the Development of a Reporting Guideline for Concept Mapping Research

Concept mapping is a phased, mixed-method approach that is increasingly used in health research to develop an understanding of complex phenomena. The six phases of concept mapping are preparation, idea generation, structuring (clustering and prioritization), data analysis, interpretation, and utilization of the map. The reporting of concept mapping research requires the development of a specific reporting guideline. We conducted a systematic review to identify candidate reporting items for inclusion in a reporting guideline. Three databases (MEDLINE, CINAHL, and PsycInfo) were searched to identify studies that used concept mapping methodology. We included 75 concept mapping studies published since 2019 from which we extracted information about the quality of reporting. A third of the studies focused on public health. We identified 71 candidate items that relate to the quality of reporting concept mapping research. The rationale for the study, the focus prompt, procedures for brainstorming, and structuring statements were consistently reported across the included studies. The process for developing the focus prompt, the rationale for the size of the stakeholder groups, and the process for determining the final concept map were generally not reported. The findings from the review will be used to inform the development of our reporting guideline for concept mapping research.

Poor reporting quality of randomized controlled trials comparing treatments of COVID-19-A retrospective cross-sectional study on the first year of publications

INTRODUCTION

Transparent and complete reporting of randomized controlled trials (RCTs) is essential for critical scientific appraisal of the results. It has been argued whether publications during the COVID-19 pandemic have met reporting standards. In this study, we assessed reporting adherence of RCTs on treatment interventions in COVID-19 patients to the CONSORT checklist and discuss which lessons can be learned to improve reporting in the future.

METHODS

This was a retrospective, cross-sectional study performed at the University Hospital RWTH Aachen, Germany. We conducted a pragmatic systematic literature search in the PubMed database to identify RCTs on treatment interventions in COVID-19 patients in the first year of publications on the topic (March 2020-February 2021). We investigated the adherence of each publication to the CONSORT checklist and assessed the association between specific predictors and percentage adherence in an exploratory multivariable regression model.

RESULTS

We analyzed 127 RCTs and found that the median percentage adherence to the CONSORT checklist was 54.3% [IQR 38.9 to 65.7]. In the exploratory multivariable regression model, the impact factor (highest tertile of impact factor compared to lowest tertile ß = 21.77, 95% CI 13.89 to 29.66, p<0.001; middle tertile compared lowest tertile ß = 11.79, 95% CI 5.74 to 17.84, p<0.001)) and authors' referral to the CONSORT statement (ß = 9.29, 95% CI 2.98 to 15.60, p = 0.004) were associated with a higher percentage adherence to the CONSORT checklist.

CONCLUSION

The reporting quality of RCTs on treatment interventions in COVID-19 patients during the first year of publications was poor. Measures to improve reporting quality are urgently needed.

Totality of evidence of the effectiveness of repurposed therapies for COVID-19: Can we use real-world studies alongside randomized controlled trials?

Rapid and robust strategies to evaluate the efficacy and effectiveness of novel and existing pharmacotherapeutic interventions (repurposed treatments) in future pandemics are required. Observational "real-world studies" (RWS) can report more quickly than randomized controlled trials (RCTs) and would have value were they to yield reliable results. Both RCTs and RWS were deployed during the coronavirus disease 2019 (COVID-19) pandemic. Comparing results between them offers a unique opportunity to determine the potential value and contribution of each. A learning review of these parallel evidence channels in COVID-19, based on quantitative modeling, can help improve speed and reliability in the evaluation of repurposed therapeutics in a future pandemic. Analysis of all-cause mortality data from 249 observational RWS and RCTs across eight treatment regimens for COVID-19 showed that RWS yield more heterogeneous results, and generally overestimate the effect size subsequently seen in RCTs. This is explained in part by a few study factors: the presence of RWS that are imbalanced for age, gender, and disease severity, and those reporting mortality at 2 weeks or less. Smaller studies of either type contributed negligibly. Analysis of evidence generated sequentially during the pandemic indicated that larger RCTs drive our ability to make conclusive decisions regarding clinical benefit of each treatment, with limited inference drawn from RWS. These results suggest that when evaluating therapies in future pandemics, (1) large RCTs, especially platform studies, be deployed early; (2) any RWS should be large and should have adequate matching of known confounders and long follow-up; (3) reporting standards and data standards for primary endpoints, explanatory factors, and key subgroups should be improved; in addition, (4) appropriate incentives should be in place to enable access to patient-level data; and (5) an overall aggregate view of all available results should be available at any given time.

Reporting of Observational Studies Explicitly Aiming to Emulate Randomized Trials: A Systematic Review

Importance

Observational (nonexperimental) studies that aim to emulate a randomized trial (ie, the target trial) are increasingly informing medical and policy decision-making, but it is unclear how these studies are reported in the literature. Consistent reporting is essential for quality appraisal, evidence synthesis, and translation of evidence to policy and practice.

Objective

To assess the reporting of observational studies that explicitly aimed to emulate a target trial.

Evidence Review

We searched Medline, Embase, PsycINFO, and Web of Science for observational studies published between March 2012 and October 2022 that explicitly aimed to emulate a target trial of a health or medical intervention. Two reviewers double-screened and -extracted data on study characteristics, key predefined components of the target trial protocol and its emulation (eligibility criteria, treatment strategies, treatment assignment, outcome[s], follow-up, causal contrast[s], and analysis plan), and other items related to the target trial emulation.

Findings

A total of 200 studies that explicitly aimed to emulate a target trial were included. These studies included 26 subfields of medicine, and 168 (84%) were published from January 2020 to October 2022. The aim to emulate a target trial was explicit in 70 study titles (35%). Forty-three studies (22%) reported use of a published reporting guideline (eg, Strengthening the Reporting of Observational Studies in Epidemiology). Eighty-five studies (43%) did not describe all key items of how the target trial was emulated and 113 (57%) did not describe the protocol of the target trial and its emulation.

Conclusion and Relevance

In this systematic review of 200 studies that explicitly aimed to emulate a target trial, reporting of how the target trial was emulated was inconsistent. A reporting guideline for studies explicitly aiming to emulate a target trial may improve the reporting of the target trial protocols and other aspects of these emulation attempts.

A scoping review establishes need for consensus guidance on reporting health equity in observational studies

OBJECTIVES

To evaluate the support from the available guidance on reporting of health equity in research for our candidate items and to identify additional items for the Strengthening Reporting of Observational studies in Epidemiology-Equity extension.

STUDY DESIGN AND SETTING

We conducted a scoping review by searching Embase, MEDLINE, CINAHL, Cochrane Methodology Register, LILACS, and Caribbean Center on Health Sciences Information up to January 2022. We also searched reference lists and gray literature for additional resources. We included guidance and assessments (hereafter termed "resources") related to conduct and/or reporting for any type of health research with or about people experiencing health inequity.

RESULTS

We included 34 resources, which supported one or more candidate items or contributed to new items about health equity reporting in observational research. Each candidate item was supported by a median of six (range: 1-15) resources. In addition, 12 resources suggested 13 new items, such as "report the background of investigators".

CONCLUSION

Existing resources for reporting health equity in observational studies aligned with our interim checklist of candidate items. We also identified additional items that will be considered in the development of a consensus-based and evidence-based guideline for reporting health equity in observational studies.

PROBE 2023 guidelines for reporting observational studies in Endodontics: A consensus-based development study

Observational studies are non-interventional studies that establish the prevalence and incidence of conditions or diseases in populations or analyse the relationship between health status and other variables. They also facilitate the development of specific research questions for future randomized trials or to answer important scientific questions when trials are not possible to carry out. This article outlines the previously documented consensus-based approach by which the Preferred Reporting items for Observational studies in Endodontics (PROBE) 2023 guidelines were developed. A steering committee of nine members was formed, including the project leaders (PD, VN). The steering committee developed an initial checklist by combining and adapting items from the STrengthening the Reporting of Observational studies in Epidemiology (STROBE) checklist and the Clinical and Laboratory Images in Publications (CLIP) principles, as well as adding several new items specifically for the specialty of Endodontics. The steering committee then established a PROBE Delphi Group (PDG) and a PROBE Online Meeting Group (POMG) to obtain expert input and feedback on the preliminary draft checklist. The PDG members participated in an online Delphi process to reach consensus on the clarity and suitability of the items present in the PROBE checklist. The POMG then held detailed discussions on the PROBE checklist generated through the online Delphi process. This online meeting was held via the Zoom platform on 7th October 2022. Following this meeting, the steering committee revised the PROBE checklist, which was piloted by several authors when preparing a manuscript describing an observational study for publication. The PROBE 2023 checklist consists of 11 sections and 58 items. Authors are now encouraged to adopt the PROBE 2023 guidelines, which will improve the overall reporting quality of observational studies in Endodontics. The PROBE 2023 checklist is freely available and can be downloaded from the PRIDE website (https://pride-endodonticguidelines.org/probe/).

The reporting adherence of observational studies published in orthodontic journals in relation to STROBE guidelines: a meta-epidemiological assessment

BACKGROUND

To facilitate clear and transparent reporting of observational studies the STROBE guidelines were developed. The aim of this investigation was to assess the reporting adherence of observational studies published in orthodontic journals in relation to STROBE guidelines. Associations between the reporting quality and study characteristics were explored.

MATERIALS AND METHOD

A search of five leading orthodontic journals was undertaken to identify observational studies published between 1st January 2021 and 31st June 2021. Data extraction was performed in duplicate and independently. Descriptive statistics and frequency distributions for the responses to each checklist item were calculated. Proportional odds ordinal logistic regression for clustered data was implemented to assess potential associations between the three-level categorical outcome (not reported, inadequate reporting, adequate reporting) and study characteristics and individual checklist items.

RESULTS

One hundred and thirty-five studies were analysed. The majority of studies were cohort designs (54.0%). Variability in the reporting of the STROBE guideline items was evident. In particular, a clear description of outcomes, exposures, predictors, potential confounders, and effect modifiers, statistical methods and participants were inadequately reported. In the multivariable analysis, the overall score test indicated that only item was a significant predictor of reporting quality (P < 0.001).

CONCLUSIONS

The findings of this study have highlighted that the reporting of observational studies published in orthodontic journals in relation to the STROBE guidelines is sub-optimal. Key areas of inadequate reporting relate to methodology and results. Key determinant of reporting quality was the STROBE item.

Safety and preliminary efficacy of sequential multiple ascending doses of solnatide to treat pulmonary permeability edema in patients with moderate to severe ARDS in a randomized, placebo-controlled, double-blind trial: preliminary evaluation of safety and feasibility in light of the COVID-19 pandemic

Background

In May 2018, the first patient was enrolled in the phase-IIb clinical trial “Safety and Preliminary Efficacy of Sequential Multiple Ascending Doses of Solnatide to Treat Pulmonary Permeability Edema in Patients with Moderate to Severe ARDS.” With the onset of the COVID-19 pandemic in early 2020, the continuation and successful execution of this clinical study was in danger. Therefore, before the Data Safety Monitoring Board (DSMB) allowed proceeding with the study and enrollment of further COVID-19 ARDS patients into it, additional assessment on possible study bias was considered mandatory.

Methods

We conducted an ad hoc interim analysis of 16 patients (5 COVID-19- ARDS patients and 11 with ARDS from different causes) from the phase-IIB clinical trial. We assessed possible differences in clinical characteristics of the ARDS patients and the impact of the pandemic on study execution.

Results

COVID-19 patients seemed to be less sick at baseline, which also showed in higher survival rates over the 28-day observation period. Trial specific outcomes regarding pulmonary edema and ventilation parameters did not differ between the groups, nor did more general indicators of (pulmonary) sepsis like oxygenation ratio and required noradrenaline doses.

Conclusion

The DSMB and the investigators did not find any evidence that patients suffering from ARDS due to SARS-CoV-2 may be at higher (or generally altered) risk when included in the trial, nor were there indications that those patients might influence the integrity of the study data altogether. For this reason, a continuation of the phase IIB clinical study activities can be justified. Researchers continuing clinical trials during the pandemic should always be aware that the exceptional circumstances may alter study results and therefore adaptations of the study design might be necessary.