Reporting Guidelines for Imaging Research

Completeness of reporting for systematic reviews of point-of-care ultrasound: a meta-research study

OBJECTIVES

Systematic reviews are often considered among the highest quality of evidence. Completely reported systematic reviews, however, are required so readers can assess for generalisability of the research to practice and risk of bias. The objective of this study was to assess the completeness of reporting for systematic reviews assessing the diagnostic accuracy of point-of-care ultrasound (POCUS) using the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Diagnostic Test Accuracy (PRISMA-DTA) checklist that was published in 2018.

DESIGN AND SETTING

In this meta-research study, MEDLINE, EMBASE and Cochrane Library databases were searched, with no date restriction, on March 1st, 2020 for systematic reviews assessing the diagnostic accuracy of POCUS. Adherence to PRISMA-DTA for the main text and abstract was scored independently and in duplicate using a modified checklist. Prespecified subgroup analyses were performed.

MAIN OUTCOME MEASURES

The primary outcome was the mean PRISMA-DTA checklist adherence for the full-text and abstract.

RESULTS

A total of 71 studies published from 2008 to 2020 met the inclusion criteria. The overall adherence for the full-text was moderate: 19.8 out of 26.0 items (76%) and for the abstract was 7.0 out of 11.0 items (64%). Although many items in the PRISMA-DTA checklist were frequently reported, several were r infrequently reported (<33% of studies), including item 5 (protocol registration), item D2 (minimally acceptable test accuracy) and item 14 (variability in target condition, index test and reference standards). Subgroup analyses showed a higher PRISMA-DTA mean adherence (SD) for high impact journals (20.9 (2.52) vs 18.9 (1.95); p<0.001), studies including supplemental materials (20.6 (2.48) vs 18.9 (2.28); p=0.004), studies citing adherence to PRISMA reporting guidelines (20.4 (1.95) vs 19.0 (3.00); p=0.038) and studies published in journals endorsing PRISMA guidelines (20.2 (2.47) vs 18.6 (2.37); p=0.025). There was variable adherence based on journal of publication (p=0.006), but not for study population (adult vs paediatric vs mixed) (p=0.62), year of publication (p=0.94), body region (p=0.78) or country (p=0.40). There was no variability in abstract adherence based on whether the abstract was structured with subheadings or not (p=0.25). A Spearman's correlation found moderate correlation between higher word counts and abstractand full-text adherence (R=0.45, p<0.001 and R=0.38, p=0.001), respectively.

CONCLUSIONS

Overall, the reporting of POCUS diagnostic accuracy systematic reviews and meta-analyses was moderate. We identified deficits in several key areas including the preregistration of systematic reviews in an online repository, handling of multiple definitions of target conditions, index tests and reference standards and specifying minimally acceptable test accuracy. Prospective registration of reviews and detailed reporting as per PRISMA-DTA during the research process could improve reporting completeness. At an editorial level, word count and supplemental material limitations may impede reporting completeness, whereas endorsement of reporting guidelines on journal websites could improve reporting.

Evidence-Based Medicine in the Domain of Nuclear Medicine, the Fifty-Year View

The discipline of evidence-based medicine (EBM), though yet unnamed, was in its infancy when Seminars in Nuclear Medicine was inaugurated in 1971; commemorating the golden anniversary of this prestigious journal and the contemporaneous reign of its editors by publishing a 50-year historical consideration of EBM seems most apropos. EBM should be understood as a system of methods to eliminate partiality and improve the quality of evidence in the performance and review of data; much of EBM revolves around ensuring that conclusions are derived from rigorous research studies that protect against bias and are widely generalizable to other groups of patients. Subdomains within EBM that we will survey in this review include methods of performing and evaluating primary studies, standards of reporting of medical studies, methods of combining and aggregating data, and finally, methods of creating clinical practice guidelines. While many practitioners of nuclear medicine may not presently be familiar with the innovations of EBM, having been introduced after their formal education was completed, with the eventual arrival of more-recently trained staff, firm recommendations from our primary research journals, and with efforts to educate practicing physicians, this shortcoming is being addressed.

How to Assess Quality of Primary Research Studies in the Medical Literature?

Accuracy studies are the cornerstone of the evaluation of new diagnostic tests. In an accuracy study, patients with a clinical suspicion of disease undergo both the new tests that is being evaluated, and the reference test or "gold-standard" test for the disease. Patients are stratified according to the test result and to whether or not the diagnosis was confirmed by the reference test. Main accuracy indices include the sensitivity (proportion of patients who test positive among those who have the disease), specificity (proportion of patients who test negative among those without the disease), as well as the positive and negative predictive values (proportion of patients with the disease among those with a positive test, and of patients without the disease among those with a negative test, respectively). In appraising an accuracy study, the reader should check that the study design and analysis follow methodological standards. The study population should be representative of the population in which the test will be used in practice. The new test should be interpreted in a blinded and independent fashion from the reference standard to avoid expectation bias, and should not be used to establish the diagnosis (incorporation bias). The reproducibility should be verified. Interpretation criteria and technical aspects should be described with enough details to allow replication. Provided that these conditions are met, the next step is to decide whether the test may be used for patient care. The clinical setting in which the test will be used, and the corresponding pretest probability of disease, will determine how best can the test be used in practice.