Measuring quality of reporting in systematic reviews of diagnostic test accuracy studies in medical imaging: comparison of PRISMA-DTA and PRISMA

OBJECTIVES

To compare the reporting quality measured by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy studies (PRISMA-DTA) vs the original PRISMA checklist for systematic reviews of diagnostic test accuracy studies in imaging and survey the use of PRISMA-DTA by researchers and endorsement by journals.

METHODS

Systematic reviews of DTA studies published in 2020 and 2021 in Quartile 1 and Quartile 3 medical imaging journals (defined by Journal Citation Reports) were identified through PubMed. The reporting of each systematic review was assessed using PRISMA-DTA, PRISMA-2009 and PRISMA-2020. The item scores and overall score were compared among the three checklists. We also examined checklist adoption by the included systematic reviews and surveyed checklist endorsement from author instructions of included journals.

RESULTS

A total of 173 systematic reviews from 66 journals were included. The use of PRISMA-DTA, compared with PRISMA-2009 and PRISMA-2020, identified more issues in the reporting of title (proportion of systematic reviews with proper reporting, 27.2% vs 98.8% vs 98.8%), abstract (39.3% vs 97.1% vs 64.7%), eligibility criteria (67.6% vs 94.2% vs 94.2%), search (28.9% vs 72.3% vs 28.9%), definitions for data extraction (14.5% vs 91.9% vs 91.9%), diagnostic accuracy measures (38.2% vs 93.6% vs 93.6%), synthesis of results (28.9% vs 89.6% vs 73.4%) and results of individual studies (40.5% vs 80.3% vs 80.3%). The overall median reporting score measured by PRISMA-DTA (72.0% (interquartile range (IQR), 66.7-77.8%)) was lower than that measured by PRISMA-2009 (88.9% (IQR, 84.0-92.6%)) and similar to that measured by PRISMA-2020 (74.1% (IQR, 66.7-77.8%)). Additionally, PRISMA-DTA was used by only 43 (24.9%) systematic reviews and endorsed by two (3.0%) journals. These trends remained consistent for reviews published in journals with diverse scientific impact.

CONCLUSIONS

The use of PRISMA-DTA may identify more reporting inadequacies compared with the original PRISMA checklists when assessing diagnostic test accuracy systematic reviews, especially in critical sections such as title, abstract and methods. However, this tool is not commonly used by researchers and is inadequately endorsed by imaging journals. Our findings suggest a strong need to use PRISMA-DTA for reporting of diagnostic test accuracy systematic reviews by authors and its endorsement by journals. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

Completeness of Reporting of Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy (DTA) of Radiological Articles Based on the PRISMA-DTA Reporting Guideline

RATIONALE AND OBJECTIVES

This study evaluated the completeness of systematic reviews and meta-analyses in radiology using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic Test Accuracy (PRISMA-DTA) and PRISMA-DTA for Abstracts guidelines between articles published before and those published after the issuance of the guideline and identify areas that have been poorly reported.

MATERIALS AND METHODS

PubMed were searched for systematic reviews on DTA with or without meta-analyses published in general radiology journals between January 1, 2016 and December 31, 2020. The identified articles were assessed for completeness of reporting according to the PRISMA-DTA. Subgroup analyses were performed for association of completeness of reporting with multiple cofactors.

RESULTS

The search identified 183 reviews from 12 journals. The mean numbers (standard deviation) of reported PRISMA-DTA and PRISMA-DTA for Abstracts items in the full texts and abstracts were 18.45 (2.02) and 5.66 (1.28), respectively. Subgroup analysis showed that compared to the corresponding reference groups, a higher mean number of reported PRISMA-DTA items was associated with publication during July 2018-December 2020 [(17.82 (2.01) vs 18.89 (1.91); p = 0.034), citation of the PRISMA-DTA [17.62 (1.86) vs 20.27 (2.02); p < 0.001], and inclusion of supplementary materials [17.64 (2) vs 19.09 (1.8); p < 0.001] on multiple-linear regression analysis.

CONCLUSION

Completeness of reporting with respect to the PRISMA-DTA and PRISMA-DTA for Abstracts has improved modestly since the publication of the PRISMA-DTA guideline; however, increasing awareness of the specific weakness provides the chance for completeness improvement.

Artificial Intelligence Compared to Radiologists for the Initial Diagnosis of Prostate Cancer on Magnetic Resonance Imaging: A Systematic Review and Recommendations for Future Studies

Computer-aided diagnosis (CAD) of prostate cancer on multiparametric magnetic resonance imaging (mpMRI), using artificial intelligence (AI), may reduce missed cancers and unnecessary biopsies, increase inter-observer agreement between radiologists, and alleviate pressures caused by rising case incidence and a shortage of specialist radiologists to read prostate mpMRI. However, well-designed evaluation studies are required to prove efficacy above current clinical practice. A systematic search of the MEDLINE, EMBASE, and arXiv electronic databases was conducted for studies that compared CAD for prostate cancer detection or classification on MRI against radiologist interpretation and a histopathological reference standard, in treatment-naïve men with a clinical suspicion of prostate cancer. Twenty-seven studies were included in the final analysis. Due to substantial heterogeneities in the included studies, a narrative synthesis is presented. Several studies reported superior diagnostic accuracy for CAD over radiologist interpretation on small, internal patient datasets, though this was not observed in the few studies that performed evaluation using external patient data. Our review found insufficient evidence to suggest the clinical deployment of artificial intelligence algorithms at present. Further work is needed to develop and enforce methodological standards, promote access to large diverse datasets, and conduct prospective evaluations before clinical adoption can be considered.

Steps toward more complete reporting of systematic reviews of diagnostic test accuracy: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA)

Reporting standards in biomedical research have been shown to be suboptimal. The publication of the PRISMA statement has improved the completeness of reporting of systematic reviews, but several issues specific to diagnostic test accuracy are not included in the PRISMA statement. Therefore, a diagnostic test accuracy extension of the PRISMA statement, PRISMA-DTA, was created. This commentary addresses completeness of reporting in systematic reviews, the PRISMA-DTA statement, and strategies for optimal uptake of reporting guidelines.