Cross-Sectional Evaluation of Open Science Practices at Imaging Journals: A Meta-Research Study

Objective: To evaluate open science policies of imaging journals, and compliance to these policies in published articles. Methods: From imaging journals listed we extracted open science policy details: protocol registration, reporting guidelines, funding, ethics and conflicts of interest (COI), data sharing, and open access publishing. The 10 most recently published studies from each journal were assessed to determine adherence to these policies. We calculated the proportion of open science policies into an Open Science Score (OSS) for all journals and articles. We evaluated relationships between OSS and journal/article level variables. Results: 82 journals/820 articles were included. The OSS of journals and articles was 58.3% and 31.8%, respectively. Of the journals, 65.9% had registration and 78.1% had reporting guideline policies. 79.3% of journals were members of COPE, 81.7% had plagiarism policies, 100% required disclosure of funding, and 97.6% required disclosure of COI and ethics approval. 81.7% had data sharing policies and 15.9% were fully open access. 7.8% of articles had a registered protocol, 8.4% followed a reporting guideline, 77.4% disclosed funding, 88.7% disclosed COI, and 85.6% reported ethics approval. 12.3% of articles shared their data. 51% of articles were available through open access or as a preprint. OSS was higher for journal with DOAJ membership (80% vs 54.2%; P < .0001). Impact factor was not correlated with journal OSS. Knowledge synthesis articles has a higher OSS scores (44.5%) than prospective/retrospective studies (32.6%, 30.0%, P < .0001). Conclusion: Imaging journals endorsed just over half of open science practices considered; however, the application of these practices at the article level was lower.

Screening for lung cancer with computed tomography: protocol for systematic reviews for the Canadian Task Force on Preventive Health Care

PURPOSE

Lung cancer is the leading cause of cancer deaths in Canada, and because early cancers are often asymptomatic screening aims to prevent mortality by detecting cancer earlier when treatment is more likely to be curative. These reviews will inform updated recommendations by the Canadian Task Force on Preventive Health Care on screening for lung cancer.

METHODS

We will update the review on the benefits and harms of screening with CT conducted for the task force in 2015 and perform de novo reviews on the comparative effects between (i) trial-based selection criteria and use of risk prediction models and (ii) trial-based nodule classification and different nodule classification systems and on patients' values and preferences. We will search Medline, Embase, and Cochrane Central (for questions on benefits and harms from 2015; comparative effects from 2012) and Medline, Scopus, and EconLit (for values and preferences from 2012) via peer-reviewed search strategies, clinical trial registries, and the reference lists of included studies and reviews. Two reviewers will screen all citations (including those in the previous review) and base inclusion decisions on consensus or arbitration by another reviewer. For benefits (i.e., all-cause and cancer-specific mortality and health-related quality of life) and harms (i.e., overdiagnosis, false positives, incidental findings, psychosocial harms from screening, and major complications and mortality from invasive procedures as a result of screening), we will include studies of adults in whom lung cancer is not suspected. We will include randomized controlled trials comparing CT screening with no screening or alternative screening modalities (e.g., chest radiography) or strategies (e.g., CT using different screening intervals, classification systems, and/or patient selection via risk models or biomarkers); non-randomized studies, including modeling studies, will be included for the comparative effects between trial-based and other selection criteria or nodule classification methods. For harms (except overdiagnosis) we will also include non-randomized and uncontrolled studies. For values and preferences, the study design may be any quantitative design that either directly or indirectly measures outcome preferences on outcomes pertaining to lung cancer screening. We will only include studies conducted in Very High Human Development Countries and having full texts in English or French. Data will be extracted by one reviewer with verification by another, with the exception of result data on mortality and cancer incidence (for calculating overdiagnosis) where duplicate extraction will occur. If two or more studies report on the same comparison and it is deemed suitable, we will pool continuous data using a mean difference or standardized mean difference, as applicable, and binary data using relative risks and a DerSimonian and Laird model unless events are rare (< 1%) where we will pool odds ratios using Peto's method or (if zero events) the reciprocal of the opposite treatment arm size correction. For pooling proportions, we will apply suitable transformation (logit or arcsine) depending on the proportions of events. If meta-analysis is not undertaken we will synthesize the data descriptively, considering clinical and methodological differences. For each outcome, two reviewers will independently assess within- and across-study risk of bias and rate the certainty of the evidence using GRADE (Grading of Recommendations Assessment, Development, and Evaluation), and reach consensus.

DISCUSSION

Since 2015, additional trials and longer follow-ups or additional data (e.g., harms, specific patient populations) from previously published trials have been published that will improve our understanding of the benefits and harms of screening. The systematic review of values and preferences will allow fulsome insights that will inform the balance of benefits and harms.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022378858.

Heterogeneity in Systematic Reviews of Medical Imaging Diagnostic Test Accuracy Studies: A Systematic Review

Importance

Systematic reviews of medical imaging diagnostic test accuracy (DTA) studies are affected by between-study heterogeneity due to a range of factors. Failure to appropriately assess the extent and causes of heterogeneity compromises the interpretability of systematic review findings.

Objective

To assess how heterogeneity has been examined in medical imaging DTA studies.

Evidence Review

The PubMed database was searched for systematic reviews of medical imaging DTA studies that performed a meta-analysis. The search was limited to the 40 journals with highest impact factor in the radiology, nuclear medicine, and medical imaging category in the InCites Journal Citation Reports of 2021 to reach a sample size of 200 to 300 included studies. Descriptive analysis was performed to characterize the imaging modality, target condition, type of meta-analysis model used, strategies for evaluating heterogeneity, and sources of heterogeneity identified. Multivariable logistic regression was performed to assess whether any factors were associated with at least 1 source of heterogeneity being identified in the included meta-analyses. Methodological quality evaluation was not performed. Data analysis occurred from October to December 2022.

Findings

A total of 242 meta-analyses involving a median (range) of 987 (119-441 510) patients across a diverse range of disease categories and imaging modalities were included. The extent of heterogeneity was adequately described (ie, whether it was absent, low, moderate, or high) in 220 studies (91%) and was most commonly assessed using the I2 statistic (185 studies [76%]) and forest plots (181 studies [75%]). Heterogeneity was rated as moderate to high in 191 studies (79%). Of all included meta-analyses, 122 (50%) performed subgroup analysis and 87 (36%) performed meta-regression. Of the 242 studies assessed, 189 (78%) included 10 or more primary studies. Of these 189 studies, 60 (32%) did not perform meta-regression or subgroup analysis. Reasons for being unable to investigate sources of heterogeneity included inadequate reporting of primary study characteristics and a low number of included primary studies. Use of meta-regression was associated with identification of at least 1 source of variability (odds ratio, 1.90; 95% CI, 1.11-3.23; P = .02).

Conclusions and Relevance

In this systematic review of assessment of heterogeneity in medical imaging DTA meta-analyses, most meta-analyses were impacted by a moderate to high level of heterogeneity, presenting interpretive challenges. These findings suggest that, despite the development and availability of more rigorous statistical models, heterogeneity appeared to be incomplete, inconsistently evaluated, or methodologically questionable in many cases, which lessened the interpretability of the analyses performed; comprehensive heterogeneity assessment should be addressed at the author level by improving personal familiarity with appropriate statistical methodology for assessing heterogeneity and involving biostatisticians and epidemiologists in study design, as well as at the editorial level, by mandating adherence to methodologic standards in primary DTA studies and DTA meta-analyses.

LI-RADS CT and MRI Ancillary Feature Association with Hepatocellular Carcinoma and Malignancy: An Individual Participant Data Meta-Analysis

Background The independent contribution of each Liver Imaging Reporting and Data System (LI-RADS) CT or MRI ancillary feature (AF) has not been established. Purpose To evaluate the association of LI-RADS AFs with hepatocellular carcinoma (HCC) and malignancy while adjusting for LI-RADS major features through an individual participant data (IPD) meta-analysis. Materials and Methods Medline, Embase, Cochrane Central Register of Controlled Trials, and Scopus were searched from January 2014 to January 2022 for studies evaluating the diagnostic accuracy of CT and MRI for HCC using LI-RADS version 2014, 2017, or 2018. Using a one-step approach, IPD across studies were pooled. Adjusted odds ratios (ORs) and 95% CIs were derived from multivariable logistic regression models of each AF combined with major features except threshold growth (excluded because of infrequent reporting). Liver observation clustering was addressed at the study and participant levels through random intercepts. Risk of bias was assessed using a composite reference standard and Quality Assessment of Diagnostic Accuracy Studies 2. Results Twenty studies comprising 3091 observations (2456 adult participants; mean age, 59 years ± 11 [SD]; 1849 [75.3%] men) were included. In total, 89% (eight of nine) of AFs favoring malignancy were associated with malignancy and/or HCC, 80% (four of five) of AFs favoring HCC were associated with HCC, and 57% (four of seven) of AFs favoring benignity were negatively associated with HCC and/or malignancy. Nonenhancing capsule (OR = 3.50 [95% CI: 1.53, 8.01]) had the strongest association with HCC. Diffusion restriction (OR = 14.45 [95% CI: 9.82, 21.27]) and mild-moderate T2 hyperintensity (OR = 10.18 [95% CI: 7.17, 14.44]) had the strongest association with malignancy. The strongest negative associations with HCC were parallels blood pool enhancement (OR = 0.07 [95% CI: 0.01, 0.49]) and marked T2 hyperintensity (OR = 0.18 [95% CI: 0.07, 0.45]). Seventeen studies (85%) had a high risk of bias. Conclusion Most LI-RADS AFs were independently associated with HCC, malignancy, or benignity as intended when adjusting for major features. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Crivellaro in this issue.

Impact of Size Thresholds on the Diagnosis of Incidental Adrenal Lesions: A Systematic Review and Meta-Analysis

BACKGROUND

Preferred size-threshold recommendations for management of incidental adrenal lesions remain controversial.

PURPOSE

This meta-analysis aimed to compare the diagnostic accuracy of different size thresholds for detecting malignancy in patients with incidental adrenal lesions on imaging.

MATERIALS AND METHODS

A systematic review of MEDLINE, Embase, Scopus, the Cochrane Library, and the gray literature, covering the period from inception to September 2021, was performed. Studies with >10 patients evaluating the diagnostic accuracy of imaging size thresholds for detecting malignancy in patients with incidental adrenal lesions and no prior history of cancer were included. Study, clinical, imaging, and accuracy data for eligible studies were independently acquired by two reviewers. Primary meta-analysis was performed using a bivariate mixed-effects regression model. Risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool.

RESULTS

From 2,690 citations, 40 studies (9,794 patients with mean age ranging from 41 to 66 years) were included. Most (36 of 40) were retrospective single-center studies. CT with or without MRI served as the index test(s). Sensitivity and specificity values, respectively, by size threshold used in the included studies were as follows: 85% (95% confidence interval [CI] 74%-91%) and 39% (95% CI 23%-57%) for 3-cm thresholds; 85% (95% CI 78%-90%) and 75% (95% CI 62%-85%) for 4-cm thresholds; 70% (95% CI 56%-81%) and 74% (95% CI 59%-85%) for 5-cm thresholds; and 75% (95% CI 67%-82%) and 77% (95% CI 62%-87%) for 6-cm thresholds. No cause for variability in sensitivity or specificity was identified on subgroup analysis of the 4-cm threshold. Nearly half of the studies (19 of 40) had at least one QUADAS-2 domain with a high risk of bias.

CONCLUSIONS

A 4-cm size threshold demonstrates the highest combined sensitivity and specificity, with a preserved specificity compared with higher size thresholds, but with a trend toward improved sensitivity. Future research reevaluating 4-5 cm size thresholds while excluding characteristically benign lesions by imaging may help redefine a size threshold that has improved specificity but preserved sensitivity, compared with the existing 4-cm threshold.

Percentage of Pancreatic Cysts on MRI With a Pancreatic Carcinoma: Systematic Review and Meta-Analysis

BACKGROUND

Pancreatic cystic lesions (PCLs) are frequent on MRI and are thought to be associated with pancreatic adenocarcinoma (PDAC) necessitating long-term surveillance based on older studies suffering from selection bias.

PURPOSE

To establish the percentage of patients with PCLs on MRI with a present or future PDAC.

STUDY TYPE

Systematic review, meta-analysis.

POPULATION

Adults with PCLs on MRI and a present or future diagnosis of PDAC were eligible. MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Scopus were searched to April 2022 (PROSPERO:CRD42022320502). Studies limited to PCLs not requiring surveillance, <100 patients, or those with a history/genetic risk of PDAC were excluded.

FIELD STRENGTH/SEQUENCE

≥1.5 T with ≥1 T2-weighted sequence.

ASSESSMENT

Two investigators extracted data, with discrepancies resolved by a third. QUADAS-2 assessed bias. PDAC was diagnosed using a composite reference standard.

STATISTICAL TESTS

A meta-analysis of proportions was performed at the patient-level with 95% confidence intervals (95% CI).

RESULTS

Eight studies with 1289 patients contributed to the percentage of patients with a present diagnosis of PDAC, and 10 studies with 3422 patients to the percentage with a future diagnosis. Of patients with PCLs on MRI, 14.8% (95% CI 2.4-34.9) had a PDAC at initial MRI, which decreased to 6.0% (2.2-11.3) for studies at low risk of bias. For patients without PDAC on initial MRI, 2.0% (1.1-3.2) developed PDAC during surveillance, similar for low risk of bias studies at 1.9% (0.7-3.6), with no clear trend of increased PDAC for longer surveillance durations. For patients without worrisome features or high-risk stigmata, 0.9% (0.1-2.2) developed PDAC during surveillance. Of 10, eight studies had a median surveillance ≥3 years (range 3-157 months). Sources of bias included retrospectively limiting PCLs to those with histopathology and inconsistent surveillance protocols.

DATA CONCLUSION

A low percentage of patients with PCLs on MRI develop PDAC while on surveillance. The first MRI revealing a PCL should be scrutinized for PDAC.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Comparative Performance of 2018 LI-RADS versus Modified LIRADS (mLI-RADS): An Individual Participant Data Meta-Analysis

BACKGROUND

LI-RADS version 2018 (v2018) is used for non-invasive diagnosis of hepatocellular carcinoma (HCC). A recently proposed modification (known as mLI-RADS) demonstrated improved sensitivity while maintaining specificity and positive predictive value (PPV) of LI-RADS category 5 (definite HCC) for HCC. However, mLI-RADS requires multicenter validation.

PURPOSE

To evaluate the performance of v2018 and mLI-RADS for liver lesions in a large, heterogeneous, multi-national cohort of patients at risk for HCC.

STUDY TYPE

Systematic review and meta-analysis using individual participant data (IPD) [Study Protocol: https://osf.io/duys4].

POPULATION

2223 observations from 1817 patients (includes all LI-RADS categories; females = 448, males = 1361, not reported = 8) at elevated risk for developing HCC (based on LI-RADS population criteria) from 12 retrospective studies.

FIELD STRENGTH/SEQUENCE

1.5T and 3T; complete liver MRI with gadoxetate disodium, including axial T2w images and dynamic axial fat-suppressed T1w images precontrast and in the arterial, portal venous, transitional, and hepatobiliary phases. Diffusion-weighted imaging was used when available.

ASSESSMENT

Liver observations were categorized using v2018 and mLI-RADS. The diagnostic performance of each system's category 5 (LR-5 and mLR-5) for HCC were compared.

STATISTICAL TESTS

The Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2 was applied to determine risk of bias and applicability. Diagnostic performances were assessed using the likelihood ratio test for sensitivity and specificity and the Wald test for PPV. The significance level was P < 0.05.

RESULTS

17% (2/12) of the studies were considered low risk of bias (244 liver observations; 164 patients). When compared to v2018, mLR-5 demonstrated higher sensitivity (61.3% vs. 46.5%, P < 0.001), similar PPV (85.3% vs. 86.3%, P = 0.89), and similar specificity (85.8% vs. 90.8%, P = 0.16) for HCC.

DATA CONCLUSION

This study confirms mLR-5 has higher sensitivity than LR-5 for HCC identification, while maintaining similar PPV and specificity, validating the mLI-RADS proposal in a heterogeneous, international cohort.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Individual Participant Data Meta-Analysis of LR-5 in LI-RADS Version 2018 versus Revised LI-RADS for Hepatocellular Carcinoma Diagnosis

Background A simplification of the Liver Imaging Reporting and Data System (LI-RADS) version 2018 (v2018), revised LI-RADS (rLI-RADS), has been proposed for imaging-based diagnosis of hepatocellular carcinoma (HCC). Single-site data suggest that rLI-RADS category 5 (rLR-5) improves sensitivity while maintaining positive predictive value (PPV) of the LI-RADS v2018 category 5 (LR-5), which indicates definite HCC. Purpose To compare the diagnostic performance of LI-RADS v2018 and rLI-RADS in a multicenter data set of patients at risk for HCC by performing an individual patient data meta-analysis. Materials and Methods Multiple databases were searched for studies published from January 2014 to January 2022 that evaluated the diagnostic performance of any version of LI-RADS at CT or MRI for diagnosing HCC. An individual patient data meta-analysis method was applied to observations from the identified studies. Quality Assessment of Diagnostic Accuracy Studies version 2 was applied to determine study risk of bias. Observations were categorized according to major features and either LI-RADS v2018 or rLI-RADS assignments. Diagnostic accuracies of category 5 for each system were calculated using generalized linear mixed models and compared using the likelihood ratio test for sensitivity and the Wald test for PPV. Results Twenty-four studies, including 3840 patients and 4727 observations, were analyzed. The median observation size was 19 mm (IQR, 11-30 mm). rLR-5 showed higher sensitivity compared with LR-5 (70.6% [95% CI: 60.7, 78.9] vs 61.3% [95% CI: 45.9, 74.7]; P < .001), with similar PPV (90.7% vs 92.3%; P = .55). In studies with low risk of bias (n = 4; 1031 observations), rLR-5 also achieved a higher sensitivity than LR-5 (72.3% [95% CI: 63.9, 80.1] vs 66.9% [95% CI: 58.2, 74.5]; P = .02), with similar PPV (83.1% vs 88.7%; P = .47). Conclusion rLR-5 achieved a higher sensitivity for identifying HCC than LR-5 while maintaining a comparable PPV at 90% or more, matching the results presented in the original rLI-RADS study. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Sirlin and Chernyak in this issue.

Best Practice for MRI Diagnostic Accuracy Research With Lessons and Examples from the LI-RADS Individual Participant Data Group

Medical imaging diagnostic test accuracy research is strengthened by adhering to best practices for study design, data collection, data documentation, and study reporting. In this review, key elements of such research are discussed, and specific recommendations provided for optimizing diagnostic accuracy study execution to improve uniformity, minimize common sources of bias and avoid potential pitfalls. Examples are provided regarding study methodology and data collection practices based on insights gained by the liver imaging reporting and data system (LI-RADS) individual participant data group, who have evaluated raw data from numerous MRI diagnostic accuracy studies for risk of bias and data integrity. The goal of this review is to outline strategies for investigators to improve research practices, and to help reviewers and readers better contextualize a study's findings while understanding its limitations. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 3.

Is There Evidence of P-Hacking in Imaging Research?

BACKGROUND

P-hacking, the tendency to run selective analyses until they become significant, is prevalent in many scientific disciplines.

PURPOSE

This study aims to assess if p-hacking exists in imaging research.

METHODS

Protocol, data, and code available here https://osf.io/xz9ku/?view_only=a9f7c2d841684cb7a3616f567db273fa. We searched imaging journals Ovid MEDLINE from 1972 to 2021. Text mining using Python script was used to collect metadata: journal, publication year, title, abstract, and P-values from abstracts. One P-value was randomly sampled per abstract. We assessed for evidence of p-hacking using a p-curve, by evaluating for a concentration of P-values just below .05. We conducted a one-tailed binomial test (α = .05 level of significance) to assess whether there were more P-values falling in the upper range (e.g., .045 < P < .05) than in the lower range (e.g., .04 < P < .045). To assess variation in results introduced by our random sampling of a single P-value per abstract, we repeated the random sampling process 1000 times and pooled results across the samples. Analysis was done (divided into 10-year periods) to determine if p-hacking practices evolved over time.

RESULTS

Our search of 136 journals identified 967,981 abstracts. Text mining identified 293,687 P-values, and a total of 4105 randomly sampled P-values were included in the p-hacking analysis. The number of journals and abstracts that were included in the analysis as a fraction and percentage of the total number was, respectively, 108/136 (80%) and 4105/967,981 (.4%). P-values did not concentrate just under .05; in fact, there were more P-values falling in the lower range (e.g., .04 < P < .045) than falling just below .05 (e.g., .045 < P < .05), indicating lack of evidence for p-hacking. Time trend analysis did not identify p-hacking in any of the five 10-year periods.

CONCLUSION

We did not identify evidence of p-hacking in abstracts published in over 100 imaging journals since 1972. These analyses cannot detect all forms of p-hacking, and other forms of bias may exist in imaging research such as publication bias and selective outcome reporting.

How to Critically Appraise and Interpret Systematic Reviews and Meta-Analyses of Diagnostic Accuracy: A User Guide

Systematic reviews of diagnostic accuracy studies can provide the best available evidence to inform decisions regarding the use of a diagnostic test. In this guide, the authors provide a practical approach for clinicians to appraise diagnostic accuracy systematic reviews and apply their results to patient care. The first step is to identify an appropriate systematic review with a research question matching the clinical scenario. The user should evaluate the rigor of the review methods to evaluate its credibility (Did the review use clearly defined eligibility criteria, a comprehensive search strategy, structured data collection, risk of bias and applicability appraisal, and appropriate meta-analysis methods?). If the review is credible, the next step is to decide whether the diagnostic performance is adequate for clinical use (Do sensitivity and specificity estimates exceed the threshold that makes them useful in clinical practice? Are these estimates sufficiently precise? Is variability in the estimates of diagnostic accuracy across studies explained?). Diagnostic accuracy systematic reviews that are judged to be credible and provide diagnostic accuracy estimates with sufficient certainty and relevance are the most useful to inform patient care. This review discusses comparative, noncomparative, and emerging approaches to systematic reviews of diagnostic accuracy using a clinical scenario and examples based on recent publications.

Diagnostic Accuracy of MRI for Solid Renal Masses: A Systematic Review and Meta-analysis

BACKGROUND

Biparametric (bp)-MRI and multiparametric (mp)-MRI may improve the diagnostic accuracy of renal mass histology.

PURPOSE

To evaluate the available evidence on the diagnostic accuracy of bp-MRI and mp-MRI for solid renal masses in differentiating malignant from benign, aggressive from indolent, and clear cell renal cell carcinoma (ccRCC) from other histology.

STUDY TYPE

Systematic review.

POPULATION

MEDLINE, EMBASE, and CENTRAL up to January 11, 2022 were searched.

FIELD STRENGTH/SEQUENCE

1.5 or 3 Tesla.

ASSESSMENT

Eligible studies evaluated the accuracy of MRI (with at least two sequences: T2, T1, dynamic contrast and diffusion-weighted imaging) for diagnosis of solid renal masses in adult patients, using histology as reference standard. Risk of bias and applicability were assessed using QUADAS-2.

STATISTICAL TESTS

Meta-analysis using a bivariate logitnormal random effects model.

RESULTS

We included 10 studies (1239 masses from approximately 1200 patients). The risk of bias was high in three studies, unclear in five studies and low in two studies. The diagnostic accuracy of malignant (vs. benign) masses was assessed in five studies (64% [179/281] malignant). The summary estimate of sensitivity was 95% (95% confidence interval [CI]: 77%-99%), and specificity was 63% (95% CI: 46%-77%). No study assessed aggressive (vs. indolent) masses. The diagnostic accuracy of ccRCC (vs. other subtypes) was evaluated in six studies (47% [455/971] ccRCC): the summary estimate of sensitivity was 85% (95% CI: 77%-90%) and specificity was 77% (95% CI: 73%-81%).

DATA CONCLUSION

Our study reveals deficits in the available evidence on MRI for diagnosis of renal mass histology. The number of studies was limited, at unclear/high risk of bias, with heterogeneous definitions of solid masses, imaging techniques, diagnostic criteria, and outcome measures.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

Prognostic Value of Cardiac MRI and FDG PET in Cardiac Sarcoidosis: A Systematic Review and Meta-Analysis

Background There is no consensus regarding the relative prognostic value of cardiac MRI and fluorodeoxyglucose (FDG) PET in cardiac sarcoidosis. Purpose To perform a systematic review and meta-analysis of the prognostic value of cardiac MRI and FDG PET for major adverse cardiac events (MACE) in cardiac sarcoidosis. Materials and Methods In this systematic review, MEDLINE, Ovid Epub, CENTRAL, Embase, Emcare, and Scopus were searched from inception until January 2022. Studies that evaluated the prognostic value of cardiac MRI or FDG PET in adults with cardiac sarcoidosis were included. The primary outcome of MACE was assessed as a composite including death, ventricular arrhythmia, and heart failure hospitalization. Summary metrics were obtained using random-effects meta-analysis. Meta-regression was used to assess covariates. Risk of bias was assessed using the Quality in Prognostic Studies, or QUIPS, tool. Results Thirty-seven studies were included (3489 patients with mean follow-up of 3.1 years ± 1.5 [SD]); 29 studies evaluated MRI (2931 patients) and 17 evaluated FDG PET (1243 patients). Five studies directly compared MRI and PET in the same patients (276 patients). Left ventricular late gadolinium enhancement (LGE) at MRI and FDG uptake at PET were both predictive of MACE (odds ratio [OR], 8.0 [95% CI: 4.3, 15.0] [P < .001] and 2.1 [95% CI: 1.4, 3.2] [P < .001], respectively). At meta-regression, results varied by modality (P = .006). LGE (OR, 10.4 [95% CI: 3.5, 30.5]; P < .001) was also predictive of MACE when restricted to studies with direct comparison, whereas FDG uptake (OR, 1.9 [95% CI: 0.82, 4.4]; P = .13) was not. Right ventricular LGE and FDG uptake were also associated with MACE (OR, 13.1 [95% CI: 5.2, 33] [P < .001] and 4.1 [95% CI: 1.9, 8.9] [P < .001], respectively). Thirty-two studies were at risk for bias. Conclusion Left and right ventricular late gadolinium enhancement at cardiac MRI and fluorodeoxyglucose uptake at PET were predictive of major adverse cardiac events in cardiac sarcoidosis. Limitations include few studies with direct comparison and risk of bias. Systematic review registration no. CRD42021214776 (PROSPERO) © RSNA, 2023 Supplemental material is available for this article.

Comparing Survival Outcomes of Patients With LI-RADS-M Hepatocellular Carcinomas and Intrahepatic Cholangiocarcinomas

BACKGROUND

There is a sparsity of data evaluating outcomes of patients with Liver Imaging Reporting and Data System (LI-RADS) (LR)-M lesions.

PURPOSE

To compare overall survival (OS) and progression free survival (PFS) between hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (iCCA) meeting LR-M criteria and to evaluate factors associated with prognosis.

STUDY TYPE

Retrospective.

SUBJECTS

Patients at risk for HCC with at least one LR-M lesion with histologic diagnosis, from 8 academic centers, yielding 120 patients with 120 LR-M lesions (84 men [mean age 62 years] and 36 women [mean age 66 years]).

FIELD STRENGTH/SEQUENCE

A 1.5 and 3.0 T/3D T₁ -weighted gradient echo, T₂ -weighted fast spin-echo.

ASSESSMENT

The imaging categorization of each lesion as LR-M was made clinically by a single radiologist at each site and patient outcome measures were collected.

STATISTICAL TESTS

OS, PFS, and potential independent predictors were evaluated by Kaplan-Meier method, log-rank test, and Cox proportional hazard model. A P value of <0.05 was considered significant.

RESULTS

A total of 120 patients with 120 LR-M lesions were included; on histology 65 were HCC and 55 were iCCA. There was similar median OS for patients with LR-M HCC compared to patients with iCCA (738 days vs. 769 days, P = 0.576). There were no significant differences between patients with HCC and iCCA in terms of sex (47:18 vs. 37:18, P = 0.549), age (63.0 ± 8.4 vs. 63.4 ± 7.8, P = 0.847), etiology of liver disease (P = 0.202), presence of cirrhosis (100% vs. 100%, P = 1.000), tumor size (4.73 ± 3.28 vs. 4.75 ± 2.58, P = 0.980), method of lesion histologic diagnosis (P = 0.646), and proportion of patients who underwent locoregional therapy (60.0% vs. 38.2%, P = 0.100) or surgery (134.8 ± 165.5 vs. 142.5 ± 205.6, P = 0.913). Using multivariable analysis, nonsurgical compared to surgical management (HR, 4.58), larger tumor size (HR, 1.19), and higher MELD score (HR, 1.12) were independently associated with worse OS.

DATA CONCLUSION

There was similar OS in patients with LR-M HCC and LR-M iCCA, suggesting that LR-M imaging features may more closely reflect patient outcomes than histology.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 5.

Comparison of image quality and depiction of microscopic fat at 2-D and 3-D T1-Weighted (T1W) chemical shift (dual-echo) MRI for evaluation of adrenal adenomas

OBJECTIVE

To compare image quality and detection of microscopic fat in adrenal adenomas imaged with 2-D and 3-D chemical shift imaging (CSI) and, to derive parameters which best match 2-D and 3-D-CSI.

METHODS

This two-phase, retrospective, and phantom + prospective study was IRB approved. First, a retrospective assessment of 50 consecutive adrenal adenomas imaged at 1.5 T with 2-D (TR minimum, Flip Angle [FA] 70°, TE 2.2/4.4 ms.) and 3-D (TR minimum, FA 10°, TE 2.2/4.4 ms.] CSI was performed. Second, phantom (varied fat: water concentration) experiments guided a prospective assessment of 12 consecutive adrenal adenomas imaged at 1.5 T with 3-D CSI (FA 10°, 18°). Two blinded radiologists independently evaluated: image quality, signal intensity (SI) cancellation (5-point Likert scale), and CSI-index ([SI.In.Phase-SI.Opposed.Phase/SI.In.Phase]*100).

RESULTS

2-D-CSI yielded higher image quality (p < 0.001) and, subjectively (p < 0.001) and quantitatively (p < 0.001) had more SI cancellation from microscopic fat. Proportion of adenomas with no detectable microscopic fat (3-D; 26-36% subjectively, 18-24% quantitatively [CSI-index < 16.5%] versus 2-D; 20-22% subjectively, 6-8% quantitatively) differed (p = 0.008-0.08 subjectively, 0.008-0.03 quantitatively) by CSI technique. Phantom experiments indicated 18°FA 3-D-CSI compared favorably to 70° 2-D-CSI for fat detection between 5% and 50%. In vivo, there was no differences in subjective or quantitative SI cancellation comparing 18°3D-CSI and 2D-CSI (p = 0.16-0.56 and 0.73-0.60). Greater SI cancellation occurred with 18°3D compared to 10°3D-CSI evaluated subjectively (p = 0.003-0.01).

CONCLUSION

2-D CSI has subjectively higher image quality and shows more signal intensity loss from microscopic fat in adrenal adenomas compared to 10° flip angle 3-D-CSI. Increasing the 3-D flip angle to 18° more closely matches depiction of microscopic fat to 2-D-CSI at 1.5 T.

Checklist for Artificial Intelligence in Medical Imaging Reporting Adherence in Peer-Reviewed and Preprint Manuscripts With the Highest Altmetric Attention Scores: A Meta-Research Study

Purpose: To establish reporting adherence to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) in diagnostic accuracy AI studies with the highest Altmetric Attention Scores (AAS), and to compare completeness of reporting between peer-reviewed manuscripts and preprints. Methods: MEDLINE, EMBASE, arXiv, bioRxiv, and medRxiv were retrospectively searched for 100 diagnostic accuracy medical imaging AI studies in peer-reviewed journals and preprint platforms with the highest AAS since the release of CLAIM to June 24, 2021. Studies were evaluated for adherence to the 42-item CLAIM checklist with comparison between peer-reviewed manuscripts and preprints. The impact of additional factors was explored including body region, models on COVID-19 diagnosis and journal impact factor. Results: Median CLAIM adherence was 48% (20/42). The median CLAIM score of manuscripts published in peer-reviewed journals was higher than preprints, 57% (24/42) vs 40% (16/42), P < .0001. Chest radiology was the body region with the least complete reporting ( P = .0352), with manuscripts on COVID-19 less complete than others (43% vs 54%, P = .0002). For studies published in peer-reviewed journals with an impact factor, the CLAIM score correlated with impact factor, rho = 0.43, P = .0040. Completeness of reporting based on CLAIM score had a positive correlation with a study’s AAS, rho = 0.68, P < .0001. Conclusions: Overall reporting adherence to CLAIM is low in imaging diagnostic accuracy AI studies with the highest AAS, with preprints reporting fewer study details than peer-reviewed manuscripts. Improved CLAIM adherence could promote adoption of AI into clinical practice and facilitate investigators building upon prior works.

Modifying LI-RADS on Gadoxetate Disodium-Enhanced MRI: A Secondary Analysis of a Prospective Observational Study

BACKGROUND

The Liver Imaging Reporting and Data System (LI-RADS) is widely used for diagnosing hepatocellular carcinoma (HCC), however, with unsatisfactory sensitivity, complex ancillary features, and inadequate integration with gadoxetate disodium (EOB)-enhanced MRI.

PURPOSE

To modify LI-RADS (mLI-RADS) on EOB-MRI.

STUDY TYPE

Secondary analysis of a prospective observational study.

POPULATION

Between July 2015 and September 2018, 224 consecutive high-risk patients (median age, 51 years; range, 26-83; 180 men; training/testing sets: 169/55 patients) with 742 (median size, 13 mm; interquartile range, 7-27; 498 HCCs) LR-3/4/5 observations.

FIELD STRENGTH/SEQUENCE

3.0 T T₂ -weighted fast spin-echo, diffusion-weighted spin-echo based echo-planar, and 3D T₁ -weighted gradient echo sequences.

ASSESSMENT

Three radiologists (with 5, 5, and 10 years of experience in liver MR imaging, respectively) blinded to the reference standard (histopathology or imaging follow-up) reviewed all MR images independently. In the training set, the optimal LI-RADS version 2018 (v2018) features selected by Random Forest analysis were used to develop mLI-RADS via decision tree analysis.

STATISTICAL TESTS

In an independent testing set, diagnostic performances of mLI-RADS, LI-RADS v2018, and the Korean Liver Cancer Association (KLCA) guidelines were computed using a generalized estimating equation model and compared with McNemar's test. A two-tailed P < 0.05 was statistically significant.

RESULTS

Five features (nonperipheral "washout," restricted diffusion, nonrim arterial phase hyperenhancement [APHE], mild-moderate T2 hyperintensity, and transitional phase hypointensity) constituted mLI-RADS, and mLR-5 was nonperipheral washout coupled with either nonrim APHE or restricted diffusion. In the testing set, mLI-RADS was significantly more sensitive (72%) and accurate (80%) than LI-RADS v2018 (sensitivity, 61%; accuracy 74%; both P < 0.001) and the KLCA guidelines (sensitivity, 64%; accuracy 74%; both P < 0.001), without sacrificing positive predictive value (mLI-RADS, 94%; LI-RADS v2018, 94%; KLCA guidelines, 92%).

DATA CONCLUSION

In high-risk patients, the EOB-MRI-based mLI-RADS was simpler and more sensitive for HCC than LI-RADS v2018 while maintaining high positive predictive value.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 2.

Canadian Association of Radiologists Guidance on Contrast Associated Acute Kidney Injury

Iodinated contrast media (ICM) is one of the most frequently administered pharmaceuticals. In Canada, over 5.4 million computed tomography (CT) examinations were performed in 2019, of which 50% were contrast enhanced. Acute kidney injury (AKI) occurring after ICM administration was historically considered a common iatrogenic complication which was managed by screening patients, prophylactic strategies, and follow up evaluation of renal function. The Canadian Association of Radiologists (CAR) initially published guidelines on the prevention of contrast induced nephropathy in 2007, with an update in 2012. However, new developments in the field have led to the availability of safer contrast agents and changes in clinical practice, prompting a complete revision of the earlier recommendations. This revised guidance document was developed by a multidisciplinary CAR Working Group of radiologists and nephrologists, and summarizes changes in practice related to contrast administration, screening, and risk stratification since the last guideline. It reviews the scientific evidence for contrast associated AKI and provides consensus-based recommendations for its prevention and management in the Canadian healthcare context. This article is a joint publication in the Canadian Association of Radiologists Journal and Canadian Journal of Kidney Health and Disease, intended to inform both communities of practice.

Canadian Association of Radiologists Guidance on Contrast-Associated Acute Kidney Injury

Purpose

Iodinated contrast media is one of the most frequently administered pharmaceuticals. In Canada, over 5.4 million computed tomography (CT) examinations were performed in 2019, of which 50% were contrast enhanced. Acute kidney injury (AKI) occurring after iodinated contrast administration was historically considered a common iatrogenic complication which was managed by screening patients, prophylactic strategies, and follow-up evaluation of renal function. The Canadian Association of Radiologists (CAR) initially published guidelines on the prevention of contrast induced nephropathy in 2007, with an update in 2012. However, new developments in the field have led to the availability of safer contrast agents and changes in clinical practice, prompting a complete revision of the earlier recommendations.

Information sources

Published literature, including clinical trials, retrospective cohort series, review articles, and case reports, along with expert opinions from radiologists and nephrologists across Canada.

Methods

The leadership of the CAR formed a working group of radiologists and nephrologists with expertise in contrast administration and patient management related to contrast-associated AKI. We conducted a comprehensive review of the published literature to evaluate the evidence about contrast as a cause of AKI, and to inform evidence-based recommendations. Based on the available literature, the working group developed consensus recommendations.

Key Findings

The working group developed 21 recommendations, on screening, choice of iodinated contrast media, prophylaxis, medication considerations, and post contrast administration management. The key changes from the 2012 guidelines were (1) Simplification of screening to a simple questionnaire, and not delaying emergent examinations due to a need for creatinine measurements (2) Prophylaxis considerations only for patients with estimated glomerular filtration rate (eGFR) less than 30 mL/min/1.73 m2 (3) Not recommending the routine discontinuation of any drugs to decrease risk of AKI, except metformin when eGFR is less than 30 mL/min/1.73 m2 and (4) Not requiring routine follow up serum creatinine measurements post iodinated contrast administration.

Limitations

We did not conduct a formal systematic review or meta-analysis. We did not evaluate our specific suggestions in the clinical environment.

Implications

Given the importance of iodinated contrast media use in diagnosis and management, and the low risk of AKI after contrast use, these guidelines aim to streamline the processes around iodinated contrast use in most clinical settings. As newer evidence arises that may change or add to the recommendations provided, the working group will revise these guidelines.

Diagnostic Accuracy of Cardiac MRI versus FDG PET for Cardiac Sarcoidosis: A Systematic Review and Meta-Analysis

Background There is limited consensus regarding the relative diagnostic performance of cardiac MRI and fluorodeoxyglucose (FDG) PET for cardiac sarcoidosis. Purpose To perform a systematic review and meta-analysis to compare the diagnostic accuracy of cardiac MRI and FDG PET for cardiac sarcoidosis. Materials and Methods Medline, Ovid Epub, Cochrane Central Register of Controlled Trials, Embase, Emcare, and Scopus were searched from inception until January 2022. Inclusion criteria included studies that evaluated the diagnostic accuracy of cardiac MRI or FDG PET for cardiac sarcoidosis in adults. Data were independently extracted by two investigators. Summary accuracy metrics were obtained by using bivariate random-effects meta-analysis. Meta-regression was used to assess the effect of different covariates. Risk of bias was assessed using the Quality Assessment Tool for Diagnostic Accuracy Studies-2 tool. The study protocol was registered a priori in the International Prospective Register of Systematic Reviews (Prospero protocol CRD42021214776). Results Thirty-three studies were included (1997 patients, 687 with cardiac sarcoidosis); 17 studies evaluated cardiac MRI (1031 patients) and 26 evaluated FDG PET (1363 patients). Six studies directly compared cardiac MRI and PET in the same patients (303 patients). Cardiac MRI had higher sensitivity than FDG PET (95% vs 84%; P = .002), with no difference in specificity (85% vs 82%; P = .85). In a sensitivity analysis restricted to studies with direct comparison, point estimates were similar to those from the overall analysis: cardiac MRI and FDG PET had sensitivities of 92% and 81% and specificities of 72% and 82%, respectively. Covariate analysis demonstrated that sensitivity for FDG PET was highest with quantitative versus qualitative evaluation (93% vs 76%; P = .01), whereas sensitivity for MRI was highest with inclusion of T2 imaging (99% vs 88%; P = .001). Thirty studies were at risk of bias. Conclusion Cardiac MRI had higher sensitivity than fluorodeoxyglucose PET for diagnosis of cardiac sarcoidosis but similar specificity. Limitations, including risk of bias and few studies with direct comparison, necessitate additional study. © RSNA, 2022 Online supplemental material is available for this article.

A Metrics-Based Research Salary Award System and Its 9-Year Impact on Publication Productivity

RATIONALE AND OBJECTIVES

Although metrics-based systems may incentivize academic output, no prior studies have evaluated the impact on publication metrics in academic radiology. This study presents a metrics-based system of awarding research protected time, and retrospectively evaluates its 9-year impact on publication productivity and impact factor.

MATERIALS AND METHODS

Based on a metrics-based algorithm to award department funded Research Protected Time (RPT), metrics pre-RPT (2003-2009) and during the RPT period (2010-2018) from an academic radiology department were retrospectively analyzed to test the hypothesis that the RPT program resulted in higher publication productivity and journal impact factor at the departmental level and for faculty members receiving the award. Comparison was made between (1) pre-RPT and RPT periods and (2) during the RPT period, between RPT and non-RPT faculty members, for annual publication productivity normalized to faculty count (Student's t test) and median impact factor (Wilcoxon rank sum test).

RESULTS

For the evaluation period of 2003-2018, 724 unique publications were identified: 15% (107/724) pre-RPT period and 85% (617/724) RPT period. Normalized annual publication productivity was higher during the RPT period compared to the Pre-RPT period (1.2 vs. 0.3, p = 0.002), and within the RPT period, higher among faculty who received RPT vs. non-RPT faculty (3.5 vs. 0.4, p = 0.002). Median impact factor was higher during the RPT period compared to pre-RPT period (2.843 vs. 2.322, p = 0.044), and within the RPT period, higher in RPT vs. non-RPT faculty (3.016 vs. 2.346, p < 0.001).

CONCLUSION

The implementation of a metrics-based system of funded, research protected time, was associated with increased publication productivity and increased impact factor.

Impact of Reference Standard on CT, MRI, and Contrast-enhanced US LI-RADS Diagnosis of Hepatocellular Carcinoma: A Meta-Analysis

See also the editorial by Ronot in this issue. Online supplemental material is available for this article.

Diagnostic accuracy of thoracic imaging modalities for the detection of COVID-19

The ongoing coronavirus disease 2019 (COVID-19) pandemic continues to present diagnostic challenges. The use of thoracic radiography has been studied as a method to improve the diagnostic accuracy of COVID-19. The ‘Living’ Cochrane Systematic Review on the diagnostic accuracy of imaging tests for COVID-19 is continuously updated as new information becomes available for study. In the most recent version, published in March 2021, a meta-analysis was done to determine the pooled sensitivity and specificity of chest X-ray (CXR) and lung ultrasound (LUS) for the diagnosis of COVID-19. CXR gave a sensitivity of 80.6% (95%CI: 69.1-88.6) and a specificity of 71.5% (95%CI: 59.8-80.8). LUS gave a sensitivity rate of 86.4% (95%CI: 72.7-93.9) and specificity of 54.6% (95%CI: 35.3-72.6). These results differed from the findings reported in the recent article in this journal where they cited the previous versions of the study in which a meta-analysis for CXR and LUS could not be performed. Additionally, the article states that COVID-19 could not be distinguished, using chest computed tomography (CT), from other respiratory diseases. However, the latest review version identifies chest CT as having a specificity of 80.0% (95%CI: 74.9-84.3), which is much higher than the previous version which indicated a specificity of 61.1% (95%CI: 42.3-77.1). Therefore, CXR, chest CT and LUS have the potential to be used in conjunction with other methods in the diagnosis of COVID-19.

Data-Driven Modification of the LI-RADS Major Feature System on Gadoxetate Disodium-Enhanced MRI: Toward Better Sensitivity and Simplicity

BACKGROUND

The Liver Imaging Reporting and Data System (LI-RADS) is widely accepted as a reliable diagnostic scheme for hepatocellular carcinoma (HCC) in at-risk patients. However, its application is hampered by substantial complexity and suboptimal diagnostic sensitivity.

PURPOSE

To propose data-driven modifications to the LI-RADS version 2018 (v2018) major feature system (rLI-RADS) on gadoxetate disodium (EOB)-enhanced magnetic resonance imaging (MRI) to improve sensitivity and simplicity while maintaining high positive predictive value (PPV) for detecting HCC.

STUDY TYPE

Retrospective.

POPULATION

Two hundred and twenty-four consecutive at-risk patients (training dataset: 169, independent testing dataset: 55) with 742 LR-3 to LR-5 liver observations (HCC: N = 498 [67%]) were analyzed from a prospective observational registry collected between July 2015 and September 2018.

FIELD STRENGTH/SEQUENCE

3.0 T/T2-weighted fast spin-echo, diffusion-weighted spin-echo based echo-planar and three-dimensional (3D) T1-weighted gradient echo sequences.

ASSESSMENT

All images were evaluated by three independent abdominal radiologists who were blinded to all clinical, pathological, and follow-up information. Composite reference standards of either histopathology or imaging follow-up were used.

STATISTICAL TESTS

In the training dataset, LI-RADS v2018 major features were used to develop rLI-RADS based on their associated PPV for HCC. In an independent testing set, diagnostic performances of LI-RADS v2018 and rLI-RADS were computed using a generalized estimating equation model and compared with McNemar's test. A P value <0.05 was considered statistically significant.

RESULTS

The median (interquartile range) size of liver observations was 13 mm (7-27 mm). The diagnostic table for rLI-RADS encompassed 9 cells, as opposed to 16 cells for LI-RADS v2018. In the testing set, compared to LI-RADS v2018, rLI-RADS category 5 demonstrated a significantly superior sensitivity (76% vs. 61%) while maintaining comparably high PPV (92.5% vs. 94.1%, P = 0.126).

DATA CONCLUSION

Compared with LI-RADS v2018, rLI-RADS demonstrated improved simplicity and significantly superior diagnostic sensitivity for HCC in at-risk patients.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY STAGE: 2.

CT/MRI and CEUS LI-RADS Major Features Association with Hepatocellular Carcinoma: Individual Patient Data Meta-Analysis

Background The Liver Imaging Reporting and Data System (LI-RADS) assigns a risk category for hepatocellular carcinoma (HCC) to imaging observations. Establishing the contributions of major features can inform the diagnostic algorithm. Purpose To perform a systematic review and individual patient data meta-analysis to establish the probability of HCC for each LI-RADS major feature using CT/MRI and contrast-enhanced US (CEUS) LI-RADS in patients at high risk for HCC. Materials and Methods Multiple databases (MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and Scopus) were searched for studies from January 2014 to September 2019 that evaluated the accuracy of CT, MRI, and CEUS for HCC detection using LI-RADS (CT/MRI LI-RADS, versions 2014, 2017, and 2018; CEUS LI-RADS, versions 2016 and 2017). Data were centralized. Clustering was addressed at the study and patient levels using mixed models. Adjusted odds ratios (ORs) with 95% CIs were determined for each major feature using multivariable stepwise logistic regression. Risk of bias was assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) (PROSPERO protocol: CRD42020164486). Results A total of 32 studies were included, with 1170 CT observations, 3341 MRI observations, and 853 CEUS observations. At multivariable analysis of CT/MRI LI-RADS, all major features were associated with HCC, except threshold growth (OR, 1.6; 95% CI: 0.7, 3.6; P = .07). Nonperipheral washout (OR, 13.2; 95% CI: 9.0, 19.2; P = .01) and nonrim arterial phase hyperenhancement (APHE) (OR, 10.3; 95% CI: 6.7, 15.6; P = .01) had stronger associations with HCC than enhancing capsule (OR, 2.4; 95% CI: 1.7, 3.5; P = .03). On CEUS images, APHE (OR, 7.3; 95% CI: 4.6, 11.5; P = .01), late and mild washout (OR, 4.1; 95% CI: 2.6, 6.6; P = .01), and size of at least 20 mm (OR, 1.6; 95% CI: 1.04, 2.5; P = .04) were associated with HCC. Twenty-five studies (78%) had high risk of bias due to reporting ambiguity or study design flaws. Conclusion Most Liver Imaging Reporting and Data System major features had different independent associations with hepatocellular carcinoma; for CT/MRI, arterial phase hyperenhancement and washout had the strongest associations, whereas threshold growth had no association. © RSNA, 2021 Online supplemental material is available for this article.

Association of Accuracy, Conclusions, and Reporting Completeness With Acceptance by Radiology Conferences and Journals

BACKGROUND

Preferential publication of studies with positive findings can lead to overestimation of diagnostic test accuracy (i.e. publication bias). Understanding the contribution of the editorial process to publication bias could inform interventions to optimize the evidence guiding clinical decisions.

PURPOSE/HYPOTHESIS

To evaluate whether accuracy estimates, abstract conclusion positivity, and completeness of abstract reporting are associated with acceptance to radiology conferences and journals.

STUDY TYPE

Meta-research.

POPULATION

Abstracts submitted to radiology conferences (European Society of Gastrointestinal and Abdominal Radiology (ESGAR) and International Society for Magnetic Resonance in Medicine (ISMRM)) from 2008 to 2018 and manuscripts submitted to radiology journals (Radiology, Journal of Magnetic Resonance Imaging [JMRI]) from 2017 to 2018. Primary clinical studies evaluating sensitivity and specificity of a diagnostic imaging test in humans with available editorial decisions were included.

ASSESSMENT

Primary variables (Youden's index [YI > 0.8 vs. <0.8], abstract conclusion positivity [positive vs. neutral/negative], number of reported items on the Standards for Reporting of Diagnostic Accuracy Studies [STARD] for Abstract guideline) and confounding variables (prospective vs. retrospective/unreported, sample size, study duration, interobserver agreement assessment, subspecialty, modality) were extracted.

STATISTICAL TESTS

Multivariable logistic regression to obtain adjusted odds ratio (OR) as a measure of the association between the primary variables and acceptance by radiology conferences and journals; 95% confidence intervals (CIs) and P-values were obtained; the threshold for statistical significance was P < 0.05.

RESULTS

A total of 1000 conference abstracts (500 ESGAR and 500 ISMRM) and 1000 journal manuscripts (505 Radiology and 495 JMRI) were included. Conference abstract acceptance was not significantly associated with YI (adjusted OR = 0.97 for YI > 0.8; CI = 0.70-1.35), conclusion positivity (OR = 1.21 for positive conclusions; CI = 0.75-1.90) or STARD for Abstracts adherence (OR = 0.96 per unit increase in reported items; CI = 0.82-1.18). Manuscripts with positive abstract conclusions were less likely to be accepted by radiology journals (OR = 0.45; CI = 0.24-0.86), while YI (OR = 0.85; CI = 0.56-1.29) and STARD for Abstracts adherence (OR = 1.06; CI = 0.87-1.30) showed no significant association. Positive conclusions were present in 86.7% of submitted conference abstracts and 90.2% of journal manuscripts.

DATA CONCLUSION

Diagnostic test accuracy studies with positive findings were not preferentially accepted by the evaluated radiology conferences or journals.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies

Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.

Utilization of imaging for active surveillance in testicular cancer: Is real-world practice concordant with guidelines?

INTRODUCTION

Imaging is an integral component of active surveillance following orchiectomy for stage 1 non-seminoma (NSGCT) and seminoma germ cell tumors. In this population-based study, we describe use of imaging among patients with early-stage testicular cancer and evaluate whether they are concordant with guideline recommendations.

METHODS

This is a population-based, retrospective cohort study to describe utilization of imaging among all patients with early-stage testicular cancer treated with active surveillance in the Canadian province of Ontario. The Ontario Cancer Registry was linked to electronic records of treatment to identify use of chest and abdomen/pelvis imaging. Data from 2000--2010 were included with followup for up to five years for patients with non-seminoma and 10 years for patients with seminoma. The key outcome of interest was the frequency of imaging at temporal milestones following orchiectomy. Compared to the most contemporaneous guidelines in Ontario, any discordant frequency of imaging was defined as underutilization or overutilization. Substantial under- or overutilization was defined as >1 imaging test less/more than what was recommended during a 12-month period.

RESULTS

The study population included 569 patients with NSGCT (median age 28) and 1107 with seminoma (median age 37). Among patients with NSGCT, adherence with body imaging was low in years 1-3 of surveillance (range 26-37%, predominantly underuse) and higher in years 4-5 (63-67%, predominantly overuse). Adherence with chest imaging was even lower (range 11-34% during years 1-5). Among patients with seminoma, adherence with abdominal and chest imaging was relatively stable and comparable throughout the 10-year followup period (range 23-47% abdomen and 28-47% chest). Multivariable analysis confirmed that underutilization of imaging was more common in recent years. NSGCT histology was associated with underutilization in years 1-2 but overutilization in years 3-5.

CONCLUSIONS

In routine clinical practice, patients with testicular cancer commonly receive imaging discordant to the protocol for active surveillance, with a substantial proportion receiving both under- and overutilization at various times during surveillance followup.

Limited Chest Ultrasound to Replace CXR in Diagnosis of Pneumothorax Post Image-Guided Transthoracic Interventions

PURPOSE

To assess the diagnostic accuracy of limited chest ultrasound in detecting pneumothorax following percutaneous transthoracic needle interventions using chest X-ray (CXR) as the reference standard.

METHODS

With IRB approval, after providing consent, asymptomatic patients after percutaneous transthoracic needle interventions were enrolled to undergo limited chest ultrasound in addition to CXR. A chest Radiologist blinded to the patient's prior imaging performed a bedside ultrasound, scanning only the first 3 anterior intercostal spaces. Pneumothorax diagnosed on CXR was categorized as small or large and on ultrasound as grades 1, 2, or 3 when detected in 1, 2, or 3 intercostal spaces, respectively.

RESULTS

38 patients underwent 36 biopsies (34 lungs, 1 pleura, and 1 mediastinum) and 2 coil localizations. CXR showed pneumothorax in 13 patients. Ultrasound was positive in 10 patients, with 9 true-positives, 1 false-positive, 4 false-negatives, and 24 true-negatives. The false positive results were due to apical subpleural bullae. The false-negative results occurred in 2 small apical and 2 focal pneumothoraces at the needle entry sites. Four pneumothoraces were categorized as large on CXR, all of which were categorized as grade 3 on ultrasound. Sensitivity and specificity of US for detection of pneumothorax of any size were 69.23% (95%CI 38.6%, 90.1%) and 96.0% (95%CI 79.6%, 99.9%), and for detection of large pneumothorax were 100% (95%CI 39.8%, 100%) and 100% (95%CI 89.7%, 100%).

CONCLUSIONS

Results of this prospective study is promising. Limited chest ultrasound could potentially replace CXR in the management of postpercutaneous transthoracic needle intervention patients.

The impact of measuring split kidney function on post-donation kidney function: A retrospective cohort study

Background

Studies have reported agreement between computed tomography (CT) and renography for the determination of split kidney function. However, their correlation with post-donation kidney function remains unclear. We compared CT measurements with renography in assessment of split kidney function (SKF) and their correlations with post-donation kidney function.

Methods

A single-centre, retrospective cohort study of 248 donors from January 1, 2009-July 31, 2019 were assessed. Pearson correlations were used to assess post-donation kidney function with renography and CT-based measurements. Furthermore, we examined high risk groups with SKF difference greater than 10% on renography and donors with post-donation eGFR less than 60 mL/min/1.73m².

Results

62% of donors were women with a mean (standard deviation) pre-donation eGFR 99 (20) and post-donation eGFR 67 (22) mL/min/1.73m² at 31 months of follow-up. Post-donation kidney function was poorly correlated with both CT-based measurements and renography, including the subgroup of donors with post-donation eGFR less than 60 mL/min/1.73m² (r less than 0.4 for all). There was agreement between CT-based measurements and renography for SKF determination (Bland-Altman agreement [bias, 95% limits of agreement] for renography vs: CT volume, 0.76%, -7.60–9.15%; modified ellipsoid,1.01%, -8.38–10.42%; CC dimension, 0.44%, -7.06–7.94); however, CT missed SKF greater than 10% found by renography in 20 out 26 (77%) of donors.

Conclusions

In a single centre study of 248 living donors, we found no correlation between CT or renography and post-donation eGFR. Further research is needed to determine optimal ways to predict remaining kidney function after donation.

Blinding practices during acute point-of-care ultrasound research: the BLIND-US meta-research study

OBJECTIVES

Point-of-care ultrasound (POCUS) diagnostic accuracy research has significant variation in blinding practices. This study characterises the blinding practices during acute POCUS research to determine whether research methodology adequately reflects POCUS use in routine clinical practice.

DESIGN, SETTINGS AND PARTICIPANTS

A search for POCUS diagnostic accuracy studies published in Emergency Medicine, Anaesthesia and Critical Care journals from January 2016 to January 2020 was performed. Studies were included if they were primary diagnostic accuracy studies. The study year, journal impact factor, population, hospital area, body region, study design, blinding of the POCUS interpreter to clinical information, whether the person performing the POCUS scan was the same person interpreting the scan, and whether the study reported incremental diagnostic yield were extracted in duplicate by two authors. Descriptive statistics were provided and prespecified subgroup analysis was performed.

MAIN OUTCOME MEASURES

The primary outcome was the number of studies that blinded the POCUS interpreter to at least some part of the clinical information. Secondary outcomes included whether the person performing the POCUS scan was the same person interpreting it and whether the study reported incremental diagnostic yield.

RESULTS

520 abstracts were screened with 97 studies included. The POCUS interpreter was blinded to clinical information in 37 studies (38.1%), not blinded in 34 studies (35.1%) and not reported in 26 studies (26.8%). The POCUS interpreter was the same person obtaining the images in 72 studies (74.2%), different in 14 studies (14.4%) and not reported in 11 studies (11.3%). Only four studies (4.1%) reported incremental diagnostic yield for POCUS. Inter-rater reliability was moderate (k=0.64). Subgroup analysis based on impact factor, body region, hospital area, patient population and study design did not show significant differences after completing pairwise comparisons.

CONCLUSIONS

Although blinding the POCUS interpreter to clinical information may be done in a perceived attempt to limit bias, this may result in accuracy estimates that do not reflect routine clinical practice. Similarly, having a different clinician perform and interpret the POCUS scan significantly limits generalisability to practice as it does not truly reflect 'point-of-care' ultrasound at all. Reporting incremental diagnostic yield from implementing POCUS into a diagnostic pathway better reflects the value of POCUS; however, this methodology was infrequently used.

TRIAL REGISTRATION NUMBER

The study protocol was registered on Open Science Framework (https://osf.io/h5fe7/).

Tweeting Bias in Diagnostic Test Accuracy Research: Does Title or Conclusion Positivity Influence Dissemination?

PURPOSE

To examine if tweeting bias exists within imaging literature by determining if diagnostic test accuracy (DTA) studies with positive titles or conclusions are tweeted more than non-positive studies.

METHODS

DTA studies published between October 2011 to April 2016 were included. Positivity of titles and conclusions were assessed independently and in duplicate, with disagreements resolved by consensus. A negative binomial regression analysis controlling for confounding variables was performed to assess the relationship between title or conclusion positivity and tweets an article received in the 100 days post-publication.

RESULTS

354 DTA studies were included. Twenty-four (7%) titles and 300 (85%) conclusions were positive (or positive with qualifier); 1 (0.3%) title and 23 (7%) conclusions were negative; and 329 (93%) titles and 26 (7%) conclusions were neutral. Studies with positive, negative, and neutral titles received a mean of 0.38, 0.00, and 0.45 tweets per study; while those with positive, negative, and neutral conclusions received a mean of 0.44, 0.61, and 0.38 tweets per study. Regression coefficients were -0.05 (SE 0.46) for positive relative to non-positive titles, and -0.09 (SE 0.31) for positive relative to non-positive conclusions. The positivity of the title (P = 0.91) or conclusion (P = 0.76) was not significantly associated with the number of tweets an article received.

CONCLUSIONS

The positivity of the title or conclusion for DTA studies does not influence the amount of tweets it receives suggesting that tweet bias is not present among imaging diagnostic accuracy studies. Study protocol available at https://osf.io/hdk2m/.

Preferred reporting items for journal and conference abstracts of systematic reviews and meta-analyses of diagnostic test accuracy studies (PRISMA-DTA for Abstracts): checklist, explanation, and elaboration

For many users of the biomedical literature, abstracts may be the only source of information about a study. Hence, abstracts should allow readers to evaluate the objectives, key design features, and main results of the study. Several evaluations have shown deficiencies in the reporting of journal and conference abstracts across study designs and research fields, including systematic reviews of diagnostic test accuracy studies. Incomplete reporting compromises the value of research to key stakeholders. The authors of this article have developed a 12 item checklist of preferred reporting items for journal and conference abstracts of systematic reviews and meta-analyses of diagnostic test accuracy studies (PRISMA-DTA for Abstracts). This article presents the checklist, examples of complete reporting, and explanations for each item of PRISMA-DTA for Abstracts.

Effects of implementing Pressure Ulcer Prevention Practice Guidelines (PUPPG) in the prevention of pressure ulcers among hospitalised elderly patients: a systematic review protocol

INTRODUCTION

Pressure ulcers are serious and potentially life-threatening problems across all age groups and across all medical specialties and care settings. The hospitalised elderly population is the most common group to develop pressure ulcers. This study aims to systematically review studies implementing pressure ulcer prevention strategies recommended in the Pressure Ulcer Prevention Practice Guidelines for the prevention of pressure ulcers among hospitalised elderly patients globally.

METHODS AND ANALYSIS

A systematic review of all studies that have assessed the use of pressure ulcer prevention strategies in hospital settings among hospitalised elderly patients shall be conducted. A comprehensive search of all published articles in Medline Ovid, Cumulative Index to Nursing and Allied Health Literature, PubMed, Embase, Cochrane library, Scopus and Web of Science will be done using terms such as pressure ulcers, prevention strategies, elderly patients and hospital. Studies will be screened for eligibility through title, abstract and full text by two independent reviewers. Study quality and risk of bias will be assessed using the Joanna Briggs Institute for Meta-Analysis of Statistics Assessment and Review Instrument. If sufficient data are available, a meta-analysis will be conducted to synthesise the effect size reported as OR with 95% CIs using both fixed and random effect models. I² statistics and visual inspection of the forest plots will be used to assess heterogeneity and identify the potential sources of heterogeneity. Publication bias will be assessed by visual inspections of funnel plots and Egger's test.

ETHICS AND DISSEMINATION

No formal ethical approval or consent is required as no primary data will be collected. We aim to publish the research findings in a peer-reviewed scientific journal to promote knowledge transfer, as well as in conferences, seminars, congresses or symposia in a traditional manner.

PROSPERO REGISTRATION NUMBER

CRD42019129088.

Barriers to reporting guideline adherence in point-of-care ultrasound research: a cross-sectional survey of authors and journal editors

OBJECTIVE

Although the literature supporting the use of point-of-care ultrasound (POCUS) continues to grow, incomplete reporting of primary diagnostic accuracy studies has previously been identified as a barrier to translating research into practice and to performing unbiased systematic reviews. This study assesses POCUS investigator and journal editor attitudes towards barriers to adhering to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 guidelines.

DESIGN, SETTING, PARTICIPANTS

Two separate surveys using a 5-point Likert scale were sent to POCUS study investigators and journal editors to assess for knowledge, attitude and behavioural barriers to the complete reporting of POCUS research. Respondents were identified based on a previous study assessing STARD 2015 adherence for POCUS studies published in emergency medicine, anaesthesia and critical care journals. Responses were anonymously linked to STARD 2015 adherence data from the previous study. Written responses were thematically grouped into the following categories: knowledge, attitude and behavioural barriers to quality reporting, or other. Likert response items are reported as median with IQRs.

MAIN OUTCOME MEASURES

The primary outcome was the median Likert score for the investigator and editor surveys assessing knowledge, attitude and behavioural beliefs about barriers to adhering to the STARD 2015 guidelines.

RESULTS

The investigator survey response rate was 18/69 (26%) and the editor response rate was 5/21 (24%). Most investigator respondents were emergency medicine practitioners (13/21, 62%). Two-thirds of investigators were aware of the STARD 2015 guidelines (12/18, 67%) and overall agreed that incomplete reporting limits generalisability and the ability to detect risk of bias (median 4 (4, 5)). Investigators felt that the STARD 2015 guidelines were useful, easy to find and easy to use (median 4 (4, 4.25); median 4 (4, 4.25) and median 4 (3, 4), respectively). There was a shared opinion held by investigators and editors that the peer review process be primarily responsible for ensuring complete research reporting (median 4 (3, 4) and median 4 (3.75, 4), respectively). Three of 18 authors (17%) felt that the English publication language of STARD 2015 was a barrier to adherence.

CONCLUSIONS

Although investigators and editors recognise the importance of completely reported research, reporting quality is still a core issue for POCUS research. The shared opinion held by investigators and editors that the peer review process be primarily responsible for reporting quality is potentially problematic; we view completely reported research as an integral part of the research process that investigators are responsible for, with the peer review process serving as another additional layer of quality control. Endorsement of reporting guidelines by journals, auditing reporting guideline adherence during the peer review process and translation of STARD 2015 guidelines into additional languages may improve reporting completeness for the acute POCUS literature.

TRIAL REGISTRATION NUMBER

Open Science Framework Registry (https://osf.io/5pzxs/).

Diagnostic Radiology Residency Assessment Tools: A Scoping Review

PURPOSE

The multifaceted nature of learning in diagnostic radiology residency requires a variety of assessment methods. However, the scope and quality of assessment tools has not been formally examined. A scoping review was performed to identify assessment tools available for radiology resident training and to evaluate the validity of these tools.

METHODS

A literature search was conducted through multiple databases and on-line resources. Inclusion criteria were defined as any tool used in assessment of radiology resident competence. Data regarding residents, evaluators and specifics of each tool was extracted. Each tool was subjected through a validation process with a customized rating scale using the 5 categories of validity: content, response process, internal structure, relations to other variables, and consequences.

RESULTS

The initial search returned 447 articles; 35 were included. The most evaluated competency being overall knowledge (31%), most common published journal was Academic Radiology (24%); evaluations were most commonly set in the United States (57%). In terms of validation, we found low adherence to modern integrated validity, with 34% of studies including a definition of validity. When specifically examining the 5 domains of validation evidence presented, most were either absent or of low rigor (70%). Only one study presented a modern definition of validation (3%, 1/35).

CONCLUSION

We identified 35 evaluation tools covering a variety of competency areas. However, few of these tools have been validated. Development of new validated assessment tools or validation of existing tools is essential for the ongoing transition to a competency-based curriculum.