How high is the inter-observer reproducibility in the LIRADS reporting system?

Systematic training of LI-RADS CT v2018 improves interobserver agreements and performances in LR categorization for focal liver lesions

AIM

To retrospectively explored whether systematic training in the use of Liver Imaging Reporting and Data System (LI-RADS) v2018 on computed tomography (CT) can improve the interobserver agreements and performances in LR categorization for focal liver lesions (FLLs) among different radiologists.

MATERIALS AND METHODS

A total of 18 visiting radiologists and the liver multiphase CT images of 70 hepatic observations in 63 patients at high risk of HCC were included in this study. The LI-RADS v2018 training procedure included three thematic lectures, with an interval of 1 month. After each seminar, the radiologists had 1 month to adopt the algorithm into their daily work. The interobserver agreements and performances in LR categorization for FLLs among the radiologists before and after training were compared.

RESULTS

After training, the interobserver agreements in classifying the LR categories for all radiologists were significantly increased for most LR categories (P < 0.001), except for LR-1 (P = 0.053). After systematic training, the areas under the curve (AUCs) for LR categorization performance for all participants were significantly increased for most LR categories (P < 0.001), except for LR-1 (P = 0.062).

CONCLUSION

Systematic training in the use of the LI-RADS can improve the interobserver agreements and performances in LR categorization for FLLs among radiologists with different levels of experience.

CT texture analysis and node-RADS CT score of mediastinal lymph nodes - diagnostic performance in lung cancer patients

BACKGROUND

Texture analysis derived from computed tomography (CT) can provide clinically relevant imaging biomarkers. Node-RADS is a recently proposed classification to categorize lymph nodes in radiological images. The present study sought to investigate the diagnostic abilities of CT texture analysis and Node-RADS to discriminate benign from malignant mediastinal lymph nodes in patients with lung cancer.

METHODS

Ninety-one patients (n = 32 females, 35%) with a mean age of 64.8 ± 10.8 years were included in this retrospective study. Texture analysis was performed using the free available Mazda software. All lymph nodes were scored accordingly to the Node-RADS classification. All primary tumors and all investigated mediastinal lymph nodes were histopathologically confirmed during clinical workup.

RESULTS

In discrimination analysis, Node-RADS score showed statistically significant differences between N0 and N1-3 (p < 0.001). Multiple texture features were different between benign and malignant lymph nodes: S(1,0)AngScMom, S(1,0)SumEntrp, S(1,0)Entropy, S(0,1)SumAverg. Correlation analysis revealed positive associations between the texture features with Node-RADS score: S(4,0)Entropy (r = 0.72, p < 0.001), S(3,0) Entropy (r = 0.72, p < 0.001), S(2,2)Entropy (r = 0.72, p < 0.001).

CONCLUSIONS

Several texture features and Node-RADS derived from CT were associated with the malignancy of mediastinal lymph nodes and might therefore be helpful for discrimination purposes. Both of the two quantitative assessments could be translated and used in clinical routine.

Interrater reliability and agreement of the liver imaging reporting and data system (LI-RADS) v2018 for the evaluation of hepatic lesions

Purpose

The liver imaging reporting and data system (LI-RADS) is a structured reporting system that categorizes hepatic observations according to major imaging features and lesion size, with an optional ancillary features contribution. This study aimed to evaluate inter-reader agreement of dynamic magnetic resonance imaging (MRI) using LI-RADS v2018 lexicon.

Material and methods

Forty-nine patients with 69 hepatic observations were included in our study. The major and ancillary features of each hepatic observation were evaluated by 2 radiologists using LI-RADS v2018, and the interreader agreement was allocated.

Results

The inter-reader agreement of major LI-RADS features was substantial; κ of non-rim arterial hyperenhancement, non-peripheral washout appearance, and enhancing capsule was 0.796, 0.799, and 0.772 (p < 0.001), respectively. The agreement of the final LI-RADS category was substantial with κ = 0.651 (p < 0.001), and weighted κ = 0.786 (p < 0.001). The inter-reader agreement of the ancillary features was substantial to almost perfect (k range from 0.718 to 1; p < 0.001). An almost perfect correlation was noted for the hepatic lesion size measurement with ICC = 0.977 (p < 0.001).

Conclusions

The major and ancillary features of the LI-RADS v2018, as well as the final category and lesions size, have substantial to almost perfect inter-reader agreement.

Optimizing diagnostic imaging data using LI-RADS and the Likert scale in patients with hepatocellular carcinoma

Purpose

The study aimed to compare the diagnostic performance of the Liver Imaging Reporting and Data System (LI-RADS), which incorporates fixed criteria, and the Likert scale (LS), which mainly depends on an overall impression in liver lesion diagnosis.

Material and methods

Diagnostic data of 110 hepatic nodules in 103 high-risk patients for hepatocellular carcinoma (HCC) were included. Data including diameter, arterial hyperenhancement, washout, and capsule were reviewed by 2 readers using LI-RADS and LS (range, score 1-5). Inter-reader agreement (IRA), intraclass agreement (ICA), and diagnostic performance were determined by Fleiss, Cohen’s k, and logistic regression, respectively.

Results

There were 53 triphasic enhanced computed tomography (CT) and 50 dynamic magnetic resonance (MR) examinations. Overall, IRA was excellent (k = 0.898). IRA was good for arterial hyperenhancement (k = 0.705), washout (k = 0.763), and capsule (k = 0.771) and excellent for diameter (k = 0.981) and tumour embolus (k = 0.927). Overall, ICA between LI-RADS and LS was fair 0.32; ICA was good for scores of 1 (k = 0.682), fair for scores of 2 (k = 0.36), moderate for scores of 5 (k = 0.52), but no agreement was found for scores of 3 (k = –0.059) and 4 (k = –0.022). LIRADS produced relatively high accuracy (87.3% vs. 80%), relatively low sensitivity (84.3% vs. 98%), and significantly higher specificity (89.83% vs. 64.4%) and positive likelihood ratio (+LR: 8.29 vs. 2.75) compared to LS approach.

Conclusions

LI-RADS revealed higher diagnostic accuracy as compared to LS with statistical proof higher specificity and +LR showing its ability to foretell malignancy in high-risk patients. We recommend the practical application of the LI-RADS system in the detection and treatment response monitoring of patients with HCC.

Does Training in LI-RADS Version 2018 Improve Readers' Agreement with the Expert Consensus and Inter-reader Agreement in MRI Interpretation?

BACKGROUND

The Liver Imaging Reporting and Data System (LI-RADS) was established for noninvasive diagnosis for hepatocellular carcinoma (HCC). However, whether training can improve readers' agreement with the expert consensus and inter-reader agreement for final categories is still unclear.

PURPOSE

To explore training effectiveness on readers' agreement with the expert consensus and inter-reader agreement.

STUDY TYPE

Prospective.

SUBJECTS

Seventy lesions in 61 patients at risk of HCC undergoing liver MRI; 20 visiting scholars.

FIELD STRENGTH/SEQUENCE

1.5 T or 3 T, Dual-echo T₁ WI, Fast spin-echo T₂ WI, SE-EPI DWI, and Dynamic multiphase fast gradient-echo T₁ WI.

ASSESSMENT

Seventy lesions assigned LI-RADS categories of LR1-LR5, LR-M, and LR-TIV by three radiologists in consensus were randomly selected, with 10 cases for each category. The consensus opinion was the standard reference. The third radiologist delivered the training. Twenty readers reviewed images independently and assigned each an LI-RADS category both before and after the training.

STATISTICAL TESTS

Accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, receiver operating characteristic (ROC) analysis, simple and weighted kappa statistics, and Fleiss kappa statistics.

RESULTS

Before and after training: readers' AUC (areas under ROC) for LR-1-LR-5, LR-M, and LR-TIV were 0.898 vs. 0.913, 0.711 vs. 0.876, 0.747 vs. 0.860, 0.724 vs. 0.815, 0.844 vs. 0.895, 0.688 vs. 0.873, and 0.720 vs. 0.948, respectively, and all improved significantly (P < 0.05), except LR-1(P = 0.25). Inter-reader agreement between readers for LR-1-LR-5, LR-M, LR-TIV were 0.725 vs. 0.751, 0.325 vs. 0.607, 0.330 vs. 0.559, 0.284 vs. 0.488, 0.447 vs. 0.648, 0.229 vs. 0.589, and 0.362 vs. 0.852, respectively, and all increased significantly (P < 0.05). For training effectiveness on both AUC and inter-reader agreement, LR-TIV, LR-M, and LR-2 improved most, and LR-1 made the least.

DATA CONCLUSION

This study shows LI-RADS training could improve reader agreement with the expert consensus and inter-reader agreement for final categories.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY STAGE: 2.

Inter-reader reliability of CT Liver Imaging Reporting and Data System according to imaging analysis methodology: a systematic review and meta-analysis

OBJECTIVES

To establish inter-reader reliability of CT Liver Imaging Reporting and Data System (LI-RADS) and explore factors that affect it.

METHODS

MEDLINE and EMBASE databases were searched from January 2014 to March 2020 to identify original articles reporting the inter-reader reliability of CT LI-RADS. The imaging analysis methodology of each study was identified, and pooled intraclass correlation coefficient (ICC) or kappa values (κ) were calculated for lesion size, major features (arterial-phase hyperenhancement [APHE], nonperipheral washout [WO], and enhancing capsule [EC]), and LI-RADS categorization (LR) using random-effects models. Subgroup analyses of pooled κ were performed for the number of readers, average reader experience, differences in reader experience, and LI-RADS version.

RESULTS

In the 12 included studies, the pooled ICC or κ of lesion size, APHE, WO, EC, and LR were 0.99 (0.96-1.00), 0.69 (0.58-0.81), 0.67 (0.53-0.82), 0.65 (0.54-0.76), and 0.70 (0.59-0.82), respectively. The experience and number of readers varied: studies using readers with ≥ 10 years of experience showed significantly higher κ for LR (0.82 vs. 0.45, p = 0.01) than those with < 10 years of reader experience. Studies with multiple readers including inexperienced readers showed significantly lower κ for APHE (0.55 vs. 0.76, p = 0.04) and LR (0.45 vs. 0.79, p = 0.02) than those with all experienced readers.

CONCLUSIONS

CT LI-RADS showed substantial inter-reader reliability for major features and LR. Inter-reader reliability differed significantly according to average reader experience and differences in reader experience. Reported results for inter-reader reliability of CT LI-RADS should be understood with consideration of the imaging analysis methodology.

KEY POINTS

• The CT Liver Imaging Reporting and Data System (LI-RADS) provides substantial inter-reader reliability for three major features and category assignment. • The imaging analysis methodology varied across studies. • The inter-reader reliability of CT LI-RADS differed significantly according to the average reader experience and the difference in reader experience.