Interobserver Agreement and Accuracy in Interpreting mpMRI of the Prostate: a Systematic Review

Developments in Ultrasound-Based Imaging for Prostate Cancer Detection

BACKGROUND

Prostate cancer is a significant health issue worldwide, but methods to screen for and diagnose this disease have significant inherent limitations. Some efforts to address these limitations have involved the use of ultrasound-based imaging methods.

METHODS

This narrative review paper focuses on recent developments in the use of medical imaging, with a focus on ultrasound and related methods, to improve the diagnosis of prostate cancer. These methods include: elastography, contrast-enhanced ultrasound, targeted contrast agents, quantitative ultrasound, multiparametric ultrasound, micro-ultrasound, and photoacoustic imaging.

RESULTS

This paper provides an update on clinically relevant imaging technologies which are in the technical and preclinical literature.

CONCLUSION

Novel methods and their performance are highlighted, including how they address limitations in current clinical care.

Diffusion weighted image-guided transitional zone scoring in the detection of transitional zone prostate cancer: comparison with current PI-RADS v2.1 scoring

PURPOSE

To compare the performance of diffusion-weighted imaging-guided transitional zone (TZ) lesion scoring on T2-weighted imaging (DWI-guided TZ scoring) to conventional PI-RADS TZ scoring.

METHODS

Forty patients carried transition zone prostate cancer (TZPCa), and 40 patients had benign prostatic hyperplasia without TZPCa. A lesion-base, one-to-one correlation between the pathologic mapping sheet and the corresponding MR imaging was conducted by consensus between the genitourinary-specialized radiologist and pathologist. DWI-guided TZ scoring was defined as evaluating the DWI/apparent diffusion coefficient (ADC) images first, identifying the suspicious foci, then correlating the foci with the T2-weighted imaging, and finally assigning the PI-RADS score based on PI-RADS v2.1. Three other radiologists independently recorded the PI-RADS v2.1 scoring for TZ and the DWI-guided TZ scoring, with a time interval of 4 weeks.

RESULTS

When a PI-RADS score of ≥ 3 was considered a positive lesion, the specificity, PPV, NPV and sensitivity between the DWI-guided TZ scoring and conventional PI-RADS TZ scoring were 0.896 vs. 0.542 (p < .001), 0.764 vs. 0.439 (p < .001), 0.853 vs. 0.759 (p = .001), and 0.687 vs. 0.676 (p = .836), respectively. When PI-RADS scores ≥ 4 was considered cancer-positive, the specificity and PPV were also higher when applying DWI-guided TZ scoring (0.986 vs. 0.944, p = .007; 0.943 vs. 0.810, p = .009, respectively); however, the sensitivity and NPV were not statistically different (0.468 vs. 0.468, p = .998; 0.785 vs. 0.776, p = .537, respectively). The interobserver agreement presented as κ-value was higher in DWI-guided TZ scoring (0.584) than in conventional PI-RADS TZ scoring (0.155) (p = .003).

CONCLUSIONS

DWI-guided TZ scoring improves the interobserver agreement, specificity, and predictive value without impairing the sensitivity.

Comparing and Combining Artificial Intelligence and Spectral/Statistical Approaches for Elevating Prostate Cancer Assessment in a Biparametric MRI: A Pilot Study

Background: Prostate cancer management optimally requires non-invasive, objective, quantitative, accurate evaluation of prostate tumors. The current research applies visual inspection and quantitative approaches, such as artificial intelligence (AI) based on deep learning (DL), to evaluate MRI. Recently, a different spectral/statistical approach has been used to successfully evaluate spatially registered biparametric MRIs for prostate cancer. This study aimed to further assess and improve the spectral/statistical approach through benchmarking and combination with AI. Methods: A zonal-aware self-supervised mesh network (Z-SSMNet) was applied to the same 42-patient cohort from previous spectral/statistical studies. Using the probability of clinical significance of prostate cancer (PCsPCa) and a detection map, the affiliated tumor volume, eccentricity was computed for each patient. Linear and logistic regression were applied to the International Society of Urological Pathology (ISUP) grade and PCsPCa, respectively. The R, p-value, and area under the curve (AUROC) from the Z-SSMNet output were computed. The Z-SSMNet output was combined with the spectral/statistical output for multiple-variate regression. Results: The R (p-value)-AUROC [95% confidence interval] from the Z-SSMNet algorithm relating ISUP to PCsPCa is 0.298 (0.06), 0.50 [0.08-1.0]; relating it to the average blob volume, it is 0.51 (0.0005), 0.37 [0.0-0.91]; relating it to total tumor volume, it is 0.36 (0.02), 0.50 [0.0-1.0]. The R (p-value)-AUROC computations showed a much poorer correlation for eccentricity derived from the Z-SSMNet detection map. Overall, DL/AI showed poorer performance relative to the spectral/statistical approaches from previous studies. Multi-variable regression fitted AI average blob size and SCR results at a level of R = 0.70 (0.000003), significantly higher than the results for the univariate regression fits for AI and spectral/statistical approaches alone. Conclusions: The spectral/statistical approaches performed well relative to Z-SSMNet. Combining Z-SSMNet with spectral/statistical approaches significantly enhanced tumor grade prediction, possibly providing an alternative to current prostate tumor assessment.

The variability in interpretation of colonic codes in CT colonography reporting: a single-centre experience

AIM

Although standardised summary codes to classify colonic findings (C-codes) on computed tomography colonography (CTC) have been used for several years, there is no clear guidance on how these codes should be interpreted. The aims of this study were to (1) establish CTC C-code demographics and reporting practice at our hospital and (2) determine the agreement between CTC reporters when using C-codes.

MATERIALS AND METHODS

Waiving ethical approval, this online questionnaire study invited all radiologists, reporting radiographers and radiology trainees who reported CTC at our hospital between 22/02/2023 and 05/03/2023. In total 20 questions were developed with 9 questions on demographics and reporting practice followed by 11 case scenarios. Agreement between participants for the case scenarios was calculated using Fleiss kappa and mean pairwise agreement.

RESULTS

18/21 (85.7%) of participants completed the questionnaire. The majority of respondents reported using C-codes "always" (17/18, 94.4%). Overall agreement for the 11 case scenarios was fair [0.39 (95% CI 0.38-0.41)] with a mean pairwise agreement of 46.9%. Agreement was significantly greater for reporters with ≤ 1000 than > 1000 CTC experience (p < 0.001), those who reported diminutive polyps than those who did not (p < 0.001), and adequate than inadequate case scenarios (p < 0.001).

CONCLUSION

This questionnaire study demonstrates variation with how C-codes are interpreted at our institution. We suggest a national survey to determine whether this is a widespread issue and to inform development of formal implementation guidance within the UK Bowel Cancer Screening Programme.

Case-by-case combination of the prostate imaging reporting and data system version 2.1 with the Likert score to reduce the false-positives of prostate MRI: a proof-of-concept study

OBJECTIVES

To retrospectively investigate whether a case-by-case combination of the Prostate Imaging Reporting and Data System version 2.1 (PI-RADS) with the Likert score improves the diagnostic performance of mpMRI for clinically significant prostate cancer (csPCa), especially by reducing false-positives.

METHODS

One hundred men received mpMRI between January 2020 and April 2021, followed by prostate biopsy. Reader 1 (R1) and reader 2 (R2) (experience of > 3000 and < 200 mpMRI readings) independently reviewed mpMRIs with the PI-RADS version 2.1. After unveiling clinical information, they were free to add (or not) a Likert score to upgrade or downgrade or reinforce the level of suspicion of the PI-RADS category attributed to the index lesion or, rather, identify a new index lesion. We calculated sensitivity, specificity, and predictive values of R1/R2 in detecting csPCa when biopsying PI-RADS ≥ 3 index-lesions (strategy 1) versus PI-RADS ≥ 3 or Likert ≥ 3 index-lesions (strategy 2), with decision curve analysis to assess the net benefit. In strategy 2, the Likert score was considered dominant in determining biopsy decisions.

RESULTS

csPCa prevalence was 38%. R1/R2 used combined PI-RADS and Likert categorization in 28%/18% of examinations relying mainly on clinical features such as prostate specific antigen level and digital rectal examination than imaging findings. The specificity/positive predictive values were 66.1/63.1% for R1 (95%CI 52.9-77.6/54.5-70.9) and 50.0/51.6% (95%CI 37.0-63.0/35.5-72.4%) for R2 in the case of PI-RADS-based readings, and 74.2/69.2% for R1 (95%CI 61.5-84.5/59.4-77.5%) and 56.6/54.2% (95%CI 43.3-69.0/37.1-76.6%) for R2 in the case of combined PI-RADS/Likert readings. Sensitivity/negative predictive values were unaffected. Strategy 2 achieved greater net benefit as a trigger of biopsy for R1 only.

CONCLUSION

Case-by-case combination of the PI-RADS version 2.1 with Likert score translated into a mild but measurable impact in reducing the false-positives of PI-RADS categorization, though greater net benefit in reducing unnecessary biopsies was found in the experienced reader only.

The Application of Prostate Specific Membrane Antigen in the Diagnosis and Treatment of Prostate Cancer: Status and Challenge

In recent years, the incidence of prostate cancer has been increasing globally. Early stage of the disease can obtain a better clinical prognosis from surgery and endocrine therapy. The progression of advanced stage varies significantly between individuals, with some patients developing metastatic castration-resistant prostate cancer after standardized treatment. Therefore, staging of prostate cancer by accurate imaging is particularly important for the clinical management of patients. Simultaneously, the development of targeted therapy is also urgent for the treatment of advanced prostate cancer. Prostate specific membrane antigen as a prostate specific target has been widely used in the diagnosis and treatment of prostate cancer. This review summarizes the latest research progress of targeted prostate specific membrane antigen in the diagnosis and treatment of prostate cancer in detail, analyzes their value and challenges.

Predicting Biochemical Recurrence of Prostate Cancer Post-Prostatectomy Using Artificial Intelligence: A Systematic Review

Background/Objectives: Biochemical recurrence (BCR) after radical prostatectomy (RP) is a significant predictor of distal metastases and mortality in prostate cancer (PCa) patients. This systematic review aims to evaluate the accuracy of artificial intelligence (AI) in predicting BCR post-RP. Methods: Adhering to PRISMA guidelines, a comprehensive literature search was conducted across Medline, Embase, Web of Science, and IEEE Xplore. Studies were included if they utilised AI to predict BCR in patients post-RP. Studies involving patients who underwent radiotherapy or salvage RP were excluded. This systematic review was registered on PROSPERO (International prospective register of systematic reviews) under the ID CRD42023482392. Results: After screening 9764 articles, 24 met the inclusion criteria. The included studies involved 27,216 patients, of whom 7267 developed BCR. AI algorithms developed using radiological parameters demonstrated higher predictive accuracy (median AUROC of 0.90) compared to algorithms based solely on pathological variables (median AUROC of 0.74) or clinicopathological variables (median AUROC of 0.81). According to the Prediction Model Risk of Bias Assessment Tool (PROBAST), the overall risk of bias was unclear in three studies due to ambiguous inclusion criteria and the exclusion of many patients because of missing follow-up data. In seven studies, the developed AI outperformed or was at least equivocal to traditional methods of BCR prediction. Conclusions: AI shows promise in predicting BCR post-RP, particularly when radiological data were used in its development. However, the significant variability in AI performance and study methodologies highlights the need for larger, standardised prospective studies with external validation prior to clinical application.

Perspectives on technology: Prostate Imaging-Reporting and Data System (PI-RADS) interobserver variability

OBJECTIVES

To explore the topic of Prostate Imaging-Reporting and Data System (PI-RADS) interobserver variability, including a discussion of major sources, mitigation approaches, and future directions.

METHODS

A narrative review of PI-RADS interobserver variability.

RESULTS

PI-RADS was developed in 2012 to set technical standards for prostate magnetic resonance imaging (MRI), reduce interobserver variability at interpretation, and improve diagnostic accuracy in the MRI-directed diagnostic pathway for detection of clinically significant prostate cancer. While PI-RADS has been validated in selected research cohorts with prostate cancer imaging experts, subsequent prospective studies in routine clinical practice demonstrate wide variability in diagnostic performance. Radiologist and biopsy operator experience are the most important contributing drivers of high-quality care among multiple interrelated factors including variability in MRI hardware and technique, image quality, and population and patient-specific factors such as prostate cancer disease prevalence. Iterative improvements in PI-RADS have helped flatten the curve for novice readers and reduce variability. Innovations in image quality reporting, administrative and organisational workflows, and artificial intelligence hold promise in improving variability even further.

CONCLUSION

Continued research into PI-RADS is needed to facilitate benchmark creation, reader certification, and independent accreditation, which are systems-level interventions needed to uphold and maintain high-quality prostate MRI across entire populations.

Diagnostic accuracy of qualitative and quantitative magnetic resonance imaging-guided contrast-enhanced ultrasound (MRI-guided CEUS) for the detection of prostate cancer: a prospective and multicenter study

PURPOSE

To evaluate the diagnostic value of MRI-guided contrast-enhanced ultrasound (CEUS) for prostate cancer (PCa) diagnosis, and characteristics of PCa in qualitative and quantitative CEUS.

MATERIAL AND METHODS

This prospective and multicenter study included 250 patients (133 in the training cohort, 57 in the validation cohort and 60 in the test cohort) who underwent MRI, MRI-guided CEUS and prostate biopsy between March 2021 and February 2023. MRI interpretation, qualitative and quantitative CEUS analysis were conducted. Multitree extreme gradient boosting (XGBoost) machine learning-based models were applied to select the eight most important quantitative parameters. Univariate and multivariate logistic regression models were constructed to select independent predictors of PCa. Diagnostic value was determined for MRI, qualitative and quantitative CEUS using the area under receiver operating characteristic curve (AUC).

RESULTS

The performance of quantitative CEUS was superior to that of the qualitative CEUS and MRI in predicting PCa. The AUC was 0.779 (95%CI 0.70-0.849), 0.756 (95%CI 0.638-0.874) and 0.759 (95%CI 0.638-0.879) of qualitative CEUS, and 0.885 (95%CI 0.831-0.940), 0.802 (95%CI 0.684-0.919) and 0.824 (95%CI 0.713-0.936) of quantitative CEUS in training, validation and test cohort, respectively. Compared with quantitative CEUS, MRI achieved less well performance for AUC 0.811 (95%CI 0.741-0.882, p = 0.099), 0.748 (95%CI 0.628-0.868, p = 0.539) and 0.737 (95%CI 0.602-0.873, p = 0.029), respectively. Moreover, the highest specificity of 80.6% was obtained by quantitative CEUS.

CONCLUSION

We developed a reliable method of MRI-guided CEUS that demonstrated enhanced performance compared to MRI. The qualitative and quantitative CEUS characteristics will contribute to improved diagnosis of PCa.

Magnetic Resonance Imaging, Clinical, and Biopsy Findings in Suspected Prostate Cancer: A Systematic Review and Meta-Analysis

Importance

Multiple strategies integrating magnetic resonance imaging (MRI) and clinical data have been proposed to determine the need for a prostate biopsy in men with suspected clinically significant prostate cancer (csPCa) (Gleason score ≥3 + 4). However, inconsistencies across different strategies create challenges for drawing a definitive conclusion.

Objective

To determine the optimal prostate biopsy decision-making strategy for avoiding unnecessary biopsies and minimizing the risk of missing csPCa by combining MRI Prostate Imaging Reporting & Data System (PI-RADS) and clinical data.

Data Sources

PubMed, Ovid MEDLINE, Embase, Web of Science, and Cochrane Library from inception to July 1, 2022.

Study Selection

English-language studies that evaluated men with suspected but not confirmed csPCa who underwent MRI PI-RADS followed by prostate biopsy were included. Each study had proposed a biopsy plan by combining PI-RADS and clinical data.

Data Extraction and Synthesis

Studies were independently assessed for eligibility for inclusion. Quality of studies was appraised using the Quality Assessment of Diagnostic Accuracy Studies 2 tool and the Newcastle-Ottawa Scale. Mixed-effects meta-analyses and meta-regression models with multimodel inference were performed. Reporting of this study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline.

Main Outcomes and Measures

Independent risk factors of csPCa were determined by performing meta-regression between the rate of csPCa and PI-RADS and clinical parameters. Yields of different biopsy strategies were assessed by performing diagnostic meta-analysis.

Results

The analyses included 72 studies comprising 36 366 patients. Univariable meta-regression showed that PI-RADS 4 (β-coefficient [SE], 7.82 [3.85]; P = .045) and PI-RADS 5 (β-coefficient [SE], 23.18 [4.46]; P < .001) lesions, but not PI-RADS 3 lesions (β-coefficient [SE], -4.08 [3.06]; P = .19), were significantly associated with a higher risk of csPCa. When considered jointly in a multivariable model, prostate-specific antigen density (PSAD) was the only clinical variable significantly associated with csPCa (β-coefficient [SE], 15.50 [5.14]; P < .001) besides PI-RADS 5 (β-coefficient [SE], 9.19 [3.33]; P < .001). Avoiding biopsy in patients with lesions with PI-RADS category of 3 or less and PSAD less than 0.10 (vs <0.15) ng/mL2 resulted in reducing 30% (vs 48%) of unnecessary biopsies (compared with performing biopsy in all suspected patients), with an estimated sensitivity of 97% (vs 95%) and number needed to harm of 17 (vs 15).

Conclusions and Relevance

These findings suggest that in patients with suspected csPCa, patient-tailored prostate biopsy decisions based on PI-RADS and PSAD could prevent unnecessary procedures while maintaining high sensitivity.

The feasibility of MR elastography with transpelvic vibration for localization of focal prostate lesion

We aimed to evaluate the feasibility of MR elastography (MRE) using a transpelvic approach. Thirty-one patients who underwent prostate MRE and had a pathological diagnosis were included in this study. MRE was obtained using a passive driver placed at the umbilicus and iliac crests. The shear stiffness, clinical data, and conventional imaging findings of prostate cancer and benign prostatic hyperplasia (BPH) were compared. Inter-reader agreements were evaluated using the intraclass coefficient class (ICC). Prostate MRE was successfully performed for all patients (100% technical success rate). Nineteen cancer and 10 BPH lesions were visualized on MRE. The mean shear stiffness of cancer was significantly higher than that of BPH (5.99 ± 1.46 kPa vs. 4.67 ± 1.54 kPa, p = 0.045). One cancer was detected on MRE but not on conventional sequences. Six tiny cancer lesions were not visualized on MRE. The mean size of cancers that were not detected on MRE was smaller than that of cancers that were visible on MRE (0.8 ± 0.3 cm vs. 2.3 ± 1.8 cm, p = 0.001). The inter-reader agreement for interpreting MRE was excellent (ICC = 0.95). Prostate MRE with transpelvic vibration is feasible without intracavitary actuators. Transpelvic prostate MRE is reliable for detecting focal lesions, including clinically significant prostate cancer and BPH.

Interpretation of Prostate Magnetic Resonance Imaging Using Prostate Imaging and Data Reporting System Version 2.1: A Primer

Prostate magnetic resonance imaging (MRI) is increasingly being used to diagnose and stage prostate cancer. The Prostate Imaging and Data Reporting System (PI-RADS) version 2.1 is a consensus-based reporting system that provides a standardized and reproducible method for interpreting prostate MRI. This primer provides an overview of the PI-RADS system, focusing on its current role in clinical interpretation. It discusses the appropriate use of PI-RADS and how it should be applied by radiologists in clinical practice to assign and report PI-RADS assessments. We also discuss the changes from prior versions and published validation studies on PI-RADS accuracy and reproducibility.

Inter-reader variability and reproducibility of the PI-QUAL score in a multicentre setting

PURPOSE

To assess the inter-reader reproducibility of the Prostate Imaging Quality (PI-QUAL) score between readers with varying clinical experience and its reproducibility at assessing imaging quality between different institutions.

METHODS

Following IRB approval, we assessed 60 consecutive prostate MRI scans performed at different academic teaching and non-academic hospitals uploaded to our institutes' PACS for second opinion or discussion in case conferences. Anonymized scans were independently reviewed using the PI-QUAL scoring sheet by three readers - two radiologists (with 1 and 12 years Prostate MRI reporting experience), and an experienced MRI technician with interest in image acquisition and quality. All readers were blinded to the site where scans were acquired.

RESULTS

Agreement coefficients between the 3 readers in paired comparison for each individual PI-QUAL score was moderate. When the scans were clustered into 2 groups according to their ability to rule in or rule out clinically significant prostate cancer [i.e., PI-QUAL score 1-3 vs PI-QUAL score 4-5], the Gwet AC1 coefficients between the three readers in paired comparison was good to very good [Gwet AC 1:0.77, 0.67, 0.836 respectively] with agreement percentage of 88.3%, 83.3% and 91.7% respectively. Agreement coefficient was higher between the experienced radiologist and the experienced MRI technician than between the less experienced trainee radiologist and the other two readers. The mean PI-QUAL score provided by each reader for the scans was significantly higher in the academic hospitals (n = 32) compared to the community hospital (n = 28) [experienced radiologist 4.6 vs 2.9; trainee radiologist 4.5 vs 2.4; experienced technologist 4.4 vs 2.4; p value < 0.001].

CONCLUSION

We observed good to very good reproducibility in the assessment of each MRI sequence and when scans were clustered into two groups [PI-QUAL 1-3 vs PI-QUAL 4-5] between readers with varying clinical experience. However, the reproducibility for each single PI-QUAL score between readers was moderate. Better definitions for each PI-QUAL score criteria may further improve reproducibility between readers. Additionally, the mean PI-QUAL score provided by all three readers was significantly higher for scans performed at academic teaching hospitals compared to community hospital.

Multivariable stratification of PI-RADS version 2.1 categories for the risk of false-positive target biopsy: Impact on prostate biopsy decisions

PURPOSE

To identify clinical and multiparametric magnetic resonance imaging (mpMRI) factors predicting false positive target biopsy (FP-TB) of prostate imaging reporting and data system version 2.1 (PI-RADSv2.1) ≥ 3 findings.

METHOD

We retrospectively included 221 men with and without previous negative prostate biopsy who underwent 3.0 T/1.5 T mpMRI for suspicious clinically significant prostate cancer (csPCa) between April 2019-July 2021. A study coordinator revised mpMRI reports provided by one of two radiologists (experience of > 1500/>500 mpMRI examinations, respectively) and matched them with the results of transperineal systematic biopsy plus fusion target biopsy (TB) of PI-RADSv2.1 ≥ 3 lesions or PI-RADSv2.1 ≤ 2 men with higher clinical risk. A multivariable model was built to identify features predicting FP-TB of index lesions, defined as the absence of csPCa (International Society of Urogenital Pathology [ISUP] ≥ 2). The model was internally validated with the bootstrap technique, receiving operating characteristics (ROC) analysis, and decision analysis.

RESULTS

Features significantly associated with FP-TB were age < 65 years (odds ratio [OR] 2.77), prostate-specific antigen density (PSAD) < 0.15 ng/mL/mL (OR 2.45), PI-RADS 4/5 category vs. category 3 (OR 0.15/0.07), and multifocality (OR 0.46), with a 0.815 area under the curve (AUC) in assessing FP-TB. When adjusting PI-RADSv2.1 categorization for the model, mpMRI showed 87.5% sensitivity and 79.9% specificity for csPCa, with a greater net benefit in triggering biopsy compared to unadjusted categorization or adjustment for PSAD only at decision analysis, from threshold probability ≥ 15%.

CONCLUSION

Adjusting PI-RADSv2.1 categories for a multivariable risk of FP-TB is potentially more effective in triggering TB of index lesions than unadjusted PI-RADS categorization or adjustment for PSAD alone.

Bridging the experience gap in prostate multiparametric magnetic resonance imaging using artificial intelligence: A prospective multi-reader comparison study on inter-reader agreement in PI-RADS v2.1, image quality and reporting time between novice and expert readers

PURPOSE

The aim of the study was to determine the impact of using a semi-automatic commercially available AI-assisted software (Quantib® Prostate) on inter-reader agreement in PI-RADS scoring at different PI-QUAL ratings and grades of reader confidence and on reporting times among novice readers in multiparametric prostate MRI.

METHODS

A prospective observational study, with a final cohort of 200 patients undergoing mpMRI scans, was performed at our institution. An expert fellowship-trained urogenital radiologist interpreted all 200 scans based on PI-RADS v2.1. The scans were divided into four equal batches of 50 patients. Four independent readers evaluated each batch with and without the use of AI-assisted software, blinded to expert and individual reports. Dedicated training sessions were held before and after each batch. Image quality rated according to PI-QUAL and reporting times were recorded. Readers' confidence was also evaluated. A final evaluation of the first batch was conducted at the end of the study to assess for any changes in performance.

RESULTS

The overall kappa coefficient differences in PI-RADS scoring agreement without and with Quantib® were 0.673 to 0.736 for Reader 1, 0.628 to 0.483 for Reader 2, 0.603 to 0.292 for Reader 3 and 0.586 to 0.613 for Reader 4. Using PI-RADS ≥ 4 as cut-off for biopsy, the AUCs with AI ranged from 0.799 (95 % CI: 0.743, 0.856) to 0.820 (95 % CI: 0.765, 0.874). Inter-reader agreements at different PI-QUAL scores were higher with the use of Quantib, particularly for readers 1 and 4, with Kappa coefficient values showing moderate to slight agreement.

CONCLUSION

Quantib® Prostate could potentially be useful in improving inter-reader agreement among less experienced to completely novice readers if used as a supplement to PACS.

Characterization of high-grade prostate cancer at multiparametric MRI: assessment of PI-RADS version 2.1 and version 2 descriptors across 21 readers with varying experience (MULTI study)

OBJECTIVE

To assess PI-RADSv2.1 and PI-RADSv2 descriptors across readers with varying experience.

METHODS

Twenty-one radiologists (7 experienced (≥ 5 years) seniors, 7 less experienced seniors and 7 juniors) assessed 240 'predefined' lesions from 159 pre-biopsy multiparametric prostate MRIs. They specified their location (peripheral, transition or central zone) and size, and scored them using PI-RADSv2.1 and PI-RADSv2 descriptors. They also described and scored 'additional' lesions if needed. Per-lesion analysis assessed the 'predefined' lesions, using targeted biopsy as reference; per-lobe analysis included 'predefined' and 'additional' lesions, using combined systematic and targeted biopsy as reference. Areas under the curve (AUCs) quantified the performance in diagnosing clinically significant cancer (csPCa; ISUP ≥ 2 cancer). Kappa coefficients (κ) or concordance correlation coefficients (CCC) assessed inter-reader agreement.

RESULTS

At per-lesion analysis, inter-reader agreement on location and size was moderate-to-good (κ = 0.60-0.73) and excellent (CCC ≥ 0.80), respectively. Agreement on PI-RADSv2.1 scoring was moderate (κ = 0.43-0.47) for seniors and fair (κ = 0.39) for juniors. Using PI-RADSv2.1, juniors obtained a significantly lower AUC (0.74; 95% confidence interval [95%CI]: 0.70-0.79) than experienced seniors (0.80; 95%CI 0.76-0.84; p = 0.008) but not than less experienced seniors (0.74; 95%CI 0.70-0.78; p = 0.75). As compared to PI-RADSv2, PI-RADSv2.1 downgraded 17 lesions/reader (interquartile range [IQR]: 6-29), of which 2 (IQR: 1-3) were csPCa; it upgraded 4 lesions/reader (IQR: 2-7), of which 1 (IQR: 0-2) was csPCa. Per-lobe analysis, which included 60 (IQR: 25-73) 'additional' lesions/reader, yielded similar results.

CONCLUSIONS

Experience significantly impacted lesion characterization using PI-RADSv2.1 descriptors. As compared to PI-RADSv2, PI-RADSv2.1 tended to downgrade non-csPCa lesions, but this effect was small and variable across readers.