Prostate Imaging and Data Reporting System Version 2 as a Radiology Performance Metric: An Analysis of 18 Abdominal Radiologists

VPAC receptor positivity in comparison with mp-MRI in the diagnosis of prostate cancer: A preliminary study

Objective

The study aimed to prospectively evaluate the feasibility of diagnosing PCa using voided urine samples and by targeting the genomic VPAC (vasoactive intestinal peptide and pituitary adenylate cyclase-activating peptide) receptors in comparison with multiparametric magnetic resonance imaging (mp-MRI) in male patients (≥40 years of age) with lower urinary tract symptoms and having a serum PSA of >4 but <15 ng/ml.

Patients and Methods

Male patients attending urological services ≥40 years old, with lower urinary tract symptoms and serum PSA levels of >4 but <15 ng/ml formed the study group. Voided urine samples were collected to target VPAC receptors on malignant cells. All patients underwent mp-MRI. A 12-core transrectal ultrasound-guided prostate biopsy was performed in all, and the results were compared for the diagnosis of PCa.

Results

A total of 61 patients with a median age of 65.33 ± 8.11 years and with a median serum PSA of 9.56 ± 2.78 ng/ml were further evaluated with both urinary biomarker assessment and mp-MRI. Histopathological (HPR) confirmation of PCa was noted in 25 (40.98%) patients and benign prostatic hyperplasia in the remaining 36 (59.01%) patients. Of the 25 patients with histologically proven PCa, the urinary biomarker (VPAC positivity) was positive for malignancy in 24 (96%), one case showed false negative results (4%) and there were no false positive cases (0%). HPR confirming PCa was seen in 3/16 patients with a PIRADS 2 score, 7/21 patients with a PIRADS 3 score, 7/14 patients with a PIRADS 4 score and 8/8 patients with a PIRADS score of 5.

Conclusions

VPAC receptor positivity of prostate cancer cells is an easy test to perform using a voided urine sample. VPAC receptor positivity can be used as an indication for prostate biopsy in patients having a negative previous biopsy but highly suspicious of cancer, in patients with an elevated serum PSA but with a normal digital rectal examination and in patients with benign features and borderline elevation of serum PSA.

Clinical and Imaging Predictors of False-Positive and False-Negative Results in Prostate Multiparametric MRI Using PI-RADS Version 2

Purpose To evaluate predictors of false-positive (FP) and false-negative (FN) results for prostate cancer at prostate multiparametric MRI (mpMRI) using the Prostate Imaging and Reporting Data System version 2 (PI-RADS v2). Materials and Methods This was a single-center retrospective cohort study of 2548 consecutive patients who underwent prostate mpMRI examinations (October 2016-July 2022) containing zero or one PI-RADS v2 category 3-5 lesions. Prostate mpMRI examinations were interpreted by 13 radiologists. FP results were defined as prospective PI-RADS v2 score of 3 or higher but benign or grade group 1 prostate cancer at subsequent combined targeted and systematic biopsy. FN results were defined as prospective PI-RADS v2 score 2 or lower but grade group 2 or higher prostate cancer at subsequent combined targeted and systematic biopsy. Predictors of FP and FN results were assessed by logistic regression. Results Among the 2548 patients (mean age, 65.7 years ± 7.6 [SD]; all male) analyzed, 52.0% (831 of 1597) had FP results and 15.8% (150 of 951) had FN results at mpMRI. FP results were more likely for younger patients (odds ratio [OR], 0.95/y; P < .001), smaller lesions (OR, 0.62/mm; P < .001), transition zone lesions (OR, 1.74 vs peripheral zone; P = .006), and patients with low prostate-specific antigen (PSA) density (OR, 0.55 per 0.1 ng/mL2 increase; P < .001). FN results were more likely for older patients (OR, 1.03/y; P = .01) and patients with high PSA density (OR, 2.05 per 0.1 ng/mL2 increase; P < .001). Conclusion PSA density and patient age independently predicted FP and FN results for detection of prostate cancer at mpMRI using PI-RADS v2. These factors are not part of the PI-RADS v2 algorithm and may inform mpMRI interpretation to improve prostate cancer diagnosis. Keywords: MR Imaging, Prostate, PI-RADS, Prostate MRI, Prostate Cancer ©RSNA, 2025.

Estimated diagnostic performance of prostate MRI performed with clinical suspicion of prostate cancer

PURPOSE

To assess the diagnostic performance of prostate MRI by estimating the proportion of clinically significant prostate cancer (csPCa) in patients without prostate pathology.

MATERIALS AND METHODS

This three-center retrospective study included prostate MRI examinations performed for clinical suspicion of csPCa (Grade group ≥ 2) between 2018 and 2022. Examinations were divided into two groups: pathological diagnosis within 1 year after the MRI (post-MRI pathology) is present and absent. Risk prediction models were developed using the extracted eleven common predictive variables from the patients with post-MRI pathology. Then, the csPCa proportion in the patients without post-MRI pathology was estimated by applying the model. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and positive and negative predictive values (PPV/NPV) of prostate MRI in diagnosing csPCa were subsequently calculated for patients with and without post-MRI prostate pathology (estimated statistics) with a positive threshold of PI-RADS ≥ 3.

RESULTS

Of 12,191 examinations enrolled (mean age, 65.7 years ± 8.4 [standard deviation]), PI-RADS 1-2 was most frequently assigned (55.4%) with the lowest pathological confirmation rate of 14.0-18.2%. Post-MRI prostate pathology was found in 5670 (46.5%) examinations. The estimated csPCa proportions across facilities were 12.6-15.3%, 18.4-31.4%, 45.7-69.9%, and 75.4-88.3% in PI-RADS scores of 1-2, 3, 4, and 5, respectively. The estimated (observed) performance statistics were as follows: AUC, 0.78-0.81 (0.76-0.79); sensitivity, 76.6-77.3%; specificity, 67.5-78.6%; PPV, 49.8-66.6% (52.0-67.7%); and NPV, 84.4-87.2% (82.4-86.6%).

CONCLUSION

We proposed a method to estimate the probabilities harboring csPCa for patients who underwent prostate MRI examinations, which allows us to understand the PI-RADS diagnostic performance with several metrics.

CLINICAL RELEVANCE STATEMENT

The reported estimated performance metrics are expected to aid in understanding the true diagnostic value of PI-RADS in the entire prostate MRI population performed with clinical suspicion of prostate cancer.

KEY POINTS

Calculating performance metrics only from patients who underwent prostate biopsy may be biased due to biopsy selection criteria, especially in PI-RADS 1-2. The estimated area under the receiver operating characteristic curve of PI-RADS in the entire prostate MRI population ranged from 0.78 to 0.81 at three facilities. The estimated statistics are expected to help us understand the true PI-RADS performance and serve as a reference for future studies.

Perspectives on technology: Prostate Imaging-Reporting and Data System (PI-RADS) interobserver variability

OBJECTIVES

To explore the topic of Prostate Imaging-Reporting and Data System (PI-RADS) interobserver variability, including a discussion of major sources, mitigation approaches, and future directions.

METHODS

A narrative review of PI-RADS interobserver variability.

RESULTS

PI-RADS was developed in 2012 to set technical standards for prostate magnetic resonance imaging (MRI), reduce interobserver variability at interpretation, and improve diagnostic accuracy in the MRI-directed diagnostic pathway for detection of clinically significant prostate cancer. While PI-RADS has been validated in selected research cohorts with prostate cancer imaging experts, subsequent prospective studies in routine clinical practice demonstrate wide variability in diagnostic performance. Radiologist and biopsy operator experience are the most important contributing drivers of high-quality care among multiple interrelated factors including variability in MRI hardware and technique, image quality, and population and patient-specific factors such as prostate cancer disease prevalence. Iterative improvements in PI-RADS have helped flatten the curve for novice readers and reduce variability. Innovations in image quality reporting, administrative and organisational workflows, and artificial intelligence hold promise in improving variability even further.

CONCLUSION

Continued research into PI-RADS is needed to facilitate benchmark creation, reader certification, and independent accreditation, which are systems-level interventions needed to uphold and maintain high-quality prostate MRI across entire populations.

Comparison of Positive Predictive Values of Biparametric MRI and Multiparametric MRI-directed Transrectal US-guided Targeted Prostate Biopsy

Background Biparametric MRI (bpMRI) of the prostate is an alternative to multiparametric MRI (mpMRI), with lower cost and increased accessibility. Studies investigating the positive predictive value (PPV) of bpMRI-directed compared with mpMRI-directed targeted biopsy are lacking in the literature. Purpose To compare the PPVs of bpMRI-directed and mpMRI-directed targeted prostate biopsies. Materials and Methods This retrospective cross-sectional study evaluated men who underwent bpMRI-directed or mpMRI-directed transrectal US (TRUS)-guided targeted prostate biopsy at a single institution from January 2015 to December 2022. The PPVs for any prostate cancer (PCa) and clinically significant PCa (International Society of Urological Pathology grade ≥2) were calculated for bpMRI and mpMRI using mixed-effects logistic regression modeling. Results A total of 1538 patients (mean age, 67 years ± 8 [SD]) with 1860 lesions underwent bpMRI-directed (55%, 849 of 1538) or mpMRI-directed (45%, 689 of 1538) prostate biopsy. When adjusted for the number of lesions and Prostate Imaging Reporting and Data System (PI-RADS) score, there was no difference in PPVs for any PCa or clinically significant PCa (P = .61 and .97, respectively) with bpMRI-directed (55% [95% CI: 51, 59] and 34% [95% CI: 30, 38], respectively) or mpMRI-directed (56% [95% CI: 52, 61] and 34% [95% CI: 30, 39], respectively) TRUS-guided targeted biopsy. PPVs for any PCa and clinically significant PCa stratified according to clinical indication were as follows: biopsy-naive men, 64% (95% CI: 59, 69) and 43% (95% CI: 39, 48) for bpMRI, 67% (95% CI: 59, 75) and 51% (95% CI: 43, 59) for mpMRI (P = .65 and .26, respectively); and active surveillance, 59% (95% CI: 49, 69) and 30% (95% CI: 22, 39) for bpMRI, 73% (95% CI: 65, 89) and 38% (95% CI: 31, 47) for mpMRI (P = .04 and .23, respectively). Conclusion There was no evidence of a difference in PPV for clinically significant PCa between bpMRI- and mpMRI-directed TRUS-guided targeted biopsy. © RSNA, 2024 Supplemental material is available for this article.

Predicting clinically significant prostate cancer following suspicious mpMRI: analyses from a high-volume center

PURPOSE

mpMRI is routinely used to stratify the risk of clinically significant prostate cancer (csPCa) in men with elevated PSA values before biopsy. This study aimed to calculate a multivariable risk model incorporating standard risk factors and mpMRI findings for predicting csPCa on subsequent prostate biopsy.

METHODS

Data from 677 patients undergoing mpMRI ultrasound fusion biopsy of the prostate at the TUM University Hospital tertiary urological center between 2019 and 2023 were analyzed. Patient age at biopsy (67 (median); 33-88 (range) (years)), PSA (7.2; 0.3-439 (ng/ml)), prostate volume (45; 10-300 (ml)), PSA density (0.15; 0.01-8.4), PI-RADS (V.2.0 protocol) score of index lesion (92.2% ≥3), prior negative biopsy (12.9%), suspicious digital rectal examination (31.2%), biopsy cores taken (12; 2-22), and pathological biopsy outcome were analyzed with multivariable logistic regression for independent associations with the detection of csPCa defined as ISUP ≥ 3 (n = 212 (35.2%)) and ISUP ≥ 2 (n = 459 (67.8%) performed on 603 patients with complete information.

RESULTS

Older age (OR: 1.64 for a 10-year increase; p < 0.001), higher PSA density (OR: 1.60 for a doubling; p < 0.001), higher PI-RADS score of the index lesion (OR: 2.35 for an increase of 1; p < 0.001), and a prior negative biopsy (OR: 0.43; p = 0.01) were associated with csPCa.

CONCLUSION

mpMRI findings are the dominant predictor for csPCa on follow-up prostate biopsy. However, PSA density, age, and prior negative biopsy history are independent predictors. They must be considered when discussing the individual risk for csPCa following suspicious mpMRI and may help facilitate the further diagnostical approach.

Decreased prostate MRI cancer detection rate due to moderate to severe susceptibility artifacts from hip prosthesis

OBJECTIVES

To evaluate the impact of susceptibility artifacts from hip prosthesis on cancer detection rate (CDR) in prostate MRI.

MATERIALS AND METHODS

This three-center retrospective study included prostate MRI studies for patients without known prostate cancer between 2017 and 2021. Exams with hip prosthesis were searched on MRI reports. The degree of susceptibility artifact on diffusion-weighted images was retrospectively categorized into mild, moderate, and severe (> 66%, 33-66%, and < 33% of the prostate volume are evaluable) by blind reviewers. CDR was defined as the number of exams with Gleason score ≥7 detected by MRI (PI-RADS ≥3) divided by the total number of exams. For each artifact grade, control exams without hip prosthesis were matched (1:6 match), and CDR was compared. The degree of CDR reduction was evaluated with ratio, and influential factors were evaluated by expanding the equation.

RESULTS

Hip arthroplasty was present in 548 (4.8%) of the 11,319 MRI exams. CDR of the cases and matched control exams for each artifact grade were as follows: mild (n = 238), 0.27 vs 0.25, CDR ratio = 1.09 [95% CI: 0.87-1.37]; moderate (n = 143), 0.18 vs 0.27, CDR ratio = 0.67 [95% CI: 0.46-0.96]; severe (n = 167), 0.22 vs 0.28, CDR ratio = 0.80 [95% CI: 0.59-1.08]. When moderate and severe artifact grades were combined, CDR ratio was 0.74 [95% CI: 0.58-0.93]. CDR reduction was mostly attributed to the increased frequency of PI-RADS 1-2.

CONCLUSION

With moderate to severe susceptibility artifacts from hip prosthesis, CDR was decreased to 74% compared to the matched control.

CLINICAL RELEVANCE STATEMENT

Moderate to severe susceptibility artifacts from hip prosthesis may cause a non-negligible CDR reduction in prostate MRI. Expanding indications for systematic prostate biopsy may be considered when PI-RADS 1-2 was assigned.

KEY POINTS

• We proposed cancer detection rate as a diagnostic performance metric in prostate MRI. • With moderate to severe susceptibility artifacts secondary to hip arthroplasty, cancer detection rate decreased to 74% compared to the matched control. • Expanding indications for systematic prostate biopsy may be considered when PI-RADS 1-2 is assigned.

Prediction of false-positive PI-RADS 5 lesions on prostate multiparametric MRI: development and internal validation of a clinical-radiological characteristics based nomogram

BACKGROUND

To develop a risk model including clinical and radiological characteristics to predict false-positive The Prostate Imaging Reporting and Data System (PI-RADS) 5 lesions.

METHODS

Data of 612 biopsy-naïve patients who had undergone multiparametric magnetic resonance imaging (mpMRI) before prostate biopsy were collected. Clinical variables and radiological variables on mpMRI were adopted. Lesions were divided into the training and validation cohort randomly. Stepwise multivariate logistic regression analysis with backward elimination was performed to screen out variables with significant difference. A diagnostic nomogram was developed in the training cohort and further validated in the validation cohort. Calibration curve and receiver operating characteristic (ROC) analysis were also performed.

RESULTS

296 PI-RADS 5 lesions in 294 patients were randomly divided into the training and validation cohort (208 : 88). 132 and 56 lesions were confirmed to be clinically significant prostate cancer in the training and validation cohort respectively. The diagnostic nomogram was developed based on prostate specific antigen density, the maximum diameter of lesion, zonality of lesion, apparent diffusion coefficient minimum value and apparent diffusion coefficient minimum value ratio. The C-index of the model was 0.821 in the training cohort and 0.871 in the validation cohort. The calibration curve showed good agreement between the estimation and observation in the two cohorts. When the optimal cutoff values of ROC were 0.288 in the validation cohort, the sensitivity, specificity, PPV, and NPV were 90.6%, 67.9%, 61.7%, and 92.7% in the validation cohort, potentially avoiding 9.7% unnecessary prostate biopsies.

CONCLUSIONS

We developed and validated a diagnostic nomogram by including 5 factors. False positive PI-RADS 5 lesions could be distinguished from clinically significant ones, thus avoiding unnecessary prostate biopsy.

Improving the Reliability of Peer Review Without a Gold Standard

Peer review plays a crucial role in accreditation and credentialing processes as it can identify outliers and foster a peer learning approach, facilitating error analysis and knowledge sharing. However, traditional peer review methods may fall short in effectively addressing the interpretive variability among reviewing and primary reading radiologists, hindering scalability and effectiveness. Reducing this variability is key to enhancing the reliability of results and instilling confidence in the review process. In this paper, we propose a novel statistical approach called "Bayesian Inter-Reviewer Agreement Rate" (BIRAR) that integrates radiologist variability. By doing so, BIRAR aims to enhance the accuracy and consistency of peer review assessments, providing physicians involved in quality improvement and peer learning programs with valuable and reliable insights. A computer simulation was designed to assign predefined interpretive error rates to hypothetical interpreting and peer-reviewing radiologists. The Monte Carlo simulation then sampled (100 samples per experiment) the data that would be generated by peer reviews. The performances of BIRAR and four other peer review methods for measuring interpretive error rates were then evaluated, including a method that uses a gold standard diagnosis. Application of the BIRAR method resulted in 93% and 79% higher relative accuracy and 43% and 66% lower relative variability, compared to "Single/Standard" and "Majority Panel" peer review methods, respectively. Accuracy was defined by the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. Variability was defined by the 95% CI around the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. BIRAR is a practical and scalable peer review method that produces more accurate and less variable assessments of interpretive quality by accounting for variability within the group's radiologists, implicitly applying a standard derived from the level of consensus within the group across various types of interpretive findings.

Cancer Detection Rate and Abnormal Interpretation Rate of Prostate MRI in Patients With Low-Grade Cancer

PURPOSE

The aim of this study was to evaluate the utility of cancer detection rate (CDR) and abnormal interpretation rate (AIR) in prostate MRI for patients with low-grade prostate cancer (PCa).

METHODS

This three-center retrospective study included patients who underwent prostate MRI from 2017 to 2021 with known low-grade PCa (Gleason score 6) without prior treatment. Patient-level highest Prostate Imaging Reporting & Data System (PI-RADS®) score and pathologic diagnosis within 1 year after MRI were used to evaluate the diagnostic performance of prostate MRI in detecting clinically significant PCa (csPCa; Gleason score ≥ 7). The metrics AIR, CDR, and CDR adjusted for pathologic confirmation rate were calculated. Radiologist-level AIR-CDR plots were shown. Simulation AIR-CDR lines were created to assess the effects of different diagnostic performances of prostate MRI and the prevalence of csPCa.

RESULTS

A total of 3,207 examinations were interpreted by 33 radiologists. Overall AIR, CDR, and CDR adjusted for pathologic confirmation rate at PI-RADS 3 to 5 (PI-RADS 4 and 5) were 51.7% (36.5%), 22.1% (18.8%), and 30.7% (24.6%), respectively. Radiologist-level AIR and CDR at PI-RADS 3 to 5 (PI-RADS 4 and 5) were in the 36.8% to 75.6% (21.9%-57.5%) range and the 16.3%-28.7% (10.9%-26.5%) range, respectively. In the simulation, changing parameters of diagnostic performance or csPCa prevalence shifted the AIR-CDR line.

CONCLUSIONS

The authors propose CDR and AIR as performance metrics in prostate MRI and report reference performance values in patients with known low-grade PCa. There was variability in radiologist-level AIR and CDR. Combined use of AIR and CDR could provide meaningful feedback for radiologists to improve their performance by showing relative performance to other radiologists.

Cancer Detection Rate and Abnormal Interpretation Rate of Prostate MRI Performed for Clinical Suspicion of Prostate Cancer

PURPOSE

To report cancer detection rate (CDR) and abnormal interpretation rate (AIR) in prostate MRI performed for clinical suspicion of prostate cancer (PCa).

MATERIALS AND METHODS

This retrospective single-institution, three-center study included patients who underwent MRI for clinical suspicion of PCa between 2017 and 2021. Patients with known PCa were excluded. Patient-level Prostate Imaging-Reporting and Data System (PI-RADS) score was extracted from the radiology report. AIR was defined as number of abnormal MRI (PI-RADS score 3-5) / total number of MRIs. CDR was defined as number of clinically significant PCa (csPCa: Gleason score ≥7) detected at abnormal MRI / total number of MRI. AIR, CDR, and CDR adjusted for pathology confirmation rate were calculated for each of three centers and pre-MRI biopsy status (biopsy-naive and previous negative biopsy).

RESULTS

A total of 9,686 examinations (8,643 unique patients) were included. AIR, CDR, and CDR adjusted for pathology confirmation rate were 45.4%, 23.8%, and 27.6% for center I; 47.2%, 20.0%, and 22.8% for center II; and 42.3%, 27.2%, and 30.1% for center III, respectively. Pathology confirmation rate ranged from 81.6% to 88.0% across three centers. AIR and CDR for biopsy-naive patients were 45.5% to 52.6% and 24.2% to 33.5% across three centers, respectively, and those for previous negative biopsy were 27.2% to 39.8% and 11.7% to 14.2% across three centers, respectively.

CONCLUSION

We reported CDR and AIR in prostate MRI for clinical suspicion of PCa. CDR needs to be adjusted for pathology confirmation rate and pre-MRI biopsy status for interfacility comparison.

PI-RADS: Where Next?

Prostate MRI plays an important role in the clinical management of localized prostate cancer, mainly assisting in biopsy decisions and guiding biopsy procedures. The Prostate Imaging Reporting and Data System (PI-RADS) has been available to radiologists since 2012, with the most up-to-date and actively used version being PI-RADS version 2.1. This review article discusses the current use of PI-RADS, including its limitations and controversies, and summarizes research that aims to improve future iterations of this system.

Prostate MRI Qualification: AJR Expert Panel Narrative Review

Prostate MRI is now established as a first-line investigation for individuals presenting with suspected localized or locally advanced prostate cancer. Successful delivery of the MRI-directed pathway for prostate cancer diagnosis relies on high-quality imaging as well as the interpreting radiologist's experience and expertise. Radiologist certification in prostate MRI may help limit interreader variability, optimize outcomes, and provide individual radiologists with documentation of meeting predefined standards. This AJR Expert Panel Narrative Review summarizes existing certification proposals, recognizing variable progress across regions in establishing prostate MRI certification programs. To our knowledge, Germany is the only country with a prostate MRI certification process that is currently available for radiologists. However, prostate MRI certification programs have also recently been proposed in the United States and United Kingdom and by European professional society consensus panels. Recommended qualification processes entail a multifaceted approach, incorporating components such as minimum case numbers, peer learning, course participation, continuing medical education credits, and feedback from pathology results. Given the diversity in health care systems, including in the provision and availability of MRI services, national organizations will likely need to take independent approaches to certification and accreditation. The relevant professional organizations should begin developing these programs or continue existing plans for implementation.

Effect of Prostate MRI Interpretation Experience on PPV Using PI-RADS Version 2: A 6-Year Assessment Among Eight Fellowship-Trained Radiologists

BACKGROUND. Understanding the effect of specific experience in prostate MRI interpretation on diagnostic performance would help inform the minimum interpretation volume to establish proficiency. OBJECTIVE. The purpose of this article is to assess for an association between increasing experience in prostate MRI interpretation and change in radiologist-level PPVs for PI-RADS version 2 (v2) categories 3, 4, and 5. METHODS. This retrospective study included prostate MRI examinations performed between July 1, 2015, and August 13, 2021, that were assigned a PI-RADS v2 category of 3, 4, or 5 and with an MRI-ultrasound fusion biopsy available as the reference standard. All examinations were among the first 100-200 prostate MRI examinations interpreted using PI-RADS v2 by fellowship-trained abdominal radiologists. Radiologists received feedback through a quality assurance program. Radiologists' experience levels were classified using progressive subsets of 50 interpreted examinations. Change with increasing experience in distribution of individual radiologists' whole-gland PPVs for Gleason sum score 7 or greater prostate cancer, stratified by PI-RADS category, was assessed by hierarchic linear mixed models. RESULTS. The study included 1300 prostate MRI examinations in 1037 patients (mean age, 66 ± 7 [SD] years), interpreted by eight radiologists (median, 13 years of postfellow-ship experience; range, 5-22 years). Aggregate PPVs were 20% (68/340) for PI-RADS category 3, 49% (318/652) for category 4, and 71% (220/308) for category 5. Interquartile ranges (IQRs) of PPVs overlapped for category 4 (51%; IQR, 42-60%) and category 5 (70%; IQR, 54-75%) for radiologists' first 50 examinations. IQRs of PPVs did not overlap between categories of greater experience; for example, at the 101-150 examination level, PPV for category 3 was 24% (IQR, 20-29%), category 4 was 55% (IQR, 54-63%), and category 5 was 81% (IQR, 77-82%). Hierarchic modeling showed no change in radiologists' absolute PPV with increasing experience (category 3, p = .27; category 4, p = .71; category 5, p = .38). CONCLUSION. Absolute PPVs at specific PI-RADS categories did not change during radiologists' first 200 included examinations. However, resolution of initial overlap in IQRs indicates improved precision of PPVs after the first 50 examinations. CLINICAL IMPACT. If implementing a minimum training threshold for fellowship-trained abdominal radiologists, 50 prostate MRI examinations may be sufficient in the context of a quality assurance program with feedback.

Utility of a Rule-Based Algorithm in the Assessment of Standardized Reporting in PI-RADS

RATIONALE AND OBJECTIVES

Adoption of the Prostate Imaging Reporting & Data System (PI-RADS) has been shown to increase detection of clinically significant prostate cancer on prostate mpMRI. We propose that a rule-based algorithm based on Regular Expression (RegEx) matching can be used to automatically categorize prostate mpMRI reports into categories as a means by which to assess for opportunities for quality improvement.

MATERIALS AND METHODS

All prostate mpMRIs performed in the Duke University Health System from January 2, 2015, to January 29, 2021, were analyzed. Exclusion criteria were applied, for a total of 5343 male patients and 6264 prostate mpMRI reports. These reports were then analyzed by our RegEx algorithm to be categorized as PI-RADS 1 through PI-RADS 5, Recurrent Disease, or "No Information Available." A stratified, random sample of 502 mpMRI reports was reviewed by a blinded clinical team to assess performance of the RegEx algorithm.

RESULTS

Compared to manual review, the RegEx algorithm achieved overall accuracy of 92.6%, average precision of 88.8%, average recall of 85.6%, and F1 score of 0.871. The clinical team also reviewed 344 cases that were classified as "No Information Available," and found that in 150 instances, no numerical PI-RADS score for any lesion was included in the impression section of the mpMRI report.

CONCLUSION

Rule-based processing is an accurate method for the large-scale, automated extraction of PI-RADS scores from the text of radiology reports. These natural language processing approaches can be used for future initiatives in quality improvement in prostate mpMRI reporting with PI-RADS.