Gleason scoring of prostatic carcinoma: Impact of a web-based tutorial on inter- and intra-observer variability

Use of the ISUP e-learning module improves interrater reliability in prostate cancer grading

AIMS

Prostate cancer (PCa) grading is an important prognostic parameter, but is subject to considerable observer variation. Previous studies have shown that interobserver variability decreases after participants were trained using an e-learning module. However, since the publication of these studies, grading of PCa has been enhanced by adopting the International Society of Urological Pathology (ISUP) 2014 grading classification. This study investigates the effect of training on interobserver variability of PCa grading, using the ISUP Education web e-learning on Gleason grading.

METHODS

The ISUP Education Prostate Test B Module was distributed among Dutch pathologists. The module uses images graded by the ISUP consensus panel consisting of 24 expert uropathologists. Participants graded the same 10 images before and after e-learning. We included those who completed the tests before and after training. We evaluated variation in PCa grading in a fully crossed study design, using linearly weighted kappa values for each pathologist, comparing them to other pathologists and to the ISUP consensus panel. We analysed the improvement in median weighted kappas before and after training, using Wilcoxon's signed rank-test.

RESULTS

We included 42 pathologists. Inter-rater reliability between pathologists improved from 0.70 before training to 0.74 after training (p=0.01). When compared with the ISUP consensus panel, five pathologists improved significantly, whereas the kappa of one pathologist was significantly lower after training. All pathologists who improved significantly, graded with less than substantial agreement before training.

CONCLUSIONS

ISUP Prostate Test B e-learning reduces variability in PCa grading. E-learning is a cost-effective method for standardisation of pathology.

Grade Group accuracy is improved by extensive prostate biopsy sampling, but unrelated to prostatectomy specimen sampling or use of immunohistochemistry

Assessing the accurate Grade Group of a prostate needle biopsy specimen is essential for choosing the adequate therapeutic modality for prostate cancer patients. However, it is well-known that biopsy Grade Group tends to up- or downgrade significantly at radical prostatectomy. We aimed to investigate the correlation between accuracy and biopsy core number, performed immunohistochemical staining (IHC) or prostatectomy specimen sampling, with the latest also being correlated with higher detection rates of adverse pathological features, e.g., positive surgical margins, higher pathological stage or presence of perineural invasion (PnI status). The study cohort consisted of 315 consecutive patients diagnosed with prostate adenocarcinoma via transrectal ultrasound-guided needle biopsy who later underwent radical prostatectomy. We grouped and compared patients based on Grade Group accuracy, presence of IHC on biopsy, margin status, pathological stage, and PnI status. Inter-observer reproducibility was also calculated. Statistical analyzes included ANOVA, Tukey's multiple comparisons post hoc test, Chi-squared test, and Fleiss kappa statistics. Undergraded cases harboured a significantly lower number of biopsy cores (p < 0.05), than accurately graded cases. Using IHC did not affect grading accuracy significantly, nor did the number of slides from prostatectomy specimens. The mean number of slides was virtually identical when margin status, pathological stage and PnI status of prostatectomy specimens were compared. Inter-observer reproducibility at our institute was calculated as fair (overall kappa = 0.29). Grade Group accuracy is significantly improved by obtaining more cores at biopsy but is unrelated to performed IHC. The extent of sampling prostatectomy specimens, however, did not affect accuracy and failed to significantly improve detection of adverse pathological features.

A multitask deep representation for Gleason score classification to support grade annotations

The Gleason grade system is the main standard to quantify the aggressiveness and progression of prostate cancer. Currently, exists a high disagreement among experts in the diagnosis and stratification of this disease. Deep learning models have emerged as an alternative to classify and support experts automatically. However, these models are limited to learn a rigid stratification rule that can be biased during training to a specific observer. Therefore, this work introduces an embedding representation that integrates an auxiliary task learning to deal with the high inter and intra appearance of the Gleason system. The proposed strategy implements as a main task a triplet loss scheme that builds a feature embedding space with respect to batches of positive and negative histological training patches. As an auxiliary task is added a cross-entropy that helps with inter-class variability of samples while adding robust representations to the main task. The proposed approach shows promising results achieving an average accuracy of 66% and 64%, for two experts without statistical difference. Additionally, reach and average accuracy of 73% in patches where both pathologists are agree, showing the robustness patterns learning from the approach.

[Interobserver reproducibility of Gleason grading in prostate biopsy samples]

INTRODUCTION

Gleason grading is the most common method of prostate cancer classification.

AIM

The aim of the authors was to assess the reproducibility of Gleason grading among pathologists using the same needle biopsy samples.

METHOD

23 pathologists examined 37 prostate cancer biopsies stained with hematoxylin and eosin. Gleason scores were categorised into 4 groups (2-4, 5-6, 7 and 8-10). Kappa statistics were used to reflect interobserver agreement.

RESULTS

Considering all participating pathologists, grouping into one of the 4 categories resulted in an overall kappa value of 0.49. For the individual categories, the worst agreement (kappa = 0.15) was seen with well differentiated carcinomas, and the best (kappa = 0.65) with poorly differentiated ones.

CONCLUSIONS

These results suggest that Gleason grading in biopsy samples is moderately reproducible. The kappa values vary according to the differentiation of the cancer, and there is not much difference between the results of the present study and those published in the literature. To increase reproducibility, trainings should be organised, and this could improve the quality of grading.

Gleason grading of prostate cancer in needle core biopsies: a comparison of general and urologic pathologists

BACKGROUND AND OBJECTIVES

The Gleason grading of prostate carcinoma (PCa) in needle core biopsies is a major determinant used in management planning. The objective of this study was to evaluate the concordance between general pathologists Gleason grading and that of a urologic pathologist in our community.

DESIGN AND SETTING

Retrospective review conducted at three tertiary care hospitals in Jeddah and Riyadh for all prostatic biopsies with carcinoma from January 2002 to January 2011.

METHODS

Gleason scores assigned by the original pathologist were compared with that of the reviewing urologic pathologists. Biopsies were originally obtained and diagnosed at different referring hospitals and independent laboratories. The kappa test was used to evaluate agreement between the original and review scores.

RESULTS

For 212 biopsies the exact concordance of the Gleason score assigned by the original pathologist and the reviewer was 38.7% (82/212). However, when grouped into the main four-score categories of 2-4, 5-6, 7, and 8 or greater, disagreement was noted in 88 (41.5%) biopsies; 87 were upgraded and 1 was downgraded on review. When grouped into two-score categories of low grade (≤6) and high grade (≥7), disagreement was noted in 32 (15%) of the biopsies.

CONCLUSION

Gleason grade score shows that there was only slight to fair agreement between outside and review scoring (kappa=0.43). When using only low versus high grade categorization, there was good agreement (kappa=0.69). Almost all of the cases with score disagreement were upgraded on review.

Measuring interobserver variation in a pathology EQA scheme using weighted κ for multiple readers

Background

A Urological Pathology External Quality Assurance (EQA) Scheme in the UK has reported observer variation in the diagnosis and grading of adenocarcinoma in prostatic biopsies using basic κ statistics, which rate all disagreements equally.

Aim

The aim of this study is to use customised weighting schemes to report κ statistics that reflect the closeness of interobserver agreement in the prostate EQA scheme.

Methods

A total of 83, 114 and 116 pathologists took part, respectively, in three web-based circulations and were classified as either expert or other readers. For analyses of diagnosis, there were 10, 8 and 8 cases in the three circulations, respectively. For analyses of Gleason Sum Score, only invasive cases were included, leaving 5, 5 and 6 cases, respectively. Analyses were conducted using customised weighting schemes with ‘pairwise-weighted’ κ for multiple readers.

Results

Analysis of diagnosis for all circulations and all readers gave a composite κ value of 0.86 and pairwise-weighted κ (κp–w) value of 0.91, both regarded as ‘almost perfect’ agreement. This was due to the high proportion of responses that showed partial agreement. Analysis of Gleason Sum Score gave κ=0.38 and κp–w=0.58 over all circulations and all readers, indicating that discrepancies occur at the boundary between adjacent grades and may not be as clinically significant as suggested by composite κ.

Conclusion

Weighted κ show higher levels of agreement than previously reported as they have the advantage of applying weighting, which reflects the relative importance of different types of discordance in diagnosis or grading. Agreement on grading remained low.

A practical application of analysing weighted kappa for panels of experts and EQA schemes in pathology

Background

Kappa statistics are frequently used to analyse observer agreement for panels of experts and External Quality Assurance (EQA) schemes and generally treat all disagreements as total disagreement. However, the differences between ordered categories may not be of equal importance (eg, the difference between grades 1 vs 2 compared with 1 vs 3). Weighted kappa can be used to adjust for this when comparing a small number of readers, but this has not as yet been applied to the large number of readers typical of a national EQA scheme.

Aim

To develop and validate a method for applying weighted kappa to a large number of readers within the context of a real dataset: the UK National Urological Pathology EQA Scheme for prostatic biopsies.

Methods

Data on Gleason grade recorded by 19 expert readers were extracted from the fixed text responses of 20 cancer cases from four circulations of the EQA scheme. Composite kappa, currently used to compute an unweighted kappa for large numbers of readers, was compared with the mean kappa for all pairwise combinations of readers. Weighted kappa generalised for multiple readers was compared with the newly developed ‘pairwise-weighted’ kappa.

Results

For unweighted analyses, the median increase from composite to pairwise kappa was 0.006 (range −0.005 to +0.052). The difference between the pairwise-weighted kappa and generalised weighted kappa for multiple readers never exceeded ±0.01.

Conclusion

Pairwise-weighted kappa is a suitable and highly accurate approximation to weighted kappa for multiple readers.