Diagnostic Agreement for High-Grade Urothelial Cell Carcinoma in Atypical Urine Cytology: A Nationwide Survey Reveals a Tendency for Overestimation in Specimens with an N/C Ratio Approaching 0.5

Large-scale validation study of an improved semiautonomous urine cytology assessment tool: AutoParis-X

BACKGROUND

Adopting a computational approach for the assessment of urine cytology specimens has the potential to improve the efficiency, accuracy, and reliability of bladder cancer screening, which has heretofore relied on semisubjective manual assessment methods. As rigorous, quantitative criteria and guidelines have been introduced for improving screening practices (e.g., The Paris System for Reporting Urinary Cytology), algorithms to emulate semiautonomous diagnostic decision-making have lagged behind, in part because of the complex and nuanced nature of urine cytology reporting.

METHODS

In this study, the authors report on the development and large-scale validation of a deep-learning tool, AutoParis-X, which can facilitate rapid, semiautonomous examination of urine cytology specimens.

RESULTS

The results of this large-scale, retrospective validation study indicate that AutoParis-X can accurately determine urothelial cell atypia and aggregate a wide variety of cell-related and cluster-related information across a slide to yield an atypia burden score, which correlates closely with overall specimen atypia and is predictive of Paris system diagnostic categories. Importantly, this approach accounts for challenges associated with the assessment of overlapping cell cluster borders, which improve the ability to predict specimen atypia and accurately estimate the nuclear-to-cytoplasm ratio for cells in these clusters.

CONCLUSIONS

The authors developed a publicly available, open-source, interactive web application that features a simple, easy-to-use display for examining urine cytology whole-slide images and determining the level of atypia in specific cells, flagging the most abnormal cells for pathologist review. The accuracy of AutoParis-X (and other semiautomated digital pathology systems) indicates that these technologies are approaching clinical readiness and necessitates full evaluation of these algorithms in head-to-head clinical trials.

Atypical urothelial cells classified according to the Paris System for Reporting Urinary Cytology: A 2-year experience with histological correlation from a Finnish tertiary care center-low rate and high risk of malignancy

BACKGROUND

The Paris System for Reporting Urinary Cytology (TPS) was issued to shift the focus of urine cytology to high-grade lesions to increase the diagnostic accuracy of urine cytology. The aim of this study was to evaluate the power of TPS in the atypical urothelial cells (AUC) category with histological correlation and follow-up.

METHODS

The data cohort consisted of 3741 voided urine samples collected during a 2-year period between January 2017 and December 2018. All samples were prospectively classified using TPS. This study focuses on the subset of 205 samples (5.5%) classified as AUC. All cytological and histological follow-up data were analyzed until 2019, and the time between each sampling was documented.

RESULTS

Of the 205 AUC cases, cytohistological correlation was possible in 97 (47.3%) cases. Of these, 36 (12.7%) were benign in histology, 27 (13.2%) were low-grade urothelial carcinomas, and 34 (16.6%) were high-grade urothelial carcinomas. Overall, the risk of malignancy was 29.8% for all cases in the AUC category, and 62.9% in the histologically confirmed cases. The risk of high-grade malignancy was 16.6% in all the AUC category samples and 35.1% in the histological follow-up group.

CONCLUSIONS

The performance of 5.5% AUC cases is considered good and within the limits proposed by TPS. TPS is widely accepted by cytotechnologists, cytopathologists, and clinicians; it improves communication and patient management.

Uncovering additional predictors of urothelial carcinoma from voided urothelial cell clusters through a deep learning-based image preprocessing technique

BACKGROUND

Urine cytology is commonly used as a screening test for high-grade urothelial carcinoma for patients with risk factors or hematuria and is an essential step in longitudinal monitoring of patients with previous bladder cancer history. However, the semisubjective nature of current reporting systems for urine cytology (e.g., The Paris System) can hamper reproducibility. For instance, the incorporation of urothelial cell clusters into the classification schema is still an item of debate and perplexity among expert cytopathologists because several previous works have disputed their diagnostic relevance.

METHODS

In this work, an automated preprocessing tool for urothelial cell cluster assessment was developed that divides urothelial cell clusters into meaningful components for downstream assessment (ie, population-based studies, workflow automation).

RESULTS

In this work, an automated preprocessing tool for urothelial cell cluster assessment was developed that divides urothelial cell clusters into meaningful components for downstream assessment (ie, population-based studies, workflow automation). Results indicate that cell cluster atypia (i.e., defined by whether the cell cluster harbored multiple atypical cells, thresholded by a minimum number of cells), cell border overlap and smoothness, and total number of clusters are important markers of specimen atypia when considering assessment of urothelial cell clusters.

CONCLUSIONS

Markers established through techniques to separate cell clusters may have wider applicability for the design and implementation of machine learning approaches for urine cytology assessment.

Perceptions of Paris: an international survey in preparation for The Paris System for Reporting Urinary Cytology 2.0 (TPS 2.0)

INTRODUCTION

An international panel of experts in the field of urinary cytopathology conducted a survey, supported by the American Society of Cytopathology, to seek opinions, gather evidence, and identify practice patterns regarding urinary cytology before and after the introduction of The Paris System for Reporting Urinary Cytopathology (TPS). Results from this survey were utilized in the development of the second edition of TPS (TPS-2.0).

MATERIALS AND METHODS

The study group, originally formed during the 2013 International Congress of Cytology, reconvened at the 2019 annual meeting of the American Society of Cytopathology. To prepare for the second edition of TPS, the group generated a survey that included 43 questions related to the taxonomy and practice of urinary cytology.

RESULTS

A total of 523 participant responses were collected, and 451 from 54 countries passed a qualifying screen. Three hundred ninety-four participants provided information about their work settings. Eighty-two percent (218/266) of responding participants use TPS. One hundred sixty-eight people who responded on their urinary cytology atypia rates reported an average decrease from 21.6% to 16%. Over three fourths of participants felt that the same criteria should be used for upper and lower tract interpretations and for instrumented and voided samples. There were varied opinions on addressing atypical squamous cells and suggestions for an expanded discussion of the issue to be included in TPS 2.0.

CONCLUSIONS

Results of the survey demonstrate strong support for TPS and show a decreased self-reported atypia rate in the laboratories using TPS. The majority of participants related that the criteria put forth for the reporting categories were user-friendly and applied with relative ease. The comment section of the survey included suggestions from the participants for further improvement of TPS. Results of this survey have been useful in fine-tuning and advancing TPS. They were considered along with recent literature to generate the second edition of TPS.

Evaluation of an artificial intelligence algorithm for assisting the Paris System in reporting urinary cytology: A pilot study

BACKGROUND

The Paris System for Reporting Urinary Cytology (TPS) has been shown to improve bladder cancer diagnosis. Advances in artificial intelligence (AI) may assist and improve the clinical workflow by applying TPS in routine diagnostic services.

METHODS

A deep-learning-based algorithm was developed to identify urothelial cancer candidate cells using whole-slide images (WSIs). In the testing cohort, 131 urine cytology slides were retrospectively retrieved and analyzed using this AI algorithm. The authors compared the performance of one cytopathologist and two cytotechnologists using AI-assisted digital urine cytology. Then, the AI-assisted WSIs were evaluated in the clinical workflow. The cytopathologist first made a diagnosis by reviewing the AI-inferred WSIs and quantitative data (nuclear-to-cytoplasmic ratio and nuclear size) for each sample. After a washout period, the same cytopathologist made a diagnosis for the same samples using direct microscopy. All diagnosis results were compared with the expert panel consensus.

RESULTS

The AI-assisted diagnosis by the two cytotechnologists and the one cytopathologist demonstrated performance results that were comparable to the expert panel consensus (sensitivity, 79.5% and 82.1% vs. 92.3%, respectively; specificity, 100% and 98.9% vs. 100%, respectively). Furthermore, the performance of the AI-assisted WSIs compared with the microscopic diagnosis by the cytopathologist demonstrated superior sensitivity (92.3% vs. 87.2%) and negative predictive value (96.8% vs. 94.8%). In addition, the AI-assisted reporting demonstrated near perfect agreement with the expert panel consensus (κ = 0.944) and the microscopic diagnosis (κ = 0.862).

CONCLUSIONS

The AI algorithm developed by the authors effectively assisted TPS-based reporting by providing AI-inferred WSIs and quantitative data.

The Paris System for Reporting Urinary Cytology: A Meta-Analysis

The Paris System (TPS) for Reporting Urinary Cytology is a standardized, evidence-based reporting system, comprising seven diagnostic categories: nondiagnostic, negative for high-grade urothelial carcinoma (NHGUC), atypical urothelial cells (AUC), suspicious for high-grade urothelial carcinoma (SHGUC), HGUC, low-grade urothelial neoplasm (LGUN), and other malignancies. This study aimed to calculate the pooled risk of high-grade malignancy (ROHM) of each category and demonstrate the diagnostic accuracy of urine cytology reported with TPS. Four databases (PubMed, Embase, Scopus, Web of Science) were searched. Specific inclusion and exclusion criteria were applied, while data were extracted and analyzed both qualitatively and quantitatively. The pooled ROHM was 17.70% for the nondiagnostic category (95% CI, 0.0650; 0.3997), 13.04% for the NHGUC (95% CI, 0.0932; 0.1796), 38.65% for the AUC (95% CI, 0.3042; 0.4759), 12.45% for the LGUN (95% CI, 0.0431; 0.3101), 76.89 for the SHGUC (95% CI, 0.7063; 0.8216), and 91.79% for the HGUC and other malignancies (95% CI, 0.8722; 0.9482). A summary ROC curve was created and the Area Under the Curve (AUC) was 0.849, while the pooled sensitivity was 0.669 (95% CI, 0.589; 0.741) and false-positive rate was 0.101 (95% CI, 0.063; 0.158). In addition, the pooled DOR of the included studies was 21.258 (95% CI, 14.336; 31.522). TPS assigns each sample into a diagnostic category linked with a specific ROHM, guiding clinical management.

The Dynamics of Pathology Dataset Creation Using Urine Cytology as an Example

INTRODUCTION

Dataset creation is one of the first tasks required for training AI algorithms but is underestimated in pathology. High-quality data are essential for training algorithms and data should be labelled accurately and include sufficient morphological diversity. The dynamics and challenges of labelling a urine cytology dataset using The Paris System (TPS) criteria are presented.

METHODS

2,454 images were labelled by pathologist consensus via video conferencing over a 14-day period. During the labelling sessions, the dynamics of the labelling process were recorded. Quality assurance images were randomly selected from images labelled in previous sessions within this study and randomly distributed throughout new labelling sessions. To assess the effect of time on the labelling process, the labelled set of images was split into 2 groups according to the median relative label time and the time taken to label images and intersession agreement were assessed.

RESULTS

Labelling sessions ranged from 24 m 11 s to 41 m 06 s in length, with a median of 33 m 47 s. The majority of the 2,454 images were labelled as benign urothelial cells, with atypical and malignant urothelial cells more sparsely represented. The time taken to label individual images ranged from 1 s to 42 s with a median of 2.9 s. Labelling times differed significantly among categories, with the median label time for the atypical urothelial category being 7.2 s, followed by the malignant urothelial category at 3.8 s and the benign urothelial category at 2.9 s. The overall intersession agreement for quality assurance images was substantial. The level of agreement differed among classes of urothelial cells - benign and malignant urothelial cell classes showed almost perfect agreement and the atypical urothelial cell class showed moderate agreement. Image labelling times seemed to speed up, and there was no evidence of worsening of intersession agreement with session time.

DISCUSSION/CONCLUSION

Important aspects of pathology dataset creation are presented, illustrating the significant resources required for labelling a large dataset. We present evidence that the time taken to categorise urine cytology images varies by diagnosis/class. The known challenges relating to the reproducibility of the AUC (atypical) category in TPS when compared to the NHGUC (benign) or HGUC (malignant) categories is also confirmed.