Toward improved endoscopic examination of urinary stones: a concordance study between endoscopic digital pictures vs microscopy

Surgical Artificial Intelligence: Endourology

Endourology is ripe with information that includes patient factors, laboratory tests, outcomes, and visual data, which is becoming increasingly complex to assess. Artificial intelligence (AI) has the potential to explore and define these relationships; however, humans might not be involved in the input, analysis, or even determining the methods of analysis. Herein, the authors present the current state of AI in endourology and highlight the need for urologists to share their proposed AI solutions for reproducibility outside of their institutions and prepare themselves to properly critique this new technology.

Advancements in stone classification: unveiling the beauty of urolithiasis

PURPOSE

Urolithiasis has become increasingly prevalent, leading to higher disability-adjusted life years and deaths. Various stone classification systems have been developed to enhance the understanding of lithogenesis, aid urologists in treatment decisions, and predict recurrence risk. The aim of this manuscript is to provide an overview of different stone classification criteria.

METHODS

Two authors conducted a review of literature on studies relating to the classification of urolithiasis. A narrative synthesis for analysis of the studies was used.

RESULTS

Stones can be categorized based on anatomical position, size, medical imaging features, risk of recurrence, etiology, composition, and morphoconstitutional analysis. The first three mentioned offer a straightforward approach to stone classification, directly influencing treatment recommendations. With the routine use of CT imaging before treatment, precise details like anatomical location, stone dimensions, and Hounsfield Units can be easily determined, aiding treatment planning. In contrast, classifying stones based on risk of recurrence and etiology is more complex due to dependencies on multiple variables, including stone composition and morphology. A classification system based on morphoconstitutional analysis, which combines morphological stone appearance and chemical composition, has demonstrated its value. It allows for the rapid identification of crystalline phase principles, the detection of crystalline conversion processes, the determination of etiopathogenesis, the recognition of lithogenic processes, the assessment of crystal formation speed, related recurrence rates, and guidance for selecting appropriate treatment modalities.

CONCLUSIONS

Recognizing that no single classification system can comprehensively cover all aspects, the integration of all classification approaches is essential for tailoring urolithiasis patient-specific management.

Endoscopic Stone Composition Identification: Is Accuracy Improved by Stone Appearance During Laser Lithotripsy?

OBJECTIVE

To evaluate if videos during laser lithotripsy increase accuracy and confidence of stone identification by urologists compared to still pictures.

METHODS

We obtained representative pictures and videos of 4 major stone types from 8 different patients during ureteroscopy with holmium laser lithotripsy. A REDCap survey was created and emailed to members of the Endourological Society. The survey included a picture followed by the corresponding video of each stone undergoing laser lithotripsy and additional clinical information. Each picture and video included multiple-choice questions about stone composition and response confidence level. Accuracy, confidence levels, and rates of rectification (change from incorrect to correct answer) or confounding (correct to incorrect) after watching videos were analyzed.

RESULTS

One hundred eighty-seven urologists responded to the survey. The accuracy rate of stone identification with pictures was 43.8% vs 46.1% with videos (P = .27). Accuracy for individual stones was low and highly variable. Video only improved accuracy for 1 cystine stone. After viewing videos, participants were more likely to rectify vs confound their answers. Urologists were more likely to be confident with videos than pictures alone (65.4% vs 53.7%, respectively; P <.001); however, confident answers were not more likely to yield accurate predictions with videos vs still pictures.

CONCLUSION

Stone identification by urologists is marginally improved with videos vs pictures alone. Overall, accuracy in stone identification is low irrespective of confidence level, picture, and lithotripsy video visualization. Urologists should be cautious in using endoscopic stone appearance to direct metabolic management.

2022 Recommendations of the AFU Lithiasis Committee: Endoscopic description of renal papillae and stones

Endoscopic observation is performed during treatments by flexible ureteroscopy to differentiate in situ between renal papillary abnormalities and stones based on their concordance with Daudon's morphological/composition descriptions adapted to endoscopy. These intraoperative visual analyses are now an integral part of the urinary stone disease diagnostic approach in addition to the morphological/structural and spectrophotometric analysis that remains the reference exam, but that loses information on the stone component representativeness due to the development of in situ laser lithotripsy. These are the first practical recommendations on the endoscopic description of renal papillae and stones. METHODOLOGY: These recommendations were developed using two methods: the Clinical Practice Recommendations (CPR) and the ADAPTE method, depending on whether the question was considered in the European Association of Urology (EAU) recommendations (https://uroweb.org/guidelines/urolithiasis [EAU Guidelines on urolithiasis. 2022]) and their adaptability to the French context.

2022 Recommendations of the AFU Lithiasis Committee: Epidemiology, stone analysis and composition

The incidence of urinary lithiasis is rising steadily in industrialized countries, and its prevalence in the general population of France is estimated at 10%. Renal colic accounts for 1-2% of emergency department consultations. At a time when the new LASER stone fragmentation techniques available to urologists will lead to ever finer in situ pulverization of stones, the exact identification of the compounds that form the stone is essential for etiological diagnosis. Constitutional analysis by infrared spectrophotometry or X-ray diffraction is therefore recommended, to be complemented by morphological typing of the calculi. METHODOLOGY: These recommendations have been drawn up using two methods: the Recommendation for Clinical Practice (RPC) method and the ADAPTE method, depending on whether or not the issue was considered in the EAU recommendations (https://uroweb.org/guidelines/urolithiasis) [EAU 2022] and their adaptability to the French context.

Predicting Urinary Stone Composition in Single-Use Flexible Ureteroscopic Images with a Convolutional Neural Network

Background and Objectives: Analysis of urine stone composition is one of the most important factors in urolithiasis treatment. This study investigated whether a convolutional neural network (CNN) can show decent results in predicting urinary stone composition even in single-use flexible ureterorenoscopic (fURS) images with relatively low resolution. Materials and Methods: This study retrospectively used surgical images from fURS lithotripsy performed by a single surgeon between January 2018 and December 2021. The ureterorenoscope was a single-use flexible ureteroscope (LithoVue, Boston Scientific). Among the images taken during surgery, a single image satisfying the inclusion and exclusion criteria was selected for each stone. Cases were divided into two groups according to whether they contained any calcium oxalate (the Calcium group) or none (the Non-calcium group). From 506 total cases, 207 stone surface images were finally included in the study. In the CNN model, the transfer learning method using Resnet-18 as a pre-trained model was used, and only endoscopic digital images and stone classification data were input to achieve minimally supervised learning. Results: There were 175 cases in the Calcium group and 32 in the Non-calcium group. After training and validation, the model was tested using the test set, and the total accuracy was 81.8%. Recall and precision of the test results were 88.2% and 88.2% in the Calcium group and 60.0% and 60.0% in the Non-calcium group, respectively. The area under the receiver operating characteristic curve of the model, which represents its classification performance, was 0.82. Conclusions: Single-use flexible ureteroscopes have financial benefits but low vision quality compared with reusable digital flexible ureteroscopes. As far as we know, this is the first artificial intelligence study using single-use fURS images. It is meaningful that the CNN performed well even under these difficult conditions because these results can further expand the possibilities of its use.

On the rocks: can urologists identify stone composition based on endoscopic images alone? A worldwide survey of urologists

PURPOSE

As part of the management of nephrolithiasis, determination of chemical composition of stones is important. Our objective in this study is to assess urologists' accuracy in making visual, intraoperative determinations of stone composition.

MATERIALS AND METHODS

We conducted a REDCap survey asking urologists to predict stone composition based on intraoperative images of 10 different pure-composition kidney stones of 7 different types: calcium oxalate monohydrate (COM), calcium oxalate dihydrate (COD), calcium phosphate (CP) apatite, CP brushite, uric acid (UA), struvite (ST) and cystine (CY). To evaluate experience, we examined specific endourologic training, years of experience, and number of ureteroscopy (URS) cases/week. A self-assessment of ability to identify stone composition was also required.

RESULTS

With a response rate of 26% (366 completed surveys out of 1,370 deliveries), the overall accuracy of our cohort was 44%. COM, ST, and COD obtained the most successful identification rates (65.9%, 55.7%, and 52.0%, respectively). The most frequent misidentified stones were CP apatite (10.7%) and CY (14.2%). Predictors of increased overall accuracy included self-perceived ability to determine composition and number of ureteroscopies per week, while years of experience did not show a positive correlation.

CONCLUSIONS

Although endoscopic stone recognition can be an important tool for surgeons, it is not reliable enough to be utilized as a single method for stone identification, suggesting that urologists need to refine their ability to successfully recognize specific stone compositions intraoperatively.

Evaluation and understanding of automated urinary stone recognition methods

OBJECTIVE

To assess the potential of automated machine-learning methods for recognizing urinary stones in endoscopy.

MATERIALS AND METHODS

Surface and section images of 123 urinary calculi (109 ex vivo and 14 in vivo stones) were acquired using ureteroscopes. The stones were more than 85% 'pure'. Six classes of urolithiasis were represented: Groups I (calcium oxalate monohydrate, whewellite), II (calcium oxalate dihydrate, weddellite), III (uric acid), IV (brushite and struvite stones), and V (cystine). The automated stone recognition methods that were developed for this study followed two types of approach: shallow classification methods and deep-learning-based methods. Their sensitivity, specificity and positive predictive value (PPV) were evaluated by simultaneously using stone surface and section images to classify them into one of the main morphological groups (subgroups were not considered in this study).

RESULTS

Using shallow methods (based on texture and colour criteria), relatively high sensitivity, specificity and PPV for the six classes were attained: 91%, 90% and 89%, respectively, for whewellite; 99%, 98% and 99% for weddellite; 88%, 89% and 88% for uric acid; 91%, 89% and 90% for struvite; 99%, 99% and 99% for cystine; and 94%, 98% and 99% for brushite. Using deep-learning methods, the sensitivity, specificity and PPV for each of the classes were as follows: 99%, 98% and 97% for whewellite; 98%, 98% and 98% for weddellite; 97%, 98% and 98% for uric acid; 97%, 97% and 96% for struvite; 99%, 99% and 99% for cystine; and 94%, 97% and 98% for brushite.

CONCLUSION

Endoscopic stone recognition is challenging, and few urologists have sufficient expertise to achieve a diagnosis performance comparable to morpho-constitutional analysis. This work is a proof of concept that artificial intelligence could be a solution, with promising results achieved for pure stones. Further studies on a larger panel of stones (pure and mixed) are needed to further develop these methods.

Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos

Objective. To assess the performance and added value of processing complete digital endoscopic video sequences for the automatic recognition of stone morphological features during a standard-of-care intra-operative session.Approach. A computer-aided video classifier was developed to predict in-situ the morphology of stone using an intra-operative digital endoscopic video acquired in a clinical setting. Using dedicated artificial intelligence (AI) networks, the proposed pipeline selects adequate frames in steady sequences of the video, ensures the presence of (potentially fragmented) stones and predicts the stone morphologies on a frame-by-frame basis. The automatic endoscopic stone recognition (A-ESR) is subsequently carried out by mixing all collected morphological observations.Main results. The proposed technique was evaluated on pure (i.e. include one morphology) and mixed (i.e. include at least two morphologies) stones involving ‘Ia/Calcium Oxalate Monohydrate’ (COM), ‘IIb/Calcium Oxalate Dihydrate’ (COD) and ‘IIIb/Uric Acid’ (UA) morphologies. The gold standard ESR was provided by a trained endo-urologist and confirmed by microscopy and infra-red spectroscopy. For the AI-training, 585 static images were collected (349 and 236 observations of stone surface and section, respectively) and used. Using the proposed video classifier, 71 digital endoscopic videos were analyzed: 50 exhibited only one morphological type and 21 displayed two. Taken together, both pure and mixed stone types yielded a mean diagnostic performances as follows: balanced accuracy = [88 ± 6] (min = 81)%, sensitivity = [80 ± 13] (min = 69)%, specificity = [95 ± 2] (min = 92)%, precision = [78 ± 12] (min = 62)% and F1-score = [78 ± 7] (min = 69)%.Significance. These results demonstrate that AI applied on digital endoscopic video sequences is a promising tool for collecting morphological information during the time-course of the stone fragmentation process without resorting to any human intervention for stone delineation or the selection of adequate steady frames.

How reliable is endoscopic stone recognition? A comparison between visual stone identification and formal stone analysis

OBJECTIVE

To assess the diagnostic accuracy and intra-observer agreement of endoscopic stone recognition compared with formal stone analysis.

INTRODUCTION

Stone analysis is a corner stone in the prevention of stone recurrence. Although X-ray diffraction and infrared spectroscopy are the recommended techniques for reliable formal stone analysis, this is not always possible, and the process takes time and is costly. Endoscopic stone recognition could be an alternative as it would give immediate information on stone composition.

MATERIAL AND METHODS

Fifteen endourologists predicted stone composition based on 100 videos from ureterorenoscopy. Diagnostic accuracy was evaluated by comparing the prediction from visual assessment with stone analysis by X-ray diffraction. After 30 days, the videos were reviewed again in a random order to assess intra-observer agreement.

RESULTS

The median diagnostic accuracy for calcium oxalate monohydrate was of 54% in questionnaire 1 (Q1) and 59% in questionnaire 2 (Q2), whereas calcium oxalate dihydrate had a median diagnostic accuracy of 75% in Q1 and 50% in Q2. The diagnostic accuracy for calcium hydroxyphosphate was 10% in Q1 and 13% in Q2. The median diagnostic accuracy for calcium hydrogen phosphate dihydrate and calcium magnesium phosphate was 0% in both questionnaires. The median diagnostic accuracy for magnesium ammonium phosphate was in 20% in Q1 and 40% in Q2. The median diagnostic accuracy for uric acid was 22% in both questionnaires. Finally, there was a diagnostic accuracy of 60% in Q1 and 80% in Q2 for cystine. The intra-observer agreement ranged between 45-72%.

CONCLUSION

Diagnostic accuracy of endoscopic stone recognition is limited and intra-observer agreement is below the threshold of acceptable agreement.

Endoscopic Papillary Abnormalities and Stone Recognition (EPSR) during Flexible Ureteroscopy: A Comprehensive Review

Introduction: The increasing efficiency of the different lasers and the improved performance of endoscopic devices have led to smaller stone fragments that impact the accuracy of microscopic evaluation (morphological and infrared). Before the stone destruction, the urologist has the opportunity to analyze the stone and the papillary abnormalities endoscopically (endoscopic papillary recognition (EPR) and endoscopic stone recognition (ESR)). Our objective was to evaluate the value for those endoscopic descriptions. Methods: The MEDLINE and EMBASE databases were searched in February 2021 for studies on endoscopic papillary recognition and endoscopic stone recognition. Results: If the ESR provided information concerning the main crystallization process, EPR provided information concerning the origin of the lithogenesis and its severity. Despite many actual limitations, those complementary descriptions could support the preventive care of the stone formers in improving the diagnosis of the lithogenesis mechanism and in identifying high-risk stone formers. Conclusion: Until the development of an Artificial Intelligence recognition, the endourologist has to learn EPSR to minimize the distortion effect of the new lasers on the stone analysis and to improve care efficiency of the stone formers patients.

Towards automatic recognition of pure and mixed stones using intra-operative endoscopic digital images

OBJECTIVE

To assess automatic computer-aided in situ recognition of the morphological features of pure and mixed urinary stones using intra-operative digital endoscopic images acquired in a clinical setting.

MATERIALS AND METHODS

In this single-centre study, a urologist with 20 years' experience intra-operatively and prospectively examined the surface and section of all kidney stones encountered. Calcium oxalate monohydrate (COM) or Ia, calcium oxalate dihydrate (COD) or IIb, and uric acid (UA) or IIIb morphological criteria were collected and classified to generate annotated datasets. A deep convolutional neural network (CNN) was trained to predict the composition of both pure and mixed stones. To explain the predictions of the deep neural network model, coarse localization heat-maps were plotted to pinpoint key areas identified by the network.

RESULTS

This study included 347 and 236 observations of stone surface and stone section, respectively; approximately 80% of all stones exhibited only one morphological type and approximately 20% displayed two. A highest sensitivity of 98% was obtained for the type 'pure IIIb/UA' using surface images. The most frequently encountered morphology was that of the type 'pure Ia/COM'; it was correctly predicted in 91% and 94% of cases using surface and section images, respectively. Of the mixed type 'Ia/COM + IIb/COD', Ia/COM was predicted in 84% of cases using surface images, IIb/COD in 70% of cases, and both in 65% of cases. With regard to mixed Ia/COM + IIIb/UA stones, Ia/COM was predicted in 91% of cases using section images, IIIb/UA in 69% of cases, and both in 74% of cases.

CONCLUSIONS

This preliminary study demonstrates that deep CNNs are a promising method by which to identify kidney stone composition from endoscopic images acquired intra-operatively. Both pure and mixed stone composition could be discriminated. Collected in a clinical setting, surface and section images analysed by a deep CNN provide valuable information about stone morphology for computer-aided diagnosis.