Towards automatic recognition of pure and mixed stones using intra-operative endoscopic digital images

Predictive Modeling of Urinary Stone Composition Using Machine Learning and Clinical Data: Implications for Treatment Strategies and Pathophysiological Insights

Purpose: Preventative strategies and surgical treatments for urolithiasis depend on stone composition. However, stone composition is often unknown until the stone is passed or surgically managed. Given that stone composition likely reflects the physiological parameters during its formation, we used clinical data from stone formers to predict stone composition. Materials and Methods: Data on stone composition, 24-hour urine, serum biochemistry, patient demographics, and medical history were prospectively collected from 777 kidney stone patients. Data were used to train gradient boosted machine and logistic regression models to distinguish calcium vs noncalcium, calcium oxalate monohydrate vs dihydrate, and calcium oxalate vs calcium phosphate vs uric acid stone types. Model performance was evaluated using the kappa score, and the influence of each predictor variable was assessed. Results: The calcium vs noncalcium model differentiated stone types with a kappa of 0.5231. The most influential predictors were 24-hour urine calcium, blood urate, and phosphate. The calcium oxalate monohydrate vs dihydrate model is the first of its kind and could discriminate stone types with a kappa of 0.2042. The key predictors were 24-hour urine urea, calcium, and oxalate. The multiclass model had a kappa of 0.3023 and the top predictors were age and 24-hour urine calcium and creatinine. Conclusions: Clinical data can be leveraged with machine learning algorithms to predict stone composition, which may help urologists determine stone type and guide their management plan before stone treatment. Investigating the most influential predictors of each classifier may improve the understanding of key clinical features of urolithiasis and shed light on pathophysiology.

Illumination matters part I: comparative analysis of light sources and illumination in flexible ureteroscopy-fundamental findings from a PEARLS analysis

PURPOSE

Illumination characteristics of flexible ureteroscopes have been evaluated in air, but not in saline, the native operative medium for endourology. The aim was to evaluate light properties of contemporary ureteroscopes in air versus saline, light distribution analysis, and color temperature.

METHODS

We evaluated the Storz Flex-Xc and Flex-X2s, Olympus V3 and P7, Pusen 7.5F and 9.2F, and OTU WiScope using a 3D printed black target board in-vitro model submerged in saline. A spectrometer was used for lux and color temperature measurements at different opening locations.

RESULTS

Illuminance was higher in saline compared to air (5679 vs. 5205 lx with Flex-Xc, p = 0.02). Illuminance in saline differed between ureteroscopes (ANOVA p < 0.001), with highest for the Flex-Xc at 100% brightness setting (5679 lx), followed by Pusen 9.2F (5280 lx), Flex-X2s (4613 lx), P7 (4371 lx), V3 (2374 lx), WiScope (582 lx) and finally Pusen 7.5F (255 lx). The same ranking was found at 50% brightness setting, with the highest ureteroscope illuminance value 34 times that of the scope with lowest illuminance. Most scopes had maximum illuminance off center, with skewness. Three scopes had two light sources, with one light source for all other scopes. Inter-scope comparisons revealed significant differences of color temperature (ANOVA p < 0.001).

CONCLUSION

The study demonstrates the presence of inhomogeneous light spread as well as large differences in illumination properties of ureteroscopes, possibly impacting on the performance of individual scopes in vivo. Additionally, the study suggests that future studies on illumination characteristics of flexible ureteroscopes should ideally be done in saline, and no longer in air.

Illumination matters Part II: advanced comparative analysis of flexible ureteroscopes in a kidney model by PEARLS

PURPOSE

The aim of the study was to evaluate illumination properties in an in-vitro kidney calyx model in saline.

DESIGN AND METHODS

We evaluated a series of contemporary flexible ureteroscopes including the Storz Flex-Xc and Flex-X2s, Olympus V3 and P7, Pusen 7.5F and 9.2F, as well as OTU WiScope using a 3D-printed closed pink kidney calyx model, submerged in saline. A spectrometer was used for illuminance and color temperature measurements at different openings located at center (direct light), 45° (direct and indirect light) and 90°(indirect light) to the axis of the scope.

RESULTS

Maximum illuminance was at the center opening for all scopes (range: 284 to 12,058 lx at 50% brightness and 454 to 11,871 lx at 100% brightness settings). The scope with the highest center illuminance (Flex-Xc) was 26 times superior to the scope with the lowest illuminance (Pusen 7.5Fr) at 100% brightness setting. For each scope, there was a peripheral illuminance drop ranging from - 43 to - 92% at 50% brightness and - 43% to - 88% at 100% brightness settings, respectively (all p < 0.01). Highest drop was for the P7 and the Pusen 9.2F. All scopes had illuminance skew, except the V3. All scopes had a warm color temperature.

CONCLUSION

Illumination properties vary between ureteroscopes in an enclosed cavity in saline, and differs at center vs 45° and 90° positions within scopes. Peripheral illuminance drop can be as high as - 92%, which is undesirable. This may affect the choice of ureteroscope and light brightness settings used in surgery by urologists.

Surgical Artificial Intelligence: Endourology

Endourology is ripe with information that includes patient factors, laboratory tests, outcomes, and visual data, which is becoming increasingly complex to assess. Artificial intelligence (AI) has the potential to explore and define these relationships; however, humans might not be involved in the input, analysis, or even determining the methods of analysis. Herein, the authors present the current state of AI in endourology and highlight the need for urologists to share their proposed AI solutions for reproducibility outside of their institutions and prepare themselves to properly critique this new technology.

Advancements in stone classification: unveiling the beauty of urolithiasis

PURPOSE

Urolithiasis has become increasingly prevalent, leading to higher disability-adjusted life years and deaths. Various stone classification systems have been developed to enhance the understanding of lithogenesis, aid urologists in treatment decisions, and predict recurrence risk. The aim of this manuscript is to provide an overview of different stone classification criteria.

METHODS

Two authors conducted a review of literature on studies relating to the classification of urolithiasis. A narrative synthesis for analysis of the studies was used.

RESULTS

Stones can be categorized based on anatomical position, size, medical imaging features, risk of recurrence, etiology, composition, and morphoconstitutional analysis. The first three mentioned offer a straightforward approach to stone classification, directly influencing treatment recommendations. With the routine use of CT imaging before treatment, precise details like anatomical location, stone dimensions, and Hounsfield Units can be easily determined, aiding treatment planning. In contrast, classifying stones based on risk of recurrence and etiology is more complex due to dependencies on multiple variables, including stone composition and morphology. A classification system based on morphoconstitutional analysis, which combines morphological stone appearance and chemical composition, has demonstrated its value. It allows for the rapid identification of crystalline phase principles, the detection of crystalline conversion processes, the determination of etiopathogenesis, the recognition of lithogenic processes, the assessment of crystal formation speed, related recurrence rates, and guidance for selecting appropriate treatment modalities.

CONCLUSIONS

Recognizing that no single classification system can comprehensively cover all aspects, the integration of all classification approaches is essential for tailoring urolithiasis patient-specific management.

Predicting stone composition via machine-learning models trained on intra-operative endoscopic digital images

OBJECTIVES

The aim of this study was to use deep learning (DL) of intraoperative images of urinary stones to predict the composition of urinary stones. In this way, the laser frequency and intensity can be adjusted in real time to reduce operation time and surgical trauma.

MATERIALS AND METHODS

A total of 490 patients who underwent holmium laser surgery during the two-year period from March 2021 to March 2023 and had stone analysis results were collected by the stone laboratory. A total of 1658 intraoperative stone images were obtained. The eight stone categories with the highest number of stones were selected by sorting. Single component stones include calcium oxalate monohydrate (W1), calcium oxalate dihydrate (W2), magnesium ammonium phosphate hexahydrate, apatite carbonate (CH) and anhydrous uric acid (U). Mixed stones include W2 + U, W1 + W2 and W1 + CH. All stones have intraoperative videos. More than 20 intraoperative high-resolution images of the stones, including the surface and core of the stones, were available for each patient via FFmpeg command screenshots. The deep convolutional neural network (CNN) ResNet-101 (ResNet, Microsoft) was applied to each image as a multiclass classification model.

RESULTS

The composition prediction rates for each component were as follows: calcium oxalate monohydrate 99% (n = 142), calcium oxalate dihydrate 100% (n = 29), apatite carbonate 100% (n = 131), anhydrous uric acid 98% (n = 57), W1 + W2 100% (n = 82), W1 + CH 100% ( n = 20) and W2 + U 100% (n = 24). The overall weighted recall of the cellular neural network component analysis for the entire cohort was 99%.

CONCLUSION

This preliminary study suggests that DL is a promising method for identifying urinary stone components from intraoperative endoscopic images. Compared to intraoperative identification of stone components by the human eye, DL can discriminate single and mixed stone components more accurately and quickly. At the same time, based on the training of stone images in vitro, it is closer to the clinical application of stone images in vivo. This technology can be used to identify the composition of stones in real time and to adjust the frequency and energy intensity of the holmium laser in time. The prediction of stone composition can significantly shorten the operation time, improve the efficiency of stone surgery and prevent the risk of postoperative infection.

2022 Recommendations of the AFU Lithiasis Committee: Laser - utilization and settings

Endocorporeal lithotripsy has progressed thanks to the development of lasers. Two laser sources are currently available: Holmium:YAG (Ho:YAG) and more recently Thulium Fiber Laser (TFL). The settings generally used are dusting, fragmentation, and "pop-corning". These are the first recommendations on laser use for stone management and their settings. Settings must be modulated and can be changed during the treatment according to the expected and obtained effects, the location and stone type that is treated. METHODOLOGY: These recommendations have been developed using two methods: the Clinical Practice Recommendation (CPR) method and the ADAPTE method, depending on whether or not the question was considered in the European Association of Urology (EAU) recommendations (https://uroweb.org/guidelines/urolithiasis [EAU Guidelines on urolithiasis. 2022]) and their adaptability to the French context.

2022 Recommendations of the AFU Lithiasis Committee: Endoscopic description of renal papillae and stones

Endoscopic observation is performed during treatments by flexible ureteroscopy to differentiate in situ between renal papillary abnormalities and stones based on their concordance with Daudon's morphological/composition descriptions adapted to endoscopy. These intraoperative visual analyses are now an integral part of the urinary stone disease diagnostic approach in addition to the morphological/structural and spectrophotometric analysis that remains the reference exam, but that loses information on the stone component representativeness due to the development of in situ laser lithotripsy. These are the first practical recommendations on the endoscopic description of renal papillae and stones. METHODOLOGY: These recommendations were developed using two methods: the Clinical Practice Recommendations (CPR) and the ADAPTE method, depending on whether the question was considered in the European Association of Urology (EAU) recommendations (https://uroweb.org/guidelines/urolithiasis [EAU Guidelines on urolithiasis. 2022]) and their adaptability to the French context.

Predicting Urinary Stone Composition in Single-Use Flexible Ureteroscopic Images with a Convolutional Neural Network

Background and Objectives: Analysis of urine stone composition is one of the most important factors in urolithiasis treatment. This study investigated whether a convolutional neural network (CNN) can show decent results in predicting urinary stone composition even in single-use flexible ureterorenoscopic (fURS) images with relatively low resolution. Materials and Methods: This study retrospectively used surgical images from fURS lithotripsy performed by a single surgeon between January 2018 and December 2021. The ureterorenoscope was a single-use flexible ureteroscope (LithoVue, Boston Scientific). Among the images taken during surgery, a single image satisfying the inclusion and exclusion criteria was selected for each stone. Cases were divided into two groups according to whether they contained any calcium oxalate (the Calcium group) or none (the Non-calcium group). From 506 total cases, 207 stone surface images were finally included in the study. In the CNN model, the transfer learning method using Resnet-18 as a pre-trained model was used, and only endoscopic digital images and stone classification data were input to achieve minimally supervised learning. Results: There were 175 cases in the Calcium group and 32 in the Non-calcium group. After training and validation, the model was tested using the test set, and the total accuracy was 81.8%. Recall and precision of the test results were 88.2% and 88.2% in the Calcium group and 60.0% and 60.0% in the Non-calcium group, respectively. The area under the receiver operating characteristic curve of the model, which represents its classification performance, was 0.82. Conclusions: Single-use flexible ureteroscopes have financial benefits but low vision quality compared with reusable digital flexible ureteroscopes. As far as we know, this is the first artificial intelligence study using single-use fURS images. It is meaningful that the CNN performed well even under these difficult conditions because these results can further expand the possibilities of its use.

Fifty years of basic and clinical renal stone research: have we achieved major breakthroughs? A debate

PURPOSE OF REVIEW

After 50 years of basic and clinical renal stone research, it is appropriate to evaluate whether breakthroughs have been achieved and if so, how they may be harnessed to combat stone disease therapeutically and prophylactically.

RECENT FINDINGS

Regarding stone therapeutics and prophylaxis, recent innovative studies are sparse. Researchers have resorted to publishing articles derived from data mining. Stone incidence and prevalence have increased during the past 50 years, suggesting the absence of any major breakthroughs. However, new sciences and technologies have created fresh opportunities. Information technology stores huge epidemiological databases leading to identification of new risk factors. Genetic coding has prompted identification of monogenic diseases associated with urolithiasis. Genome-wide association studies in combination with epigenomics, transcriptomics, proteomics, and metabolomics are providing new insights. High-throughput and culture-independent techniques promise to define the impact of microbiome on stone formation while artificial intelligent techniques contribute to diagnosis and prediction of treatment outcomes. These technologies, as well as those which are advancing surgical treatment of stones represent major breakthroughs in stone research.

SUMMARY

Although efforts to cure stones have not yielded major breakthroughs, technological advances have improved surgical management of this disease and represent significant headway in applied stone research.

Deep morphological recognition of kidney stones using intra-operative endoscopic digital videos

Objective. To assess the performance and added value of processing complete digital endoscopic video sequences for the automatic recognition of stone morphological features during a standard-of-care intra-operative session.Approach. A computer-aided video classifier was developed to predict in-situ the morphology of stone using an intra-operative digital endoscopic video acquired in a clinical setting. Using dedicated artificial intelligence (AI) networks, the proposed pipeline selects adequate frames in steady sequences of the video, ensures the presence of (potentially fragmented) stones and predicts the stone morphologies on a frame-by-frame basis. The automatic endoscopic stone recognition (A-ESR) is subsequently carried out by mixing all collected morphological observations.Main results. The proposed technique was evaluated on pure (i.e. include one morphology) and mixed (i.e. include at least two morphologies) stones involving ‘Ia/Calcium Oxalate Monohydrate’ (COM), ‘IIb/Calcium Oxalate Dihydrate’ (COD) and ‘IIIb/Uric Acid’ (UA) morphologies. The gold standard ESR was provided by a trained endo-urologist and confirmed by microscopy and infra-red spectroscopy. For the AI-training, 585 static images were collected (349 and 236 observations of stone surface and section, respectively) and used. Using the proposed video classifier, 71 digital endoscopic videos were analyzed: 50 exhibited only one morphological type and 21 displayed two. Taken together, both pure and mixed stone types yielded a mean diagnostic performances as follows: balanced accuracy = [88 ± 6] (min = 81)%, sensitivity = [80 ± 13] (min = 69)%, specificity = [95 ± 2] (min = 92)%, precision = [78 ± 12] (min = 62)% and F1-score = [78 ± 7] (min = 69)%.Significance. These results demonstrate that AI applied on digital endoscopic video sequences is a promising tool for collecting morphological information during the time-course of the stone fragmentation process without resorting to any human intervention for stone delineation or the selection of adequate steady frames.

How reliable is endoscopic stone recognition? A comparison between visual stone identification and formal stone analysis

OBJECTIVE

To assess the diagnostic accuracy and intra-observer agreement of endoscopic stone recognition compared with formal stone analysis.

INTRODUCTION

Stone analysis is a corner stone in the prevention of stone recurrence. Although X-ray diffraction and infrared spectroscopy are the recommended techniques for reliable formal stone analysis, this is not always possible, and the process takes time and is costly. Endoscopic stone recognition could be an alternative as it would give immediate information on stone composition.

MATERIAL AND METHODS

Fifteen endourologists predicted stone composition based on 100 videos from ureterorenoscopy. Diagnostic accuracy was evaluated by comparing the prediction from visual assessment with stone analysis by X-ray diffraction. After 30 days, the videos were reviewed again in a random order to assess intra-observer agreement.

RESULTS

The median diagnostic accuracy for calcium oxalate monohydrate was of 54% in questionnaire 1 (Q1) and 59% in questionnaire 2 (Q2), whereas calcium oxalate dihydrate had a median diagnostic accuracy of 75% in Q1 and 50% in Q2. The diagnostic accuracy for calcium hydroxyphosphate was 10% in Q1 and 13% in Q2. The median diagnostic accuracy for calcium hydrogen phosphate dihydrate and calcium magnesium phosphate was 0% in both questionnaires. The median diagnostic accuracy for magnesium ammonium phosphate was in 20% in Q1 and 40% in Q2. The median diagnostic accuracy for uric acid was 22% in both questionnaires. Finally, there was a diagnostic accuracy of 60% in Q1 and 80% in Q2 for cystine. The intra-observer agreement ranged between 45-72%.

CONCLUSION

Diagnostic accuracy of endoscopic stone recognition is limited and intra-observer agreement is below the threshold of acceptable agreement.

Prediction of the composition of urinary stones using deep learning

PURPOSE

This study aimed to predict the composition of urolithiasis using deep learning from urinary stone images.

MATERIALS AND METHODS

We classified 1,332 stones into 31 classes according to the stone composition. The top 4 classes with a frequency of 110 or more (class 1: calcium oxalate monohydrate [COM] 100%, class 2: COM 80%+struvite 20%, class 3: COM 60%+calcium oxalate dihydrate [COD] 40%, class 4: uric acid 100%) were selected. With the 965 stone images of the top 4 classes, we used the seven convolutional neural networks (CNN) to classify urinary stones and compared their classification performances.

RESULTS

Among the seven models, Xception_Ir0.001 showed the highest accuracy, precision, and recall and was selected as the CNN model to predict the stone composition. The sensitivity and specificity for the 4 classes by Xception_Ir0.001 were as follows: class 1 (94.24%, 91.73%), class 2 (85.42%, 96.14%), class 3 (86.86%, 99.59%), and class 4 (94.96%, 98.82%). The sensitivity and specificity of the individual components of the stones were as follows. COM (98.82%, 94.96%), COD (86.86%, 99.64%), struvite (85.42%, 95.59%), and uric acid (94.96%, 98.82%). The area under the curves for class 1, 2, 3, and 4 were 0.98, 0.97, 1.00, and 1.00, respectively.

CONCLUSIONS

This study showed the feasibility of deep learning for the diagnostic ability to assess urinary stone composition from images. It can be an alternative tool for conventional stone analysis and provide decision support to urologists, improving the effectiveness of diagnosis and treatment.