Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound

The use of deep learning in interventional radiotherapy (brachytherapy): A review with a focus on open source and open data

Deep learning advanced to one of the most important technologies in almost all medical fields. Especially in areas, related to medical imaging it plays a big role. However, in interventional radiotherapy (brachytherapy) deep learning is still in an early phase. In this review, first, we investigated and scrutinised the role of deep learning in all processes of interventional radiotherapy and directly related fields. Additionally, we summarised the most recent developments. For better understanding, we provide explanations of key terms and approaches to solving common deep learning problems. To reproduce results of deep learning algorithms both source code and training data must be available. Therefore, a second focus of this work is on the analysis of the availability of open source, open data and open models. In our analysis, we were able to show that deep learning plays already a major role in some areas of interventional radiotherapy, but is still hardly present in others. Nevertheless, its impact is increasing with the years, partly self-propelled but also influenced by closely related fields. Open source, data and models are growing in number but are still scarce and unevenly distributed among different research groups. The reluctance in publishing code, data and models limits reproducibility and restricts evaluation to mono-institutional datasets. The conclusion of our analysis is that deep learning can positively change the workflow of interventional radiotherapy but there is still room for improvements when it comes to reproducible results and standardised evaluation methods.

The effect of deep learning-based lesion segmentation on failure load calculations of metastatic femurs using finite element analysis

Bone ranks as the third most frequent tissue affected by cancer metastases, following the lung and liver. Bone metastases are often painful and may result in pathological fracture, which is a major cause of morbidity and mortality in cancer patients. To quantify fracture risk, finite element (FE) analysis has shown to be a promising tool, but metastatic lesions are typically not specifically segmented and therefore their mechanical properties may not be represented adequately. Deep learning methods potentially provide the opportunity to automatically segment these lesions and change the mechanical properties more adequately. In this study, our primary focus was to gain insight into the performance of an automatic segmentation algorithm for femoral metastatic lesions using deep learning methods and the subsequent effects on FE outcomes. The aims were to determine the similarity between manual segmentation and automatic segmentation; the differences in predicted failure load between FE models with automatically segmented osteolytic and mixed lesions and the models with CT-based lesion values (the gold standard); and the effect on the BOne Strength (BOS) score (failure load adjusted for body weight) and subsequent fracture risk assessments. From two patient cohorts, a total number of 50 femurs with osteolytic and mixed metastatic lesions were included in this study. The femurs were segmented from CT images and transferred into FE meshes. The material behavior was implemented as non-linear isotropic. These FE models were considered as gold standard (Finite Element no Segmented Lesion: FE-no-SL), whereby the local calcium equivalent density of both femur and metastatic lesion was extracted from CT-values. Lesions in the femur were manually segmented by two biomechanical experts after which final lesion segmentation for each femur was obtained based on consensus of opinions between two observers. Subsequently, a self-configuring variant of the popular deep learning model U-Net known as nnU-Net was used to automatically segment metastatic lesions within the femur. For these models with segmented lesions (Finite Element with Segmented Lesion: FE-with-SL), the calcium equivalent density within the metastatic lesions was set to zero after being segmented by the neural network, simulating absence of load-bearing capacity of these lesions. The models (either with or without automatically segmented lesions) were loaded incrementally in axial direction until failure was simulated. Dice coefficient was used to evaluate the similarity of the manual and automatic segmentation. Mean calcium equivalent density values within the automatically segmented lesions were calculated. Failure loads and patterns were determined. Furthermore, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for both groups by comparing the predictions to the occurrence or absence of actual fracture within the patient cohorts. The automatic segmentation algorithm performed in a none-robust manner. Dice coefficients describing the similarity between consented manual and automatic segmentations were relatively low (mean 0.45 ± standard deviation 0.33, median 0.54). Failure load difference between the FE-no-SL and FE-with-SL groups varied from 0 % to 48 % (mean 6.6 %). Correlation analysis of failure loads between the two groups showed a strong relationship (R2 > 0.9). From the 50 cases, four cases showed clear deviations for which models with automatic lesion segmentation (FE-with-SL) showed considerably lower failure loads. In the whole database including osteolytic and mixed lesions, sensitivity and NPV remained the same, but specificity and PPV decreased from 94 % to 83 %, and from 78 % to 54 % respectively from FE-no-SL to FE-with-SL. This study indicates that the nnU-Net yielded none-robust outcomes in femoral lesion segmentation and that other segmentation algorithms should be considered. However, the difference in failure pattern and failure load between FE models with automatically segmented osteolytic and mixed lesions were relatively small in most cases with a few exceptions. On the other hand, the accuracy of fracture risk assessment using the BOS score was lower compared to the FE-no-SL. In conclusion, this study showed that automatic lesion segmentation is a none-solved issue and therefore, quantifying lesion characteristics and the subsequent effect on the fracture risk using deep learning will remain challenging.

Artificial Intelligence in Andrology: From Semen Analysis to Image Diagnostics

Artificial intelligence (AI) in medicine has gained a lot of momentum in the last decades and has been applied to various fields of medicine. Advances in computer science, medical informatics, robotics, and the need for personalized medicine have facilitated the role of AI in modern healthcare. Similarly, as in other fields, AI applications, such as machine learning, artificial neural networks, and deep learning, have shown great potential in andrology and reproductive medicine. AI-based tools are poised to become valuable assets with abilities to support and aid in diagnosing and treating male infertility, and in improving the accuracy of patient care. These automated, AI-based predictions may offer consistency and efficiency in terms of time and cost in infertility research and clinical management. In andrology and reproductive medicine, AI has been used for objective sperm, oocyte, and embryo selection, prediction of surgical outcomes, cost-effective assessment, development of robotic surgery, and clinical decision-making systems. In the future, better integration and implementation of AI into medicine will undoubtedly lead to pioneering evidence-based breakthroughs and the reshaping of andrology and reproductive medicine.

Boundary delineation in transrectal ultrasound images for region of interest of prostate

Accurate and robust prostate segmentation in transrectal ultrasound (TRUS) images is of great interest for ultrasound-guided brachytherapy for prostate cancer. However, the current practice of manual segmentation is difficult, time-consuming, and prone to errors. To overcome these challenges, we developed an accurate prostate segmentation framework (A-ProSeg) for TRUS images. The proposed segmentation method includes three innovation steps: (1) acquiring the sequence of vertices by using an improved polygonal segment-based method with a small number of radiologist-defined seed points as prior points; (2) establishing an optimal machine learning-based method by using the improved evolutionary neural network; and (3) obtaining smooth contours of the prostate region of interest using the optimized machine learning-based method. The proposed method was evaluated on 266 patients who underwent prostate cancer brachytherapy. The proposed method achieved a high performance against the ground truth with a Dice similarity coefficient of 96.2% ± 2.4%, a Jaccard similarity coefficient of 94.4% ± 3.3%, and an accuracy of 95.7% ± 2.7%; these values are all higher than those obtained using state-of-the-art methods. A sensitivity evaluation on different noise levels demonstrated that our method achieved high robustness against changes in image quality. Meanwhile, an ablation study was performed, and the significance of all the key components of the proposed method was demonstrated.

The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

Radiomics based automated quality assessment for T2W prostate MR images

PURPOSE

The guidelines for prostate cancer recommend the use of MRI in the prostate cancer pathway. Due to the variability in prostate MR image quality, the reliability of this technique in the detection of prostate cancer is highly variable in clinical practice. This leads to the need for an objective and automated assessment of image quality to ensure an adequate acquisition and hereby to improve the reliability of MRI. The aim of this study is to investigate the feasibility of Blind/referenceless image spatial quality evaluator (Brisque) and radiomics in automated image quality assessment of T2-weighted (T2W) images.

METHOD

Anonymized axial T2W images from 140 patients were scored for quality using a five-point Likert scale (low, suboptimal, acceptable, good, very good quality) in consensus by two readers. Images were dichotomized into clinically acceptable (very good, good and acceptable quality images) and clinically unacceptable (low and suboptimal quality images) in order to train and verify the model. Radiomics and Brisque features were extracted from a central cuboid volume including the prostate. A reduced feature set was used to fit a Linear Discriminant Analysis (LDA) model to predict image quality. Two hundred times repeated 5-fold cross-validation was used to train the model and test performance by assessing the classification accuracy, the discrimination accuracy as receiver operating curve - area under curve (ROC-AUC), and by generating confusion matrices.

RESULTS

Thirty-four images were classified as clinically unacceptable and 106 were classified as clinically acceptable. The accuracy of the independent test set (mean ± standard deviation) was 85.4 ± 5.5%. The ROC-AUC was 0.856 (0.851 - 0.861) (mean; 95% confidence interval).

CONCLUSIONS

Radiomics AI can automatically detect a significant portion of T2W images of suboptimal image quality. This can help improve image quality at the time of acquisition, thus reducing repeat scans and improving diagnostic accuracy.

Artificial intelligence applications in brachytherapy: A literature review

PURPOSE

Artificial intelligence (AI) has the potential to simplify and optimize various steps of the brachytherapy workflow, and this literature review aims to provide an overview of the work done in this field.

METHODS AND MATERIALS

We conducted a literature search in June 2022 on PubMed, Embase, and Cochrane for papers that proposed AI applications in brachytherapy.

RESULTS

A total of 80 papers satisfied inclusion/exclusion criteria. These papers were categorized as follows: segmentation (24), registration and image processing (6), preplanning (13), dose prediction and treatment planning (11), applicator/catheter/needle reconstruction (16), and quality assurance (10). AI techniques ranged from classical models such as support vector machines and decision tree-based learning to newer techniques such as U-Net and deep reinforcement learning, and were applied to facilitate small steps of a process (e.g., optimizing applicator selection) or even automate the entire step of the workflow (e.g., end-to-end preplanning). Many of these algorithms demonstrated human-level performance and offer significant improvements in speed.

CONCLUSIONS

AI has potential to augment, automate, and/or accelerate many steps of the brachytherapy workflow. We recommend that future studies adhere to standard reporting guidelines. We also stress the importance of using larger sample sizes and reporting results using clinically interpretable measures.

Ultrasound Prostate Segmentation Using Adaptive Selection Principal Curve and Smooth Mathematical Model

Accurate prostate segmentation in ultrasound images is crucial for the clinical diagnosis of prostate cancer and for performing image-guided prostate surgery. However, it is challenging to accurately segment the prostate in ultrasound images due to their low signal-to-noise ratio, the low contrast between the prostate and neighboring tissues, and the diffuse or invisible boundaries of the prostate. In this paper, we develop a novel hybrid method for segmentation of the prostate in ultrasound images that generates accurate contours of the prostate from a range of datasets. Our method involves three key steps: (1) application of a principal curve-based method to obtain a data sequence comprising data coordinates and their corresponding projection index; (2) use of the projection index as training input for a fractional-order-based neural network that increases the accuracy of results; and (3) generation of a smooth mathematical map (expressed via the parameters of the neural network) that affords a smooth prostate boundary, which represents the output of the neural network (i.e., optimized vertices) and matches the ground truth contour. Experimental evaluation of our method and several other state-of-the-art segmentation methods on datasets of prostate ultrasound images generated at multiple institutions demonstrated that our method exhibited the best capability. Furthermore, our method is robust as it can be applied to segment prostate ultrasound images obtained at multiple institutions based on various evaluation metrics.

Carotid Vessel-Wall-Volume Ultrasound Measurement via a UNet++ Ensemble Algorithm Trained on Small Data Sets

Vessel wall volume (VWV) is a 3-D ultrasound measurement for the assessment of therapy in patients with carotid atherosclerosis. Deep learning can be used to segment the media-adventitia boundary (MAB) and lumen-intima boundary (LIB) and to quantify VWV automatically; however, it typically requires large training data sets with expert manual segmentation, which are difficult to obtain. In this study, a UNet++ ensemble approach was developed for automated VWV measurement, trained on five small data sets (n = 30 participants) and tested on 100 participants with clinically diagnosed coronary artery disease enrolled in a multicenter CAIN trial. The Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), Pearson correlation coefficient (r), Bland-Altman plots and coefficient of variation (CoV) were used to evaluate algorithm segmentation accuracy, agreement and reproducibility. The UNet++ ensemble yielded DSCs of 91.07%-91.56% and 87.53%-89.44% and ASSDs of 0.10-0.11 mm and 0.33-0.39 mm for the MAB and LIB, respectively; the algorithm VWV measurements were correlated (r = 0.763-0.795, p < 0.001) with manual segmentations, and the CoV for VWV was 8.89%. In addition, the UNet++ ensemble trained on 30 participants achieved a performance similar to that of U-Net and Voxel-FCN trained on 150 participants. These results suggest that our approach could provide accurate and reproducible carotid VWV measurements using relatively small training data sets, supporting deep learning applications for monitoring atherosclerosis progression in research and clinical trials.

Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy

PURPOSE

The purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target.

METHODS AND MATERIALS

Our surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case.

RESULTS

In the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians.

CONCLUSIONS

We successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

Diagnostic Accuracy of AI for Opportunistic Screening of Abdominal Aortic Aneurysm in CT: A Systematic Review and Narrative Synthesis

In this review, we focused on the applicability of artificial intelligence (AI) for opportunistic abdominal aortic aneurysm (AAA) detection in computed tomography (CT). We used the academic search system PubMed as the primary source for the literature search and Google Scholar as a supplementary source of evidence. We searched through 2 February 2022. All studies on automated AAA detection or segmentation in noncontrast abdominal CT were included. For bias assessment, we developed and used an adapted version of the QUADAS-2 checklist. We included eight studies with 355 cases, of which 273 (77%) contained AAA. The highest risk of bias and level of applicability concerns were observed for the "patient selection" domain, due to the 100% pathology rate in the majority (75%) of the studies. The mean sensitivity value was 95% (95% CI 100-87%), the mean specificity value was 96.6% (95% CI 100-75.7%), and the mean accuracy value was 95.2% (95% CI 100-54.5%). Half of the included studies performed diagnostic accuracy estimation, with only one study having data on all diagnostic accuracy metrics. Therefore, we conducted a narrative synthesis. Our findings indicate high study heterogeneity, requiring further research with balanced noncontrast CT datasets and adherence to reporting standards in order to validate the high sensitivity value obtained.

Evaluation of an Object Detection Algorithm for Shrapnel and Development of a Triage Tool to Determine Injury Severity

Emergency medicine in austere environments rely on ultrasound imaging as an essential diagnostic tool. Without extensive training, identifying abnormalities such as shrapnel embedded in tissue, is challenging. Medical professionals with appropriate expertise are limited in resource-constrained environments. Incorporating artificial intelligence models to aid the interpretation can reduce the skill gap, enabling identification of shrapnel, and its proximity to important anatomical features for improved medical treatment. Here, we apply a deep learning object detection framework, YOLOv3, for shrapnel detection in various sizes and locations with respect to a neurovascular bundle. Ultrasound images were collected in a tissue phantom containing shrapnel, vein, artery, and nerve features. The YOLOv3 framework, classifies the object types and identifies the location. In the testing dataset, the model was successful at identifying each object class, with a mean Intersection over Union and average precision of 0.73 and 0.94, respectively. Furthermore, a triage tool was developed to quantify shrapnel distance from neurovascular features that could notify the end user when a proximity threshold is surpassed, and, thus, may warrant evacuation or surgical intervention. Overall, object detection models such as this will be vital to compensate for lack of expertise in ultrasound interpretation, increasing its availability for emergency and military medicine.