Automatic multi-catheter detection using deeply supervised convolutional neural network in MRI-guided HDR prostate brachytherapy

A review of artificial intelligence in brachytherapy

Artificial intelligence (AI) has the potential to revolutionize brachytherapy's clinical workflow. This review comprehensively examines the application of AI, focusing on machine learning and deep learning, in various aspects of brachytherapy. We analyze AI's role in making brachytherapy treatments more personalized, efficient, and effective. The applications are systematically categorized into seven categories: imaging, preplanning, treatment planning, applicator reconstruction, quality assurance, outcome prediction, and real-time monitoring. Each major category is further subdivided based on cancer type or specific tasks, with detailed summaries of models, data sizes, and results presented in corresponding tables. Additionally, we discuss the limitations, challenges, and ethical concerns of current AI applications, along with perspectives on future directions. This review offers insights into the current advancements, challenges, and the impact of AI on treatment paradigms, encouraging further research to expand its clinical utility.

Clinical Application of Deep Learning-Assisted Needles Reconstruction in Prostate Ultrasound Brachytherapy

PURPOSE

High dose rate (HDR) prostate brachytherapy (BT) procedure requires image-guided needle insertion. Given that general anesthesia is often employed during the procedure, minimizing overall planning time is crucial. In this study, we explore the clinical feasibility and time-saving potential of artificial intelligence (AI)-driven auto-reconstruction of transperineal needles in the context of ultrasound (US)-guided prostate BT planning.

METHODS AND MATERIALS

This study included a total of 102 US-planned BT images from a single institution and split into 3 groups: 50 for model training and validation, 11 to evaluate reconstruction accuracy (test set), and 41 to evaluate the AI tool in a clinical implementation (clinical set). Reconstruction accuracy for the test set was evaluated by comparing the performance of AI-derived and manually reconstructed needles from 5 medical physicists on the 3D-US scans after treatment. The needle total reconstruction time for the clinical set was defined as the timestamp difference from scan acquisition to the start of dose calculations and was compared with values recorded before the clinical implementation of the AI-assisted tool.

RESULTS

A mean error of (0.44 ± 0.32) mm was found between the AI-reconstructed and the human consensus needle positions in the test set, with 95.0% of AI needle points falling below 1 mm from their human-made counterparts. Post-hoc analysis showed that only one of the human observers' reconstructions were significantly different from the others including the AIs. In the clinical set, the AI algorithm achieved a true positive reconstruction rate of 93.4% with only 4.5% of these needles requiring manual corrections from the planner before dosimetry. Total time required to perform AI-assisted catheter reconstruction on clinical cases was on average 15.2 min lower (P < .01) compared with procedure without AI assistance.

CONCLUSIONS

This study demonstrates the feasibility of an AI-assisted needle reconstructing tool for 3D-US-based HDR prostate BT. This is a step toward treatment planning automation and increased efficiency in HDR prostate BT.

Towards multimodal visualization of esophageal motility: fusion of manometry, impedance, and videofluoroscopic image sequences

PURPOSE

Dysphagia is the inability or difficulty to swallow normally. Standard procedures for diagnosing the exact disease are, among others, X-ray videofluoroscopy, manometry and impedance examinations, usually performed consecutively. In order to gain more insights, ongoing research is aiming to collect these different modalities at the same time, with the goal to present them in a joint visualization. One idea to create a combined view is the projection of the manometry and impedance values onto the right location in the X-ray images. This requires to identify the exact sensor locations in the images.

METHODS

This work gives an overview of the challenges associated with the sensor detection task and proposes a robust approach to detect the sensors in X-ray image sequences, ultimately allowing to project the manometry and impedance values onto the right location in the images.

RESULTS

The developed sensor detection approach is evaluated on a total of 14 sequences from different patients, achieving a F1-score of 86.36%. To demonstrate the robustness of the approach, another study is performed by adding different levels of noise to the images, with the performance of our sensor detection method only slightly decreasing in these scenarios. This robust sensor detection provides the basis to accurately project manometry and impedance values onto the images, allowing to create a multimodal visualization of the swallow process. The resulting visualizations are evaluated qualitatively by domain experts, indicating a great benefit of this proposed fused visualization approach.

CONCLUSION

Using our preprocessing and sensor detection method, we show that the sensor detection task can be successfully approached with high accuracy. This allows to create a novel, multimodal visualization of esophageal motility, helping to provide more insights into swallow disorders of patients.

Artificial intelligence in high-dose-rate brachytherapy treatment planning for cervical cancer: a review

Cervical cancer remains a significant global health concern, characterized by high morbidity and mortality rates. High-dose-rate brachytherapy (HDR-BT) is a critical component of cervical cancer treatment, requiring precise and efficient treatment planning. However, the process is labor-intensive, heavily reliant on operator expertise, and prone to variability due to factors such as applicator shifts and organ filling changes. Recent advancements in artificial intelligence (AI), particularly in medical image processing, offer significant potential for automating and standardizing treatment planning in HDR-BT. This review examines the progress and challenge of AI applications in HDR-BT treatment planning, focusing on automatic segmentation, applicator reconstruction, dose calculation, and plan optimization. By addressing current limitations and exploring future directions, this paper aims to guide the integration of AI into clinical practice, ultimately improving treatment accuracy, reducing preparation time, and enhancing patient outcomes.

Artificial intelligence in interventional radiotherapy (brachytherapy): Enhancing patient-centered care and addressing patients' needs

This review explores the integration of artificial intelligence (AI) in interventional radiotherapy (IRT), emphasizing its potential to streamline workflows and enhance patient care. Through a systematic analysis of 78 relevant papers spanning from 2002 to 2024, we identified significant advancements in contouring, treatment planning, outcome prediction, and quality assurance. AI-driven approaches offer promise in reducing procedural times, personalizing treatments, and improving treatment outcomes for oncological patients. However, challenges such as clinical validation and quality assurance protocols persist. Nonetheless, AI presents a transformative opportunity to optimize IRT and meet evolving patient needs.

A Review of Artificial Intelligence in Brachytherapy

Artificial intelligence (AI) has the potential to revolutionize brachytherapy's clinical workflow. This review comprehensively examines the application of AI, focusing on machine learning and deep learning, in facilitating various aspects of brachytherapy. We analyze AI's role in making brachytherapy treatments more personalized, efficient, and effective. The applications are systematically categorized into seven categories: imaging, preplanning, treatment planning, applicator reconstruction, quality assurance, outcome prediction, and real-time monitoring. Each major category is further subdivided based on cancer type or specific tasks, with detailed summaries of models, data sizes, and results presented in corresponding tables. This review offers insights into the current advancements, challenges, and the impact of AI on treatment paradigms, encouraging further research to expand its clinical utility.

Rapid multi-catheter segmentation for magnetic resonance image-guided catheter-based interventions

BACKGROUND

Magnetic resonance imaging (MRI) is the gold standard for delineating cancerous lesions in soft tissue. Catheter-based interventions require the accurate placement of multiple long, flexible catheters at the target site. The manual segmentation of catheters in MR images is a challenging and time-consuming task. There is a need for automated catheter segmentation to improve the efficiency of MR-guided procedures.

PURPOSE

To develop and assess a machine learning algorithm for the detection of multiple catheters in magnetic resonance images used during catheter-based interventions.

METHODS

In this work, a 3D U-Net was trained to retrospectively segment catheters in scans acquired during clinical MR-guided high dose rate (HDR) prostate brachytherapy cases. To assess confidence in segmentation, multiple AI models were trained. On clinical test cases, average segmentation results were used to plan the brachytherapy delivery. Dosimetric parameters were compared to the original clinical plan. Data was obtained from 35 patients who underwent HDR prostate brachytherapy for focal disease with a total of 214 image volumes. 185 image volumes from 30 patients were used for training using a five-fold cross validation split to divide the data for training and validation. To generate confidence measures of segmentation accuracy, five trained models were generated. The remaining five patients (29 volumes) were used to test the performance of the trained model by comparison to manual segmentations of three independent observers and assessment of dosimetric impact on the final clinical brachytherapy plans.

RESULTS

The network successfully identified 95% of catheters in the test set at a rate of 0.89 s per volume. The multi-model method identified the small number of cases where AI segmentation of individual catheters was poor, flagging the need for user input. AI-based segmentation performed as well as segmentations by independent observers. Plan dosimetry using AI-segmented catheters was comparable to the original plan.

CONCLUSION

The vast majority of catheters were accurately identified by AI segmentation, with minimal impact on plan outcomes. The use of multiple AI models provided confidence in the segmentation accuracy and identified catheter segmentations that required further manual assessment. Real-time AI catheter segmentation can be used during MR-guided insertions to assess deflections and for rapid planning of prostate brachytherapy.

The use of deep learning in interventional radiotherapy (brachytherapy): A review with a focus on open source and open data

Deep learning advanced to one of the most important technologies in almost all medical fields. Especially in areas, related to medical imaging it plays a big role. However, in interventional radiotherapy (brachytherapy) deep learning is still in an early phase. In this review, first, we investigated and scrutinised the role of deep learning in all processes of interventional radiotherapy and directly related fields. Additionally, we summarised the most recent developments. For better understanding, we provide explanations of key terms and approaches to solving common deep learning problems. To reproduce results of deep learning algorithms both source code and training data must be available. Therefore, a second focus of this work is on the analysis of the availability of open source, open data and open models. In our analysis, we were able to show that deep learning plays already a major role in some areas of interventional radiotherapy, but is still hardly present in others. Nevertheless, its impact is increasing with the years, partly self-propelled but also influenced by closely related fields. Open source, data and models are growing in number but are still scarce and unevenly distributed among different research groups. The reluctance in publishing code, data and models limits reproducibility and restricts evaluation to mono-institutional datasets. The conclusion of our analysis is that deep learning can positively change the workflow of interventional radiotherapy but there is still room for improvements when it comes to reproducible results and standardised evaluation methods.

Development trends and knowledge framework in the application of magnetic resonance imaging in prostate cancer: a bibliometric analysis from 1984 to 2022

Background

Prostate cancer (PCa) is the most common tumor of the male genitourinary system. With the development of imaging technology, the role of magnetic resonance imaging (MRI) in the management of PCa is increasing. The present study summarizes research on the application of MRI in the field of PCa using bibliometric analysis and predicts future research hotspots.

Methods

Articles regarding the application of MRI in PCa between January 1, 1984 and June 30, 2022 were selected from the Web of Science Core Collection (WoSCC) on November 6, 2022. Microsoft Excel 2016 and the Bibliometrix Biblioshiny R-package software were used for data analysis and bibliometric indicator extraction. CiteSpace (version 6.1.R3) was used to visualize literature feature clustering, including co-occurrence analysis of countries, institutions, authors, references, and burst keywords analysis.

Results

A total of 10,230 articles were included in the study. Turkbey was the most prolific author. The USA was the most productive country and had strong partnerships with other countries. The most productive institution was Memorial Sloan Kettering Cancer Center. Journal of Magnetic Resonance Imaging and Radiology were the most productive and highest impact factor (IF) journals in the field, respectively. Timeline views showed that "#1 multiparametric magnetic resonance imaging", "#4 pi-rads", and "#8 psma" were currently the latest research hotspots. Keywords burst analysis showed that "machine learning", "psa density", "multi parametric mri", "deep learning", and "artificial intelligence" were the most frequently used keywords in the past 3 years.

Conclusions

MRI has a wide range of applications in PCa. The USA is the leading country in this field, with a concentration of highly productive and high-level institutions. Meanwhile, it can be projected that "deep learning", "radiomics", and "artificial intelligence" will be research hotspots in the future.

Artificial intelligence applications in brachytherapy: A literature review

PURPOSE

Artificial intelligence (AI) has the potential to simplify and optimize various steps of the brachytherapy workflow, and this literature review aims to provide an overview of the work done in this field.

METHODS AND MATERIALS

We conducted a literature search in June 2022 on PubMed, Embase, and Cochrane for papers that proposed AI applications in brachytherapy.

RESULTS

A total of 80 papers satisfied inclusion/exclusion criteria. These papers were categorized as follows: segmentation (24), registration and image processing (6), preplanning (13), dose prediction and treatment planning (11), applicator/catheter/needle reconstruction (16), and quality assurance (10). AI techniques ranged from classical models such as support vector machines and decision tree-based learning to newer techniques such as U-Net and deep reinforcement learning, and were applied to facilitate small steps of a process (e.g., optimizing applicator selection) or even automate the entire step of the workflow (e.g., end-to-end preplanning). Many of these algorithms demonstrated human-level performance and offer significant improvements in speed.

CONCLUSIONS

AI has potential to augment, automate, and/or accelerate many steps of the brachytherapy workflow. We recommend that future studies adhere to standard reporting guidelines. We also stress the importance of using larger sample sizes and reporting results using clinically interpretable measures.

Automated treatment planning framework for brachytherapy of cervical cancer using 3D dose predictions

Objective. To lay the foundation for automated knowledge-based brachytherapy treatment planning using 3D dose estimations, we describe an optimization framework to convert brachytherapy dose distributions directly into dwell times (DTs).Approach. A dose rate kernelḋ(r,θ,φ)was produced by exporting 3D dose for one dwell position from the treatment planning system and normalizing by DT. By translating and rotating this kernel to each dwell position, scaling by DT and summing over all dwell positions, dose was computed (Dcalc). We used a Python-coded COBYLA optimizer to iteratively determine the DTs that minimize the mean squared error betweenDcalcand reference doseDref, computed using voxels withDref80%-120% of prescription. As validation of the optimization, we showed that the optimizer replicates clinical plans whenDref= clinical dose in 40 patients treated with tandem-and-ovoid (T&O) or tandem-and-ring (T&R) and 0-3 needles. Then we demonstrated automated planning in 10 T&O usingDref= dose predicted from a convolutional neural network developed in past work. Validation and automated plans were compared to clinical plans using mean absolute differences (MAD=1N∑n=1Nabsxn-xn') over all voxels (xn= Dose,N= #voxels) and DTs (xn= DT,N= #dwell positions), mean differences (MD) in organD2ccand high-risk CTV D90 over all patients (where positive indicates higher clinical dose), and mean Dice similarity coefficients (DSC) for 100% isodose contours.Main results. Validation plans agreed well with clinical plans (MADdose= 1.1%, MADDT= 4 s or 0.8% of total plan time,D2ccMD = -0.2% to 0.2% and D90 MD = -0.6%, DSC = 0.99). For automated plans, MADdose= 6.5% and MADDT= 10.3 s (2.1%). The slightly higher clinical metrics in automated plans (D2ccMD = -3.8% to 1.3% and D90 MD = -5.1%) were due to higher neural network dose predictions. The overall shape of the automated dose distributions were similar to clinical doses (DSC = 0.91).Significance. Automated planning with 3D dose predictions could provide significant time savings and standardize treatment planning across practitioners, regardless of experience.

Automatic reconstruction of interstitial needles using CT images in post-operative cervical cancer brachytherapy based on deep learning

Purpose

The purpose of this study was to investigate the precision of deep learning (DL)-based auto-reconstruction in localizing interstitial needles in post-operative cervical cancer brachytherapy (BT) using three-dimensional (3D) computed tomography (CT) images.

Material and methods

A convolutional neural network (CNN) was developed and presented for automatic reconstruction of interstitial needles. Data of 70 post-operative cervical cancer patients who received CT-based BT were used to train and test this DL model. All patients were treated with three metallic needles. Dice similarity coefficient (DSC), 95% Hausdorff distance (95% HD), and Jaccard coefficient (JC) were applied to evaluate the geometric accuracy of auto-reconstruction for each needle. Dose-volume indexes (DVI) between manual and automatic methods were used to analyze the dosimetric difference. Correlation between geometric metrics and dosimetric difference was evaluated using Spearman correlation analysis.

Results

The mean DSC values of DL-based model were 0.88, 0.89, and 0.90 for three metallic needles. Wilcoxon signed-rank test indicated no significant dosimetric differences in all BT planning structures between manual and automatic reconstruction methods (p > 0.05). Spearman correlation analysis demonstrated weak link between geometric metrics and dosimetry differences.

Conclusions

DL-based reconstruction method can be used to precisely localize the interstitial needles in 3D-CT images. The proposed automatic approach could improve the consistency of treatment planning for post-operative cervical cancer brachytherapy.

Deep Learning in MRI-guided Radiation Therapy: A Systematic Review

MRI-guided radiation therapy (MRgRT) offers a precise and adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed. MRI-guided radiation therapy offers a precise, adaptive approach to treatment planning. Deep learning applications which augment the capabilities of MRgRT are systematically reviewed with emphasis placed on underlying methods. Studies are further categorized into the areas of segmentation, synthesis, radiomics, and real time MRI. Finally, clinical implications, current challenges, and future directions are discussed.

MRI-based prostate and dominant lesion segmentation using cascaded scoring convolutional neural network

PURPOSE

Dose escalation to dominant intraprostatic lesions (DILs) is a novel treatment strategy to improve the treatment outcome of prostate radiation therapy. Treatment planning requires accurate and fast delineation of the prostate and DILs. In this study, a 3D cascaded scoring convolutional neural network is proposed to automatically segment the prostate and DILs from MRI.

METHODS AND MATERIALS

The proposed cascaded scoring convolutional neural network performs end-to-end segmentation by locating a region-of-interest (ROI), identifying the object within the ROI, and defining the target. A scoring strategy, which is learned to judge the segmentation quality of DIL, is integrated into cascaded convolutional neural network to solve the challenge of segmenting the irregular shapes of the DIL. To evaluate the proposed method, 77 patients who underwent MRI and PET/CT were retrospectively investigated. The prostate and DIL ground truth contours were delineated by experienced radiologists. The proposed method was evaluated with five-fold cross validation and holdout testing.

RESULTS

The average centroid distance, volume difference, and Dice similarity coefficient (DSC) value for prostate/DIL are 4.3±7.5mm/3.73±3.78mm, 4.5±7.9cc/0.41±0.59cc and 89.6±8.9%/84.3±11.9%, respectively. Comparable results were obtained in the holdout test. Similar or superior segmentation outcomes were seen when compared the results of the proposed method to those of competing segmentation approaches CONCLUSIONS: : The proposed automatic segmentation method can accurately and simultaneously segment both the prostate and DILs. The intended future use for this algorithm is focal boost prostate radiation therapy. This article is protected by copyright. All rights reserved.

Multi-organ auto-delineation in head-and-neck MRI for radiation therapy using regional convolutional neural network

Magnetic resonance imaging (MRI) allows accurate and reliable organ delineation for many disease sites in radiation therapy because MRI is able to offer superb soft-tissue contrast. Manual organ-at-risk delineation is labor-intensive and time-consuming. This study aims to develop a deep-learning-based automated multi-organ segmentation method to release the labor and accelerate the treatment planning process for head-and-neck (HN) cancer radiotherapy. A novel regional convolutional neural network (R-CNN) architecture, namely, mask scoring R-CNN, has been developed in this study. In the proposed model, a deep attention feature pyramid network is used as a backbone to extract the coarse features given by MRI, followed by feature refinement using R-CNN. The final segmentation is obtained through mask and mask scoring networks taking those refined feature maps as input. With the mask scoring mechanism incorporated into conventional mask supervision, the classification error can be highly minimized in conventional mask R-CNN architecture. A cohort of 60 HN cancer patients receiving external beam radiation therapy was used for experimental validation. Five-fold cross-validation was performed for the assessment of our proposed method. The Dice similarity coefficients of brain stem, left/right cochlea, left/right eye, larynx, left/right lens, mandible, optic chiasm, left/right optic nerve, oral cavity, left/right parotid, pharynx, and spinal cord were 0.89 ± 0.06, 0.68 ± 0.14/0.68 ± 0.18, 0.89 ± 0.07/0.89 ± 0.05, 0.90 ± 0.07, 0.67 ± 0.18/0.67 ± 0.10, 0.82 ± 0.10, 0.61 ± 0.14, 0.67 ± 0.11/0.68 ± 0.11, 0.92 ± 0.07, 0.85 ± 0.06/0.86 ± 0.05, 0.80 ± 0.13, and 0.77 ± 0.15, respectively. After the model training, all OARs can be segmented within 1 min.

Deep-learning-assisted algorithm for catheter reconstruction during MR-only gynecological interstitial brachytherapy

Magnetic resonance imaging (MRI) offers excellent soft‐tissue contrast enabling the contouring of targets and organs at risk during gynecological interstitial brachytherapy procedure. Despite its advantage, one of the main obstacles preventing a transition to an MRI‐only workflow is that implanted plastic catheters are not reliably visualized on MR images. This study aims to evaluate the feasibility of a deep‐learning‐based algorithm for semiautomatic reconstruction of interstitial catheters during an MR‐only workflow. MR images of 20 gynecological patients were used in this study. Note that 360 catheters were reconstructed using T1‐ and T2‐weighted images by five experienced brachytherapy planners. The mean of the five reconstructed paths were used for training (257 catheters), validation (15 catheters), and testing/evaluation (88 catheters). To automatically identify and localize the catheters, a two‐dimensional (2D) U‐net algorithm was used to find their approximate location in each image slice. Once localized, thresholding was applied to those regions to find the extrema, as catheters appear as bright and dark regions in T1‐ and T2‐weighted images, respectively. The localized dwell positions of the proposed algorithm were compared to the ground truth reconstruction. Reconstruction time was also evaluated. A total of 34 009 catheter dwell positions were evaluated between the algorithm and all planners to estimate the reconstruction variability. The average variation was 0.97 ± 0.66 mm. The average reconstruction time for this approach was 11 ± 1 min, compared with 46 ± 10 min for the expert planners. This study suggests that the proposed deep learning, MR‐based framework has potential to replace the conventional manual catheter reconstruction. The adoption of this approach in the brachytherapy workflow is expected to improve treatment efficiency while reducing planning time, resources, and human errors.

Prostate and dominant intraprostatic lesion segmentation on PET/CT using cascaded regional-net

Focal boost to dominant intraprostatic lesions (DILs) has recently been proposed for prostate radiation therapy. Accurate and fast delineation of the prostate and DILs is thus required during treatment planning. In this paper, we develop a learning-based method using positron emission tomography (PET)/computed tomography (CT) images to automatically segment the prostate and its DILs. To enable end-to-end segmentation, a deep learning-based method, called cascaded regional-Net, is utilized. The first network, referred to as dual attention network, is used to segment the prostate via extracting comprehensive features from both PET and CT images. A second network, referred to as mask scoring regional convolutional neural network (MSR-CNN), is used to segment the DILs from the PET and CT within the prostate region. Scoring strategy is used to diminish the misclassification of the DILs. For DIL segmentation, the proposed cascaded regional-Net uses two steps to remove normal tissue regions, with the first step cropping images based on prostate segmentation and the second step using MSR-CNN to further locate the DILs. The binary masks of DILs and prostates of testing patients are generated on the PET/CT images by the trained model. For evaluation, we retrospectively investigated 49 prostate cancer patients with PET/CT images acquired. The prostate and DILs of each patient were contoured by radiation oncologists and set as the ground truths and targets. We used five-fold cross-validation and a hold-out test to train and evaluate our method. The mean surface distance and DSC values were 0.666 ± 0.696 mm and 0.932 ± 0.059 for the prostate and 0.814 ± 1.002 mm and 0.801 ± 0.178 for the DILs among all 49 patients. The proposed method has shown promise for facilitating prostate and DIL delineation for DIL focal boost prostate radiation therapy.

Deep learning-based motion tracking using ultrasound images

PURPOSE

Ultrasound (US) imaging is an established imaging modality capable of offering video-rate volumetric images without ionizing radiation. It has the potential for intra-fraction motion tracking in radiation therapy. In this study, a deep learning-based method has been developed to tackle the challenges in motion tracking using US imaging.

METHODS

We present a Markov-like network, which is implemented via generative adversarial networks, to extract features from sequential US frames (one tracked frame followed by untracked frames) and thereby estimate a set of deformation vector fields (DVFs) through the registration of the tracked frame and the untracked frames. The positions of the landmarks in the untracked frames are finally determined by shifting landmarks in the tracked frame according to the estimated DVFs. The performance of the proposed method was evaluated on the testing dataset by calculating the tracking error (TE) between the predicted and ground truth landmarks on each frame.

RESULTS

The proposed method was evaluated using the MICCAI CLUST 2015 dataset which was collected using seven US scanners with eight types of transducers and the Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) dataset which was acquired using GE Vivid E95 ultrasound scanners. The CLUST dataset contains 63 2D and 22 3D US image sequences respectively from 42 and 18 subjects, and the CAMUS dataset includes 2D US images from 450 patients. On CLUST dataset, our proposed method achieved a mean tracking error of 0.70 ± 0.38 mm for the 2D sequences and 1.71 ± 0.84 mm for the 3D sequences for those public available annotations. And on CAMUS dataset, a mean tracking error of 0.54 ± 1.24 mm for the landmarks in the left atrium was achieved.

CONCLUSIONS

A novel motion tracking algorithm using US images based on modern deep learning techniques has been demonstrated in this study. The proposed method can offer millimeter-level tumor motion prediction in real time, which has the potential to be adopted into routine tumor motion management in radiation therapy.

Emerging technologies in brachytherapy

Brachytherapy is a mature treatment modality. The literature is abundant in terms of review articles and comprehensive books on the latest established as well as evolving clinical practices. The intent of this article is to part ways and look beyond the current state-of-the-art and review emerging technologies that are noteworthy and perhaps may drive the future innovations in the field. There are plenty of candidate topics that deserve a deeper look, of course, but with practical limits in this communicative platform, we explore four topics that perhaps is worthwhile to review in detail at this time. First, intensity modulated brachytherapy (IMBT) is reviewed. The IMBT takes advantage ofanisotropicradiation profile generated through intelligent high-density shielding designs incorporated onto sources and applicators such to achieve high quality plans. Second, emerging applications of 3D printing (i.e. additive manufacturing) in brachytherapy are reviewed. With the advent of 3D printing, interest in this technology in brachytherapy has been immense and translation swift due to their potential to tailor applicators and treatments customizable to each individual patient. This is followed by, in third, innovations in treatment planning concerning catheter placement and dwell times where new modelling approaches, solution algorithms, and technological advances are reviewed. And, fourth and lastly, applications of a new machine learning technique, called deep learning, which has the potential to improve and automate all aspects of brachytherapy workflow, are reviewed. We do not expect that all ideas and innovations reviewed in this article will ultimately reach clinic but, nonetheless, this review provides a decent glimpse of what is to come. It would be exciting to monitor as IMBT, 3D printing, novel optimization algorithms, and deep learning technologies evolve over time and translate into pilot testing and sensibly phased clinical trials, and ultimately make a difference for cancer patients. Today's fancy is tomorrow's reality. The future is bright for brachytherapy.

Synthetic CT-aided multiorgan segmentation for CBCT-guided adaptive pancreatic radiotherapy

PURPOSE

The delineation of organs at risk (OARs) is fundamental to cone-beam CT (CBCT)-based adaptive radiotherapy treatment planning, but is time consuming, labor intensive, and subject to interoperator variability. We investigated a deep learning-based rapid multiorgan delineation method for use in CBCT-guided adaptive pancreatic radiotherapy.

METHODS

To improve the accuracy of OAR delineation, two innovative solutions have been proposed in this study. First, instead of directly segmenting organs on CBCT images, a pretrained cycle-consistent generative adversarial network (cycleGAN) was applied to generating synthetic CT images given CBCT images. Second, an advanced deep learning model called mask-scoring regional convolutional neural network (MS R-CNN) was applied on those synthetic CT to detect the positions and shapes of multiple organs simultaneously for final segmentation. The OAR contours delineated by the proposed method were validated and compared with expert-drawn contours for geometric agreement using the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS).

RESULTS

Across eight abdominal OARs including duodenum, large bowel, small bowel, left and right kidneys, liver, spinal cord, and stomach, the geometric comparisons between automated and expert contours are as follows: 0.92 (0.89-0.97) mean DSC, 2.90 mm (1.63-4.19 mm) mean HD95, 0.89 mm (0.61-1.36 mm) mean MSD, and 1.43 mm (0.90-2.10 mm) mean RMS. Compared to the competing methods, our proposed method had significant improvements (p < 0.05) in all the metrics for all the eight organs. Once the model was trained, the contours of eight OARs can be obtained on the order of seconds.

CONCLUSIONS

We demonstrated the feasibility of a synthetic CT-aided deep learning framework for automated delineation of multiple OARs on CBCT. The proposed method could be implemented in the setting of pancreatic adaptive radiotherapy to rapidly contour OARs with high accuracy.

Automated delineation of head and neck organs at risk using synthetic MRI-aided mask scoring regional convolutional neural network

PURPOSE

Auto-segmentation algorithms offer a potential solution to eliminate the labor-intensive, time-consuming, and observer-dependent manual delineation of organs-at-risk (OARs) in radiotherapy treatment planning. This study aimed to develop a deep learning-based automated OAR delineation method to tackle the current challenges remaining in achieving reliable expert performance with the state-of-the-art auto-delineation algorithms.

METHODS

The accuracy of OAR delineation is expected to be improved by utilizing the complementary contrasts provided by computed tomography (CT) (bony-structure contrast) and magnetic resonance imaging (MRI) (soft-tissue contrast). Given CT images, synthetic MR images were firstly generated by a pre-trained cycle-consistent generative adversarial network. The features of CT and synthetic MRI were then extracted and combined for the final delineation of organs using mask scoring regional convolutional neural network. Both in-house and public datasets containing CT scans from head-and-neck (HN) cancer patients were adopted to quantitatively evaluate the performance of the proposed method against current state-of-the-art algorithms in metrics including Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square distance (RMS).

RESULTS

Across all of 18 OARs in our in-house dataset, the proposed method achieved an average DSC, HD95, MSD, and RMS of 0.77 (0.58-0.90), 2.90 mm (1.32-7.63 mm), 0.89 mm (0.42-1.85 mm), and 1.44 mm (0.71-3.15 mm), respectively, outperforming the current state-of-the-art algorithms by 6%, 16%, 25%, and 36%, respectively. On public datasets, for all nine OARs, an average DSC of 0.86 (0.73-0.97) were achieved, 6% better than the competing methods.

CONCLUSION

We demonstrated the feasibility of a synthetic MRI-aided deep learning framework for automated delineation of OARs in HN radiotherapy treatment planning. The proposed method could be adopted into routine HN cancer radiotherapy treatment planning to rapidly contour OARs with high accuracy.

Self-supervised learning for accelerated 3D high-resolution ultrasound imaging

PURPOSE

Ultrasound (US) imaging has been widely used in diagnosis, image-guided intervention, and therapy, where high-quality three-dimensional (3D) images are highly desired from sparsely acquired two-dimensional (2D) images. This study aims to develop a deep learning-based algorithm to reconstruct high-resolution (HR) 3D US images only reliant on the acquired sparsely distributed 2D images.

METHODS

We propose a self-supervised learning framework using cycle-consistent generative adversarial network (cycleGAN), where two independent cycleGAN models are trained with paired original US images and two sets of low-resolution (LR) US images, respectively. The two sets of LR US images are obtained through down-sampling the original US images along the two axes, respectively. In US imaging, in-plane spatial resolution is generally much higher than through-plane resolution. By learning the mapping from down-sampled in-plane LR images to original HR US images, cycleGAN can generate through-plane HR images from original sparely distributed 2D images. Finally, HR 3D US images are reconstructed by combining the generated 2D images from the two cycleGAN models.

RESULTS

The proposed method was assessed on two different datasets. One is automatic breast ultrasound (ABUS) images from 70 breast cancer patients, the other is collected from 45 prostate cancer patients. By applying a spatial resolution enhancement factor of 3 to the breast cases, our proposed method achieved the mean absolute error (MAE) value of 0.90 ± 0.15, the peak signal-to-noise ratio (PSNR) value of 37.88 ± 0.88 dB, and the visual information fidelity (VIF) value of 0.69 ± 0.01, which significantly outperforms bicubic interpolation. Similar performances have been achieved using the enhancement factor of 5 in these breast cases and using the enhancement factors of 5 and 10 in the prostate cases.

CONCLUSIONS

We have proposed and investigated a new deep learning-based algorithm for reconstructing HR 3D US images from sparely acquired 2D images. Significant improvement on through-plane resolution has been achieved by only using the acquired 2D images without any external atlas images. Its self-supervision capability could accelerate HR US imaging.

Therapeutic applications of radioactive sources: from image-guided brachytherapy to radio-guided surgical resection

It is well known nowadays that radioactivity can destroy the living cells it interacts with. It is therefore unsurprising that radioactive sources, such as iodine-125, were historically developed for treatment purposes within radiation oncology with the goal of damaging malignant cells. However, since then, new techniques have been invented that make creative use of the same radioactivity properties of these sources for medical applications. Here, we review two distinct kinds of therapeutic uses of radioactive sources with applications to prostate, cervical, and breast cancer: brachytherapy and radioactive seed localization. In brachytherapy (BT), the radioactive sources are used for internal radiation treatment. Current approaches make use of real-time image guidance, for instance by means of magnetic resonance imaging, ultrasound, computed tomography, and sometimes positron emission tomography, depending on clinical availability and cancer type. Such image-guided BT for prostate and cervical cancer presents a promising alternative and/or addition to external beam radiation treatments or surgical resections. Radioactive sources can also be used for radio-guided tumor localization during surgery, for which the example of iodine-125 seed use in breast cancer is given. Radioactive seed localization (RSL) is increasingly popular as an alternative tumor localization technique during breast cancer surgery. Advantages of applying RSL include added flexibility in the clinical scheduling logistics, an increase in tumor localization accuracy, and higher patient satisfaction; safety measures do however have to be employed. We exemplify the implementation of RSL in a clinic through experiences at the Netherlands Cancer Institute.

Deep learning applications in automatic segmentation and reconstruction in CT-based cervix brachytherapy

Purpose

Motivated by recent advances in deep learning, the purpose of this study was to investigate a deep learning method in automatic segment and reconstruct applicators in computed tomography (CT) images for cervix brachytherapy treatment planning.

Material and methods

U-Net model was developed for applicator segmentation in CT images. Sixty cervical cancer patients with Fletcher applicator were divided into training data and validation data according to ratio of 50 : 10, and another 10 patients with Fletcher applicator were employed to test the model. Dice similarity coefficient (DSC) and 95^th percentile Hausdorff distance (HD95) were used to evaluate the model. Segmented applicator coordinates were calculated and applied into RT structure file. Tip error and shaft error of applicators were evaluated. Dosimetric differences between manual reconstruction and deep learning-based reconstruction were compared.

Results

The averaged overall 10 test patients’ DSC, HD95, and reconstruction time were 0.89, 1.66 mm, and 17.12 s, respectively. The average tip error was 0.80 mm, and the average shaft error was less than 0.50 mm. The dosimetric differences between manual reconstruction and automatic reconstruction were 0.29% for high-risk clinical target volume (HR-CTV) D_90%, and less than 2.64% for organs at risk D_2cc at a scenario of doubled maximum shaft error.

Conclusions

We proposed a deep learning-based reconstruction method to localize Fletcher applicator in three-dimensional CT images. The achieved accuracy and efficiency confirmed our method as clinically attractive. It paves the way for the automation of brachytherapy treatment planning.

A review of deep learning based methods for medical image multi-organ segmentation

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.

Intensity non-uniformity correction in MR imaging using residual cycle generative adversarial network

Correcting or reducing the effects of voxel intensity non-uniformity (INU) within a given tissue type is a crucial issue for quantitative magnetic resonance (MR) image analysis in daily clinical practice. Although having no severe impact on visual diagnosis, the INU can highly degrade the performance of automatic quantitative analysis such as segmentation, registration, feature extraction and radiomics. In this study, we present an advanced deep learning based INU correction algorithm called residual cycle generative adversarial network (res-cycle GAN), which integrates the residual block concept into a cycle-consistent GAN (cycle-GAN). In cycle-GAN, an inverse transformation was implemented between the INU uncorrected and corrected magnetic resonance imaging (MRI) images to constrain the model through forcing the calculation of both an INU corrected MRI and a synthetic corrected MRI. A fully convolution neural network integrating residual blocks was applied in the generator of cycle-GAN to enhance end-to-end raw MRI to INU corrected MRI transformation. A cohort of 55 abdominal patients with T1-weighted MR INU images and their corrections with a clinically established and commonly used method, namely, N4ITK were used as a pair to evaluate the proposed res-cycle GAN based INU correction algorithm. Quantitatively comparisons of normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC) indices, and spatial non-uniformity (SNU) were made among the proposed method and other approaches. Our res-cycle GAN based method achieved an NMAE of 0.011 ± 0.002, a PSNR of 28.0 ± 1.9 dB, an NCC of 0.970 ± 0.017, and a SNU of 0.298 ± 0.085. Our proposed method has significant improvements (p < 0.05) in NMAE, PSNR, NCC and SNU over other algorithms including conventional GAN and U-net. Once the model is well trained, our approach can automatically generate the corrected MR images in a few minutes, eliminating the need for manual setting of parameters.