Extra-dimensional Demons: a method for incorporating missing tissue in deformable image registration

MUsculo-Skeleton-Aware (MUSA) deep learning for anatomically guided head-and-neck CT deformable registration

Deep-learning-based deformable image registration (DL-DIR) has demonstrated improved accuracy compared to time-consuming non-DL methods across various anatomical sites. However, DL-DIR is still challenging in heterogeneous tissue regions with large deformation. In fact, several state-of-the-art DL-DIR methods fail to capture the large, anatomically plausible deformation when tested on head-and-neck computed tomography (CT) images. These results allude to the possibility that such complex head-and-neck deformation may be beyond the capacity of a single network structure or a homogeneous smoothness regularization. To address the challenge of combined multi-scale musculoskeletal motion and soft tissue deformation in the head-and-neck region, we propose a MUsculo-Skeleton-Aware (MUSA) framework to anatomically guide DL-DIR by leveraging the explicit multiresolution strategy and the inhomogeneous deformation constraints between the bony structures and soft tissue. The proposed method decomposes the complex deformation into a bulk posture change and residual fine deformation. It can accommodate both inter- and intra- subject registration. Our results show that the MUSA framework can consistently improve registration accuracy and, more importantly, the plausibility of deformation for various network architectures. The code will be publicly available at https://github.com/HengjieLiu/DIR-MUSA.

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

Deformable image registration (DIR) is a versatile tool used in many applications in radiotherapy (RT). DIR algorithms have been implemented in many commercial treatment planning systems providing accessible and easy-to-use solutions. However, the geometric uncertainty of DIR can be large and difficult to quantify, resulting in barriers to clinical practice. Currently, there is no agreement in the RT community on how to quantify these uncertainties and determine thresholds that distinguish a good DIR result from a poor one. This review summarises the current literature on sources of DIR uncertainties and their impact on RT applications. Recommendations are provided on how to handle these uncertainties for patient-specific use, commissioning, and research. Recommendations are also provided for developers and vendors to help users to understand DIR uncertainties and make the application of DIR in RT safer and more reliable.

Artificial general intelligence for radiation oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale.

Deep-learning based fast and accurate 3D CT deformable image registration in lung cancer

BACKGROUND

Deformable Image Registration (DIR) is an essential technique required in many applications of radiation oncology. However, conventional DIR approaches typically take several minutes to register one pair of 3D CT images and the resulting deformable vector fields (DVFs) are only specific to the pair of images used, making it less appealing for clinical application.

PURPOSE

A deep-learning-based DIR method using CT images is proposed for lung cancer patients to address the common drawbacks of the conventional DIR approaches and in turn can accelerate the speed of related applications, such as contour propagation, dose deformation, adaptive radiotherapy (ART), etc. METHODS: A deep neural network based on VoxelMorph was developed to generate DVFs using CT images collected from 114 lung cancer patients. Two models were trained with the weighted mean absolute error (wMAE) loss and structural similarity index matrix (SSIM) loss (optional) (i.e., the MAE model and the M+S model). In total, 192 pairs of initial CT (iCT) and verification CT (vCT) were included as a training dataset and the other independent 10 pairs of CTs were included as a testing dataset. The vCTs usually were taken 2 weeks after the iCTs. The synthetic CTs (sCTs) were generated by warping the vCTs according to the DVFs generated by the pre-trained model. The image quality of the synthetic CTs was evaluated by measuring the similarity between the iCTs and the sCTs generated by the proposed methods and the conventional DIR approaches, respectively. Per-voxel absolute CT-number-difference volume histogram (CDVH) and MAE were used as the evaluation metrics. The time to generate the sCTs was also recorded and compared quantitatively. Contours were propagated using the derived DVFs and evaluated with SSIM. Forward dose calculations were done on the sCTs and the corresponding iCTs. Dose volume histograms (DVHs) were generated based on dose distributions on both iCTs and sCTs generated by two models, respectively. The clinically relevant DVH indices were derived for comparison. The resulted dose distributions were also compared using 3D Gamma analysis with thresholds of 3 mm/3%/10% and 2 mm/2%/10%, respectively.

RESULTS

The two models (wMAE and M+S) achieved a speed of 263.7±163 / 265.8±190 ms and a MAE of 13.15±3.8 / 17.52±5.8 HU for the testing dataset, respectively. The average SSIM scores of 0.987±0.006 and 0.988±0.004 were achieved by the two proposed models, respectively. For both models, CDVH of a typical patient showed that less than 5% of the voxels had a per-voxel absolute CT-number-difference larger than 55 HU. The dose distribution calculated based on a typical sCT showed differences of ≤2cGy[RBE] for clinical target volume (CTV) D95 and D5 , within ±0.06% for total lung V5 , ≤1.5cGy[RBE] for heart and esophagus Dmean , and ≤6cGy[RBE] for cord Dmax compared to the dose distribution calculated based on the iCT. The good average 3D Gamma passing rates (> 96% for 3 mm/3%/10% and > 94% for 2 mm/2%/10%, respectively) were also observed.

CONCLUSION

A deep neural network-based DIR approach was proposed and has been shown to be reasonably accurate and efficient to register the initial CTs and verification CTs in lung cancer.

Applicability and usage of dose mapping/accumulation in radiotherapy

Dose mapping/accumulation (DMA) is a topic in radiotherapy (RT) for years, but has not yet found its widespread way into clinical RT routine. During the ESTRO Physics workshop 2021 on "commissioning and quality assurance of deformable image registration (DIR) for current and future RT applications", we built a working group on DMA from which we present the results of our discussions in this article. Our aim in this manuscript is to shed light on the current situation of DMA in RT and to highlight the issues that hinder consciously integrating it into clinical RT routine. As a first outcome of our discussions, we present a scheme where representative RT use cases are positioned, considering expected anatomical variations and the impact of dose mapping uncertainties on patient safety, which we have named the DMA landscape (DMAL). This tool is useful for future reference when DMA applications get closer to clinical day-to-day use. Secondly, we discussed current challenges, lightly touching on first-order effects (related to the impact of DIR uncertainties in dose mapping), and focusing in detail on second-order effects often dismissed in the current literature (as resampling and interpolation, quality assurance considerations, and radiobiological issues). Finally, we developed recommendations, and guidelines for vendors and users. Our main point include: Strive for context-driven DIR (by considering their impact on clinical decisions/judgements) rather than perfect DIR; be conscious of the limitations of the implemented DIR algorithm; and consider when dose mapping (with properly quantified uncertainties) is a better alternative than no mapping.

Deep-Learning-based Fast and Accurate 3D CT Deformable Image Registration in Lung Cancer

PURPOSE

In some proton therapy facilities, patient alignment relies on two 2D orthogonal kV images, taken at fixed, oblique angles, as no 3D on-the-bed imaging is available. The visibility of the tumor in kV images is limited since the patient's 3D anatomy is projected onto a 2D plane, especially when the tumor is behind high-density structures such as bones. This can lead to large patient setup errors. A solution is to reconstruct the 3D CT image from the kV images obtained at the treatment isocenter in the treatment position.

METHODS

An asymmetric autoencoder-like network built with vision-transformer blocks was developed. The data was collected from 1 head and neck patient: 2 orthogonal kV images (1024x1024 voxels), 1 3D CT with padding (512x512x512) acquired from the in-room CT-on-rails before kVs were taken and 2 digitally-reconstructed-radiograph (DRR) images (512x512) based on the CT. We resampled kV images every 8 voxels and DRR and CT every 4 voxels, thus formed a dataset consisting of 262,144 samples, in which the images have a dimension of 128 for each direction. In training, both kV and DRR images were utilized, and the encoder was encouraged to learn the jointed feature map from both kV and DRR images. In testing, only independent kV images were used. The full-size synthetic CT (sCT) was achieved by concatenating the sCTs generated by the model according to their spatial information. The image quality of the synthetic CT (sCT) was evaluated using mean absolute error (MAE) and per-voxel-absolute-CT-number-difference volume histogram (CDVH).

RESULTS

The model achieved a speed of 2.1s and a MAE of <40HU. The CDVH showed that <5% of the voxels had a per-voxel-absolute-CT-number-difference larger than 185 HU.

CONCLUSION

A patient-specific vision-transformer-based network was developed and shown to be accurate and efficient to reconstruct 3D CT images from kV images.

Dose evaluations of organs at risk and predictions of gastrointestinal toxicity after re-irradiation with stereotactic body radiation therapy for pancreatic cancer by deformable image registration

Purpose

Re-irradiation of locally recurrent pancreatic cancer may be an optimal choice as a local ablative therapy. However, dose constraints of organs at risk (OARs) predictive of severe toxicity remain unknown. Therefore, we aim to calculate and identify accumulated dose distributions of OARs correlating with severe adverse effects and determine possible dose constraints regarding re-irradiation.

Methods

Patients receiving two courses of stereotactic body radiation therapy (SBRT) for the same irradiated regions (the primary tumors) due to local recurrence were included. All doses of the first and second plans were recalculated to an equivalent dose of 2 Gy per fraction (EQD₂). Deformable image registration with the workflow "Dose Accumulation-Deformable" of the MIM^® System (version: 6.6.8) was performed for dose summations. Dose-volume parameters predictive of grade 2 or more toxicities were identified, and the receiver operating characteristic (ROC) curve was used to determine optimal thresholds of dose constraints.

Results

Forty patients were included in the analysis. Only the V ₁₀ of the stomach [hazard ratio (HR): 1.02 (95% CI:1.00-1.04), P = 0.035] and D _mean of the intestine [HR: 1.78 (95% CI: 1.00-3.18), P = 0.049] correlated with grade 2 or more gastrointestinal toxicity. Hence, the equation of probability of such toxicity was P = 1 1 + e - ( - 4.155 + 0.579 D mean of the intestine + 0.021 V 10  of the stomach ) Additionally, the area under the ROC curve and threshold of dose constraints of V ₁₀ of the stomach and D _mean of the intestine were 0.779 and 77.575 cc, 0.769 and 4.22 Gy₃ (α/β = 3), respectively. The area under the ROC curve of the equation was 0.821.

Conclusion

The V ₁₀ of the stomach and D _mean of the intestine may be vital parameters to predict grade 2 or more gastrointestinal toxicity, of which the threshold of dose constraints may be beneficial for the practice of re-irradiation of locally relapsed pancreatic cancer.

Joint synthesis and registration network for deformable MR-CBCT image registration for neurosurgical guidance

Objective.The accuracy of navigation in minimally invasive neurosurgery is often challenged by deep brain deformations (up to 10 mm due to egress of cerebrospinal fluid during neuroendoscopic approach). We propose a deep learning-based deformable registration method to address such deformations between preoperative MR and intraoperative CBCT.Approach.The registration method uses a joint image synthesis and registration network (denoted JSR) to simultaneously synthesize MR and CBCT images to the CT domain and perform CT domain registration using a multi-resolution pyramid. JSR was first trained using a simulated dataset (simulated CBCT and simulated deformations) and then refined on real clinical images via transfer learning. The performance of the multi-resolution JSR was compared to a single-resolution architecture as well as a series of alternative registration methods (symmetric normalization (SyN), VoxelMorph, and image synthesis-based registration methods).Main results.JSR achieved median Dice coefficient (DSC) of 0.69 in deep brain structures and median target registration error (TRE) of 1.94 mm in the simulation dataset, with improvement from single-resolution architecture (median DSC = 0.68 and median TRE = 2.14 mm). Additionally, JSR achieved superior registration compared to alternative methods-e.g. SyN (median DSC = 0.54, median TRE = 2.77 mm), VoxelMorph (median DSC = 0.52, median TRE = 2.66 mm) and provided registration runtime of less than 3 s. Similarly in the clinical dataset, JSR achieved median DSC = 0.72 and median TRE = 2.05 mm.Significance.The multi-resolution JSR network resolved deep brain deformations between MR and CBCT images with performance superior to other state-of-the-art methods. The accuracy and runtime support translation of the method to further clinical studies in high-precision neurosurgery.

Synthesis of pseudo-CT images from pelvic MRI images based on MD-CycleGAN model for radiotherapy

OBJECTIVE

A multi-discriminator-based cycle generative adversarial network (MD-CycleGAN) model was proposed to synthesize higher-quality pseudo-CT from MRI.

APPROACH

The MRI and CT images obtained at the simulation stage with cervical cancer were selected to train the model. The generator adopted the DenseNet as the main architecture. The local and global discriminators based on convolutional neural network jointly discriminated the authenticity of the input image data. In the testing phase, the model was verified by four-fold cross-validation method. In the prediction stage, the data were selected to evaluate the accuracy of the pseudo-CT in anatomy and dosimetry, and they were compared with the pseudo-CT synthesized by GAN with generator based on the architectures of ResNet, sU-Net, and FCN.

MAIN RESULTS

There are significant differences(P<0.05) in the four-fold-cross validation results on peak signal-to-noise ratio and structural similarity index metrics between the pseudo-CT obtained based on MD-CycleGAN and the ground truth CT (CTgt). The pseudo-CT synthesized by MD-CycleGAN had closer anatomical information to the CTgt with root mean square error of 47.83±2.92 HU and normalized mutual information value of 0.9014±0.0212 and mean absolute error value of 46.79±2.76 HU. The differences in dose distribution between the pseudo-CT obtained by MD-CycleGAN and the CTgt were minimal. The mean absolute dose errors of Dosemax, Dosemin and Dosemean based on the planning target volume were used to evaluate the dose uncertainty of the four pseudo-CT. The u-values of the Wilcoxon test were 55.407, 41.82 and 56.208, and the differences were statistically significant. The 2%/2 mm-based gamma pass rate (%) of the proposed method was 95.45±1.91, and the comparison methods (ResNet_GAN, sUnet_GAN and FCN_GAN) were 93.33±1.20, 89.64±1.63 and 87.31±1.94, respectively.

SIGNIFICANCE

The pseudo-CT obtained based on MD-CycleGAN have higher imaging quality and are closer to the CTgt in terms of anatomy and dosimetry than other GAN models.

Semi-Supervised Deep Learning-Based Image Registration Method with Volume Penalty for Real-Time Breast Tumor Bed Localization

Breast-conserving surgery requires supportive radiotherapy to prevent cancer recurrence. However, the task of localizing the tumor bed to be irradiated is not trivial. The automatic image registration could significantly aid the tumor bed localization and lower the radiation dose delivered to the surrounding healthy tissues. This study proposes a novel image registration method dedicated to breast tumor bed localization addressing the problem of missing data due to tumor resection that may be applied to real-time radiotherapy planning. We propose a deep learning-based nonrigid image registration method based on a modified U-Net architecture. The algorithm works simultaneously on several image resolutions to handle large deformations. Moreover, we propose a dedicated volume penalty that introduces the medical knowledge about tumor resection into the registration process. The proposed method may be useful for improving real-time radiation therapy planning after the tumor resection and, thus, lower the surrounding healthy tissues' irradiation. The data used in this study consist of 30 computed tomography scans acquired in patients with diagnosed breast cancer, before and after tumor surgery. The method is evaluated using the target registration error between manually annotated landmarks, the ratio of tumor volume, and the subjective visual assessment. We compare the proposed method to several other approaches and show that both the multilevel approach and the volume regularization improve the registration results. The mean target registration error is below 6.5 mm, and the relative volume ratio is close to zero. The registration time below 1 s enables the real-time processing. These results show improvements compared to the classical, iterative methods or other learning-based approaches that do not introduce the knowledge about tumor resection into the registration process. In future research, we plan to propose a method dedicated to automatic localization of missing regions that may be used to automatically segment tumors in the source image and scars in the target image.

Research on obtaining pseudo CT images based on stacked generative adversarial network

Background

To investigate the feasibility of using a stacked generative adversarial network (sGAN) to synthesize pseudo computed tomography (CT) images based on ultrasound (US) images.

Methods

The pre-radiotherapy US and CT images of 75 patients with cervical cancer were selected for the training set of pseudo-image synthesis. In the first stage, labeled US images were used as the first conditional GAN input to obtain low-resolution pseudo CT images, and in the second stage, a super-resolution reconstruction GAN was used. The pseudo CT image obtained in the first stage was used as an input, following which a high-resolution pseudo CT image with clear texture and accurate grayscale information was obtained. Five cross validation tests were performed to verify our model. The mean absolute error (MAE) was used to compare each pseudo CT with the same patient's real CT image. Also, another 10 cases of patients with cervical cancer, before radiotherapy, were selected for testing, and the pseudo CT image obtained using the neural style transfer (NSF) and CycleGAN methods were compared with that obtained using the sGAN method proposed in this study. Finally, the dosimetric accuracy of pseudo CT images was verified by phantom experiments.

Results

The MAE metric values between the pseudo CT obtained based on sGAN, and the real CT in five-fold cross validation are 66.82±1.59 HU, 66.36±1.85 HU, 67.26±2.37 HU, 66.34±1.75 HU, and 67.22±1.30 HU, respectively. The results of the metrics, namely, normalized mutual information (NMI), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR), between the pseudo CT images obtained using the sGAN method and the ground truth CT (CTgt) images were compared with those of the other two methods via the paired t-test, and the differences were statistically significant. The dice similarity coefficient (DSC) measurement results showed that the pseudo CT images obtained using the sGAN method were more similar to the CTgt images of organs at risk. The dosimetric phantom experiments also showed that the dose distribution between the pseudo CT images synthesized by the new method was similar to that of the CTgt images.

Conclusions

Compared with NSF and CycleGAN methods, the sGAN method can obtain more accurate pseudo CT images, thereby providing a new method for image guidance in radiotherapy for cervical cancer.

Imaging Study of Pseudo-CT Synthesized From Cone-Beam CT Based on 3D CycleGAN in Radiotherapy

Purpose

To propose a synthesis method of pseudo-CT (CT_CycleGAN) images based on an improved 3D cycle generative adversarial network (CycleGAN) to solve the limitations of cone-beam CT (CBCT), which cannot be directly applied to the correction of radiotherapy plans.

Methods

The improved U-Net with residual connection and attention gates was used as the generator, and the discriminator was a full convolutional neural network (FCN). The imaging quality of pseudo-CT images is improved by adding a 3D gradient loss function. Fivefold cross-validation was performed to validate our model. Each pseudo CT generated is compared against the real CT image (ground truth CT, CT_gt) of the same patient based on mean absolute error (MAE) and structural similarity index (SSIM). The dice similarity coefficient (DSC) coefficient was used to evaluate the segmentation results of pseudo CT and real CT. 3D CycleGAN performance was compared to 2D CycleGAN based on normalized mutual information (NMI) and peak signal-to-noise ratio (PSNR) metrics between the pseudo-CT and CT_gt images. The dosimetric accuracy of pseudo-CT images was evaluated by gamma analysis.

Results

The MAE metric values between the CT_CycleGAN and the real CT in fivefold cross-validation are 52.03 ± 4.26HU, 50.69 ± 5.25HU, 52.48 ± 4.42HU, 51.27 ± 4.56HU, and 51.65 ± 3.97HU, respectively, and the SSIM values are 0.87 ± 0.02, 0.86 ± 0.03, 0.85 ± 0.02, 0.85 ± 0.03, and 0.87 ± 0.03 respectively. The DSC values of the segmentation of bladder, cervix, rectum, and bone between CT_CycleGAN and real CT images are 91.58 ± 0.45, 88.14 ± 1.26, 87.23 ± 2.01, and 92.59 ± 0.33, respectively. Compared with 2D CycleGAN, the 3D CycleGAN based pseudo-CT image is closer to the real image, with NMI values of 0.90 ± 0.01 and PSNR values of 30.70 ± 0.78. The gamma pass rate of the dose distribution between CT_CycleGAN and CT_gt is 97.0% (2%/2 mm).

Conclusion

The pseudo-CT images obtained based on the improved 3D CycleGAN have more accurate electronic density and anatomical structure.

Diffeomorphic Registration With Intensity Transformation and Missing Data: Application to 3D Digital Pathology of Alzheimer's Disease

This paper examines the problem of diffeomorphic image registration in the presence of differing image intensity profiles and sparsely sampled, missing, or damaged tissue. Our motivation comes from the problem of aligning 3D brain MRI with 100-micron isotropic resolution to histology sections at 1 × 1 × 1,000-micron resolution with multiple varying stains. We pose registration as a penalized Bayesian estimation, exploiting statistical models of image formation where the target images are modeled as sparse and noisy observations of the atlas. In this injective setting, there is no assumption of symmetry between atlas and target. Cross-modality image matching is achieved by jointly estimating polynomial transformations of the atlas intensity. Missing data is accommodated via a multiple atlas selection procedure where several atlas images may be of homogeneous intensity and correspond to "background" or "artifact." The two concepts are combined within an Expectation-Maximization algorithm, where atlas selection posteriors and deformation parameters are updated iteratively and polynomial coefficients are computed in closed form. We validate our method with simulated images, examples from neuropathology, and a standard benchmarking dataset. Finally, we apply it to reconstructing digital pathology and MRI in standard atlas coordinates. By using a standard convolutional neural network to detect tau tangles in histology slices, this registration method enabled us to quantify the 3D density distribution of tauopathy throughout the medial temporal lobe of an Alzheimer's disease postmortem specimen.

A momentum-based diffeomorphic demons framework for deformable MR-CT image registration

Neuro-navigated procedures require a high degree of geometric accuracy but are subject to geometric error from complex deformation in the deep brain-e.g. regions about the ventricles due to egress of cerebrospinal fluid (CSF) upon neuroendoscopic approach or placement of a ventricular shunt. We report a multi-modality, diffeomorphic, deformable registration method using momentum-based acceleration of the Demons algorithm to solve the transformation relating preoperative MRI and intraoperative CT as a basis for high-precision guidance. The registration method (pMI-Demons) extends the mono-modality, diffeomorphic form of the Demons algorithm to multi-modality registration using pointwise mutual information (pMI) as a similarity metric. The method incorporates a preprocessing step to nonlinearly stretch CT image values and incorporates a momentum-based approach to accelerate convergence. Registration performance was evaluated in phantom and patient images: first, the sensitivity of performance to algorithm parameter selection (including update and displacement field smoothing, histogram stretch, and the momentum term) was analyzed in a phantom study over a range of simulated deformations; and second, the algorithm was applied to registration of MR and CT images for four patients undergoing minimally invasive neurosurgery. Performance was compared to two previously reported methods (free-form deformation using mutual information (MI-FFD) and symmetric normalization using mutual information (MI-SyN)) in terms of target registration error (TRE), Jacobian determinant (J), and runtime. The phantom study identified optimal or nominal settings of algorithm parameters for translation to clinical studies. In the phantom study, the pMI-Demons method achieved comparable registration accuracy to the reference methods and strongly reduced outliers in TRE (p [Formula: see text] 0.001 in Kolmogorov-Smirnov test). Similarly, in the clinical study: median TRE  =  1.54 mm (0.83-1.66 mm interquartile range, IQR) for pMI-Demons compared to 1.40 mm (1.02-1.67 mm IQR) for MI-FFD and 1.64 mm (0.90-1.92 mm IQR) for MI-SyN. The pMI-Demons and MI-SyN methods yielded diffeomorphic transformations (J  >  0) that preserved topology, whereas MI-FFD yielded unrealistic (J  <  0) deformations subject to tissue folding and tearing. Momentum-based acceleration gave a ~35% speedup of the pMI-Demons method, providing registration runtime of 10.5 min (reduced to 2.2 min on GPU), compared to 15.5 min for MI-FFD and 34.7 min for MI-SyN. The pMI-Demons method achieved registration accuracy comparable to MI-FFD and MI-SyN, maintained diffeomorphic transformation similar to MI-SyN, and accelerated runtime in a manner that facilitates translation to image-guided neurosurgery.

Research on pseudo-CT imaging technique based on an ultrasound deformation field with binary mask in radiotherapy

This study aimed to investigate the reliability of pseudo-computed tomography (pseudo-CT) imaging based on ultrasound (US) deformation fields under different binary masks in radiotherapy.We used 3-dimensional (3D) CT and US images, including those acquired during CT simulation positioning, and cone-beam CT (CBCT) and US images acquired 1 week after treating 3 patients with cervical cancer. Image data of 3 different layers were selected from the US images, and 3D CT images of each patient were selected. For US image registration, the following were created and applied: binary masks of the region of interest overlapping (ROIO) between the US image based on simulation positioning and US image for positioning verification, region of interest (ROI), whole overlapping (wholeO), and whole imaging region (whole). Accordingly, the deformation field was obtained and applied to CT images (CTsim), and different pseudo-CT images were acquired. Similarities between the pseudo-CT and CBCT images were compared, and registration accuracies between pseudo-CT images under different binary masks and CTsim were compared and discussed.A pair t test was conducted to normalized mutual information values of the registration accuracy between the pseudo-CT image based on ROIO binary mask and CTsim with other methods (P < .05), and the difference was statistically significant. A pair t test of normalized gray mean-squared errors was also performed (P < .05), and the difference was statistically significant. The similarity function means between pseudo-CT, that is, based on ROIO, ROI, wholeO, whole, and no binary mask, and CBCT were 0.9084, 0.8365, 0.7800, 0.6830, and 0.5518, respectively.Pseudo-CT based on ROIO binary mask best matched with CTsim and achieved the highest similarity with CBCT.

Deformable image registration applied to lung SBRT: Usefulness and limitations

Radiation therapy (RT) of the lung requires deformation analysis. Deformable image registration (DIR) is the fundamental method to quantify deformations for various applications: motion compensation, contour propagation, dose accumulation, etc. DIR is therefore unavoidable in lung RT. DIR algorithms have been studied for decades and are now available both within commercial and academic packages. However, they are complex and have limitations that every user must be aware of before clinical implementation. In this paper, the main applications of DIR for lung RT with their associated uncertainties and their limitations are reviewed.

Integration of free-hand 3D ultrasound and mobile C-arm cone-beam CT: Feasibility and characterization for real-time guidance of needle insertion

This work presents development of an integrated ultrasound (US)-cone-beam CT (CBCT) system for image-guided needle interventions, combining a low-cost ultrasound system (Interson VC 7.5MHz, Pleasanton, CA) with a mobile C-arm for fluoroscopy and CBCT via use of a surgical tracker. Imaging performance of the ultrasound system was characterized in terms of depth-dependent contrast-to-noise ratio (CNR) and spatial resolution. US-CBCT system was evaluated in phantom studies simulating three needle-based procedures: drug delivery, tumor ablation, and lumbar puncture. Low-cost ultrasound provided flexibility but exhibited modest CNR and spatial resolution that is likely limited to fairly superficial applications within a ∼10cm depth of view. Needle tip localization demonstrated target registration error 2.1-3.0mm using fiducial-based registration.

MIND Demons: Symmetric Diffeomorphic Deformable Registration of MR and CT for Image-Guided Spine Surgery

Intraoperative localization of target anatomy and critical structures defined in preoperative MR/CT images can be achieved through the use of multimodality deformable registration. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality-independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, finds a deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the integrated velocity fields, a modality-insensitive similarity function suitable to multimodality images, and smoothness on the diffeomorphisms themselves. Direct optimization without relying on the exponential map and stationary velocity field approximation used in conventional diffeomorphic Demons is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, normalized MI (NMI) Demons, and MIND with a diffusion-based registration method (MIND-elastic). The method yielded sub-voxel invertibility (0.008 mm) and nonzero-positive Jacobian determinants. It also showed improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.7 mm compared to 11.3, 3.1, 5.6, and 2.4 mm for MI FFD, LMI FFD, NMI Demons, and MIND-elastic methods, respectively. Validation in clinical studies demonstrated realistic deformations with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

Temporal subtraction contrast-enhanced dedicated breast CT

The development of a framework of deformable image registration and segmentation for the purpose of temporal subtraction contrast-enhanced breast CT is described. An iterative histogram-based two-means clustering method was used for the segmentation. Dedicated breast CT images were segmented into background (air), adipose, fibroglandular and skin components. Fibroglandular tissue was classified as either normal or contrast-enhanced then divided into tiers for the purpose of categorizing degrees of contrast enhancement. A variant of the Demons deformable registration algorithm, intensity difference adaptive Demons (IDAD), was developed to correct for the large deformation forces that stemmed from contrast enhancement. In this application, the accuracy of the proposed method was evaluated in both mathematically-simulated and physically-acquired phantom images. Clinical usage and accuracy of the temporal subtraction framework was demonstrated using contrast-enhanced breast CT datasets from five patients. Registration performance was quantified using normalized cross correlation (NCC), symmetric uncertainty coefficient, normalized mutual information (NMI), mean square error (MSE) and target registration error (TRE). The proposed method outperformed conventional affine and other Demons variations in contrast enhanced breast CT image registration. In simulation studies, IDAD exhibited improvement in MSE (0-16%), NCC (0-6%), NMI (0-13%) and TRE (0-34%) compared to the conventional Demons approaches, depending on the size and intensity of the enhancing lesion. As lesion size and contrast enhancement levels increased, so did the improvement. The drop in the correlation between the pre- and post-contrast images for the largest enhancement levels in phantom studies is less than 1.2% (150 Hounsfield units). Registration error, measured by TRE, shows only submillimeter mismatches between the concordant anatomical target points in all patient studies. The algorithm was implemented using a parallel processing architecture resulting in rapid execution time for the iterative segmentation and intensity-adaptive registration techniques. Characterization of contrast-enhanced lesions is improved using temporal subtraction contrast-enhanced dedicated breast CT. Adaptation of Demons registration forces as a function of contrast-enhancement levels provided a means to accurately align breast tissue in pre- and post-contrast image acquisitions, improving subtraction results. Spatial subtraction of the aligned images yields useful diagnostic information with respect to enhanced lesion morphology and uptake.

Validation of a semi-automatic co-registration of MRI scans in patients with brain tumors during treatment follow-up

There is an expanding research interest in high-grade gliomas because of their significant population burden and poor survival despite the extensive standard multimodal treatment. One of the obstacles is the lack of individualized monitoring of tumor characteristics and treatment response before, during and after treatment. We have developed a two-stage semi-automatic method to co-register MRI scans at different time points before and after surgical and adjuvant treatment of high-grade gliomas. This two-stage co-registration includes a linear co-registration of the semi-automatically derived mask of the preoperative contrast-enhancing area or postoperative resection cavity, brain contour and ventricles between different time points. The resulting transformation matrix was then applied in a non-linear manner to co-register conventional contrast-enhanced T1 -weighted images. Targeted registration errors were calculated and compared with linear and non-linear co-registered images. Targeted registration errors were smaller for the semi-automatic non-linear co-registration compared with both the non-linear and linear co-registered images. This was further visualized using a three-dimensional structural similarity method. The semi-automatic non-linear co-registration allowed for optimal correction of the variable brain shift at different time points as evaluated by the minimal targeted registration error. This proposed method allows for the accurate evaluation of the treatment response, essential for the growing research area of brain tumor imaging and treatment response evaluation in large sets of patients. Copyright © 2016 John Wiley & Sons, Ltd.

MIND Demons for MR-to-CT Deformable Image Registration In Image-Guided Spine Surgery

PURPOSE

Localization of target anatomy and critical structures defined in preoperative MR images can be achieved by means of multi-modality deformable registration to intraoperative CT. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration.

METHOD

The method, called MIND Demons, solves for the deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the velocity fields and the diffeomorphisms, a modality-insensitive similarity function suitable to multi-modality images, and constraints on geodesics in Lagrangian coordinates. Direct optimization (without relying on an exponential map of stationary velocity fields used in conventional diffeomorphic Demons) is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, in phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to conventional mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, and normalized MI (NMI) Demons.

RESULT

The method yielded sub-voxel invertibility (0.006 mm) and nonsingular spatial Jacobians with capability to preserve local orientation and topology. It demonstrated improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.5 mm compared to 10.9, 2.3, and 4.6 mm for MI FFD, LMI FFD, and NMI Demons methods, respectively. Validation in clinical studies demonstrated realistic deformation with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

CONCLUSIONS

A modality-independent deformable registration method has been developed to estimate a viscoelastic diffeomorphic map between preoperative MR and intraoperative CT. The method yields registration accuracy suitable to application in image-guided spine surgery across a broad range of anatomical sites and modes of deformation.

Toward adaptive radiotherapy for head and neck patients: Uncertainties in dose warping due to the choice of deformable registration algorithm

PURPOSE

The aims of this work were to evaluate the performance of several deformable image registration (DIR) algorithms implemented in our in-house software (NiftyReg) and the uncertainties inherent to using different algorithms for dose warping.

METHODS

The authors describe a DIR based adaptive radiotherapy workflow, using CT and cone-beam CT (CBCT) imaging. The transformations that mapped the anatomy between the two time points were obtained using four different DIR approaches available in NiftyReg. These included a standard unidirectional algorithm and more sophisticated bidirectional ones that encourage or ensure inverse consistency. The forward (CT-to-CBCT) deformation vector fields (DVFs) were used to propagate the CT Hounsfield units and structures to the daily geometry for "dose of the day" calculations, while the backward (CBCT-to-CT) DVFs were used to remap the dose of the day onto the planning CT (pCT). Data from five head and neck patients were used to evaluate the performance of each implementation based on geometrical matching, physical properties of the DVFs, and similarity between warped dose distributions. Geometrical matching was verified in terms of dice similarity coefficient (DSC), distance transform, false positives, and false negatives. The physical properties of the DVFs were assessed calculating the harmonic energy, determinant of the Jacobian, and inverse consistency error of the transformations. Dose distributions were displayed on the pCT dose space and compared using dose difference (DD), distance to dose difference, and dose volume histograms.

RESULTS

All the DIR algorithms gave similar results in terms of geometrical matching, with an average DSC of 0.85 ± 0.08, but the underlying properties of the DVFs varied in terms of smoothness and inverse consistency. When comparing the doses warped by different algorithms, we found a root mean square DD of 1.9% ± 0.8% of the prescribed dose (pD) and that an average of 9% ± 4% of voxels within the treated volume failed a 2%pD DD-test (DD2%-pp). Larger DD2%-pp was found within the high dose gradient (21% ± 6%) and regions where the CBCT quality was poorer (28% ± 9%). The differences when estimating the mean and maximum dose delivered to organs-at-risk were up to 2.0%pD and 2.8%pD, respectively.

CONCLUSIONS

The authors evaluated several DIR algorithms for CT-to-CBCT registrations. In spite of all methods resulting in comparable geometrical matching, the choice of DIR implementation leads to uncertainties in dose warped, particularly in regions of high gradient and/or poor imaging quality.

Improving anatomical mapping of complexly deformed anatomy for external beam radiotherapy and brachytherapy dose accumulation in cervical cancer

PURPOSE

In the treatment of cervical cancer, large anatomical deformations, caused by, e.g., tumor shrinkage, bladder and rectum filling changes, organ sliding, and the presence of the brachytherapy (BT) applicator, prohibit the accumulation of external beam radiotherapy (EBRT) and BT dose distributions. This work proposes a structure-wise registration with vector field integration (SW+VF) to map the largely deformed anatomies between EBRT and BT, paving the way for 3D dose accumulation between EBRT and BT.

METHODS

T2w-MRIs acquired before EBRT and as a part of the MRI-guided BT procedure for 12 cervical cancer patients, along with the manual delineations of the bladder, cervix-uterus, and rectum-sigmoid, were used for this study. A rigid transformation was used to align the bony anatomy in the MRIs. The proposed SW+VF method starts by automatically segmenting features in the area surrounding the delineated organs. Then, each organ and feature pair is registered independently using a feature-based nonrigid registration algorithm developed in-house. Additionally, a background transformation is calculated to account for areas far from all organs and features. In order to obtain one transformation that can be used for dose accumulation, the organ-based, feature-based, and the background transformations are combined into one vector field using a weighted sum, where the contribution of each transformation can be directly controlled by its extent of influence (scope size). The optimal scope sizes for organ-based and feature-based transformations were found by an exhaustive analysis. The anatomical correctness of the mapping was independently validated by measuring the residual distances after transformation for delineated structures inside the cervix-uterus (inner anatomical correctness), and for anatomical landmarks outside the organs in the surrounding region (outer anatomical correctness). The results of the proposed method were compared with the results of the rigid transformation and nonrigid registration of all structures together (AST).

RESULTS

The rigid transformation achieved a good global alignment (mean outer anatomical correctness of 4.3 mm) but failed to align the deformed organs (mean inner anatomical correctness of 22.4 mm). Conversely, the AST registration produced a reasonable alignment for the organs (6.3 mm) but not for the surrounding region (16.9 mm). SW+VF registration achieved the best results for both regions (3.5 and 3.4 mm for the inner and outer anatomical correctness, respectively). All differences were significant (p < 0.02, Wilcoxon rank sum test). Additionally, optimization of the scope sizes determined that the method was robust for a large range of scope size values.

CONCLUSIONS

The novel SW+VF method improved the mapping of large and complex deformations observed between EBRT and BT for cervical cancer patients. Future studies that quantify the mapping error in terms of dose errors are required to test the clinical applicability of dose accumulation by the SW+VF method.

Evaluation of deformable image registration methods for dose monitoring in head and neck radiotherapy

In the context of head and neck cancer (HNC) adaptive radiation therapy (ART), the two purposes of the study were to compare the performance of multiple deformable image registration (DIR) methods and to quantify their impact for dose accumulation, in healthy structures. Fifteen HNC patients had a planning computed tomography (CT0) and weekly CTs during the 7 weeks of intensity-modulated radiation therapy (IMRT). Ten DIR approaches using different registration methods (demons or B-spline free form deformation (FFD)), preprocessing, and similarity metrics were tested. Two observers identified 14 landmarks (LM) on each CT-scan to compute LM registration error. The cumulated doses estimated by each method were compared. The two most effective DIR methods were the demons and the FFD, with both the mutual information (MI) metric and the filtered CTs. The corresponding LM registration accuracy (precision) was 2.44 mm (1.30 mm) and 2.54 mm (1.33 mm), respectively. The corresponding LM estimated cumulated dose accuracy (dose precision) was 0.85 Gy (0.93 Gy) and 0.88 Gy (0.95 Gy), respectively. The mean uncertainty (difference between maximal and minimal dose considering all the 10 methods) to estimate the cumulated mean dose to the parotid gland (PG) was 4.03 Gy (SD = 2.27 Gy, range: 1.06-8.91 Gy).

Prior-based artifact correction (PBAC) in computed tomography

PURPOSE

Image quality in computed tomography (CT) often suffers from artifacts which may reduce the diagnostic value of the image. In many cases, these artifacts result from missing or corrupt regions in the projection data, e.g., in the case of metal, truncation, and limited angle artifacts. The authors propose a generalized correction method for different kinds of artifacts resulting from missing or corrupt data by making use of available prior knowledge to perform data completion.

METHODS

The proposed prior-based artifact correction (PBAC) method requires prior knowledge in form of a planning CT of the same patient or in form of a CT scan of a different patient showing the same body region. In both cases, the prior image is registered to the patient image using a deformable transformation. The registered prior is forward projected and data completion of the patient projections is performed using smooth sinogram inpainting. The obtained projection data are used to reconstruct the corrected image.

RESULTS

The authors investigate metal and truncation artifacts in patient data sets acquired with a clinical CT and limited angle artifacts in an anthropomorphic head phantom data set acquired with a gantry-based flat detector CT device. In all cases, the corrected images obtained by PBAC are nearly artifact-free. Compared to conventional correction methods, PBAC achieves better artifact suppression while preserving the patient-specific anatomy at the same time. Further, the authors show that prominent anatomical details in the prior image seem to have only minor impact on the correction result.

CONCLUSIONS

The results show that PBAC has the potential to effectively correct for metal, truncation, and limited angle artifacts if adequate prior data are available. Since the proposed method makes use of a generalized algorithm, PBAC may also be applicable to other artifacts resulting from missing or corrupt data.

dPIRPLE: a joint estimation framework for deformable registration and penalized-likelihood CT image reconstruction using prior images

Sequential imaging studies are conducted in many clinical scenarios. Prior images from previous studies contain a great deal of patient-specific anatomical information and can be used in conjunction with subsequent imaging acquisitions to maintain image quality while enabling radiation dose reduction (e.g., through sparse angular sampling, reduction in fluence, etc). However, patient motion between images in such sequences results in misregistration between the prior image and current anatomy. Existing prior-image-based approaches often include only a simple rigid registration step that can be insufficient for capturing complex anatomical motion, introducing detrimental effects in subsequent image reconstruction. In this work, we propose a joint framework that estimates the 3D deformation between an unregistered prior image and the current anatomy (based on a subsequent data acquisition) and reconstructs the current anatomical image using a model-based reconstruction approach that includes regularization based on the deformed prior image. This framework is referred to as deformable prior image registration, penalized-likelihood estimation (dPIRPLE). Central to this framework is the inclusion of a 3D B-spline-based free-form-deformation model into the joint registration-reconstruction objective function. The proposed framework is solved using a maximization strategy whereby alternating updates to the registration parameters and image estimates are applied allowing for improvements in both the registration and reconstruction throughout the optimization process. Cadaver experiments were conducted on a cone-beam CT testbench emulating a lung nodule surveillance scenario. Superior reconstruction accuracy and image quality were demonstrated using the dPIRPLE algorithm as compared to more traditional reconstruction methods including filtered backprojection, penalized-likelihood estimation (PLE), prior image penalized-likelihood estimation (PIPLE) without registration, and prior image penalized-likelihood estimation with rigid registration of a prior image (PIRPLE) over a wide range of sampling sparsity and exposure levels.

Validation of nonrigid registration in pretreatment and follow-up PET/CT scans for quantification of tumor residue in lung cancer patients

Nonrigid registrations of pre‐ and postradiotherapy (RT) PET/CT scans of NSCLC patients were performed with different algorithms and validated tracking internal landmarks. Dice overlap ratios (DR) of high FDG‐uptake areas in registered PET/CT scans were then calculated to study patterns of relapse. For 22 patients, pre‐ and post‐RT PET/CT scans were registered first rigidly and then nonrigidly. For three patients, two types (based on Demons or Morphons) of nonrigid registration algorithms each with four different parameter settings were applied and assessed using landmark validation. The two best performing methods were tested on all patients, who were then classified into three groups: large (Group 1), minor (Group 2) or insufficient improvement (Group 3) of registration accuracy. For Group 1 and 2, DRs between high FDG‐uptake areas in pre‐ and post‐RT PET scans were determined. Distances between corresponding landmarks on deformed pre‐RT and post‐RT scans decreased for all registration methods. Differences between Demons and Morphons methods were smaller than 1 mm. For Group 1, landmark distance decreased from 9.5 ± 2.1 mm to 3.8 ± 1.2 mm (mean ± 1 SD, p < 0.001), and for Group 3 from 13.6 ± 3.2 mm to 8.0 ± 2.2 mm (p=0.02). No significant change was observed for Group 2 where distances decreased from 5.6 ± 1.3 mm to 4.5 ± 1.1 mm (p=0.02). DRs of high FDG‐uptake areas improved significantly after nonrigid registration for most patients in Group 1. Landmark validation of nonrigid registration methods for follow‐up CT imaging in NSCLC is necessary. Nonrigid registration significantly improves matching between pre‐ and post‐RT CT scans for a subset of patients, although not in all patients. Hence, the quality of the registration needs to be assessed for each patient individually. Successful nonrigid registration increased the overlap between pre‐ and post‐RT high FDG‐uptake regions.

PACS number: 87.57.Q‐, 87.57.C‐, 87.57.N‐, 87.57.‐s, 87.55.‐x, 87.55.D‐, 87.55.dh, 87.57.uk, 87.57.nj

The distance discordance metric-a novel approach to quantifying spatial uncertainties in intra- and inter-patient deformable image registration

Previous methods to estimate the inherent accuracy of deformable image registration (DIR) have typically been performed relative to a known ground truth, such as tracking of anatomic landmarks or known deformations in a physical or virtual phantom. In this study, we propose a new approach to estimate the spatial geometric uncertainty of DIR using statistical sampling techniques that can be applied to the resulting deformation vector fields (DVFs) for a given registration. The proposed DIR performance metric, the distance discordance metric (DDM), is based on the variability in the distance between corresponding voxels from different images, which are co-registered to the same voxel at location (X) in an arbitrarily chosen 'reference' image. The DDM value, at location (X) in the reference image, represents the mean dispersion between voxels, when these images are registered to other images in the image set. The method requires at least four registered images to estimate the uncertainty of the DIRs, both for inter- and intra-patient DIR. To validate the proposed method, we generated an image set by deforming a software phantom with known DVFs. The registration error was computed at each voxel in the 'reference' phantom and then compared to DDM, inverse consistency error (ICE), and transitivity error (TE) over the entire phantom. The DDM showed a higher Pearson correlation (Rp) with the actual error (Rp ranged from 0.6 to 0.9) in comparison with ICE and TE (Rp ranged from 0.2 to 0.8). In the resulting spatial DDM map, regions with distinct intensity gradients had a lower discordance and therefore, less variability relative to regions with uniform intensity. Subsequently, we applied DDM for intra-patient DIR in an image set of ten longitudinal computed tomography (CT) scans of one prostate cancer patient and for inter-patient DIR in an image set of ten planning CT scans of different head and neck cancer patients. For both intra- and inter-patient DIR, the spatial DDM map showed large variation over the volume of interest (the pelvis for the prostate patient and the head for the head and neck patients). The highest discordance was observed in the soft tissues, such as the brain, bladder, and rectum, due to higher variability in the registration. The smallest DDM values were observed in the bony structures in the pelvis and the base of the skull. The proposed metric, DDM, provides a quantitative tool to evaluate the performance of DIR when a set of images is available. Therefore, DDM can be used to estimate and visualize the uncertainty of intra- and/or inter-patient DIR based on the variability of the registration rather than the absolute registration error.

Deformable image registration for cone-beam CT guided transoral robotic base-of-tongue surgery

Transoral robotic surgery (TORS) offers a minimally invasive approach to resection of base-of-tongue tumors. However, precise localization of the surgical target and adjacent critical structures can be challenged by the highly deformed intraoperative setup. We propose a deformable registration method using intraoperative cone-beam computed tomography (CBCT) to accurately align preoperative CT or MR images with the intraoperative scene. The registration method combines a Gaussian mixture (GM) model followed by a variation of the Demons algorithm. First, following segmentation of the volume of interest (i.e. volume of the tongue extending to the hyoid), a GM model is applied to surface point clouds for rigid initialization (GM rigid) followed by nonrigid deformation (GM nonrigid). Second, the registration is refined using the Demons algorithm applied to distance map transforms of the (GM-registered) preoperative image and intraoperative CBCT. Performance was evaluated in repeat cadaver studies (25 image pairs) in terms of target registration error (TRE), entropy correlation coefficient (ECC) and normalized pointwise mutual information (NPMI). Retraction of the tongue in the TORS operative setup induced gross deformation >30 mm. The mean TRE following the GM rigid, GM nonrigid and Demons steps was 4.6, 2.1 and 1.7 mm, respectively. The respective ECC was 0.57, 0.70 and 0.73, and NPMI was 0.46, 0.57 and 0.60. Registration accuracy was best across the superior aspect of the tongue and in proximity to the hyoid (by virtue of GM registration of surface points on these structures). The Demons step refined registration primarily in deeper portions of the tongue further from the surface and hyoid bone. Since the method does not use image intensities directly, it is suitable to multi-modality registration of preoperative CT or MR with intraoperative CBCT. Extending the 3D image registration to the fusion of image and planning data in stereo-endoscopic video is anticipated to support safer, high-precision base-of-tongue robotic surgery.