A clustering approach to 4D MRI retrospective sorting for the investigation of different surrogates

Embedding-Alignment Fusion-Based Graph Convolution Network With Mixed Learning Strategy for 4D Medical Image Reconstruction

In recent years, 4D medical image involving structural and motion information of tissue has attracted increasing attention. The key to the 4D image reconstruction is to stack the 2D slices based on matching the aligned motion states. In this study, the distribution of the 2D slices with the different motion states is modeled as a manifold graph, and the reconstruction is turned to be the graph alignment. An embedding-alignment fusion-based graph convolution network (GCN) with a mixed-learning strategy is proposed to align the graphs. Herein, the embedding and alignment processes of graphs interact with each other to realize a precise alignment with retaining the manifold distribution. The mixed strategy of self- and semi-supervised learning makes the alignment sparse to avoid the mismatching caused by outliers in the graph. In the experiment, the proposed 4D reconstruction approach is validated on the different modalities including Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Ultrasound (US). We evaluate the reconstruction accuracy and compare it with those of state-of-the-art methods. The experiment results demonstrate that our approach can reconstruct a more accurate 4D image.

Transfer-learning is a key ingredient to fast deep learning-based 4D liver MRI reconstruction

Time-resolved volumetric magnetic resonance imaging (4D MRI) could be used to address organ motion in image-guided interventions like tumor ablation. Current 4D reconstruction techniques are unsuitable for most interventional settings because they are limited to specific breathing phases, lack temporal/spatial resolution, and have long prior acquisitions or reconstruction times. Deep learning-based (DL) 4D MRI approaches promise to overcome these shortcomings but are sensitive to domain shift. This work shows that transfer learning (TL) combined with an ensembling strategy can help alleviate this key challenge. We evaluate four approaches: pre-trained models from the source domain, models directly trained from scratch on target domain data, models fine-tuned from a pre-trained model and an ensemble of fine-tuned models. For that the data base was split into 16 source and 4 target domain subjects. Comparing ensemble of fine-tuned models (N = 10) with directly learned models, we report significant improvements (P < 0.001) of the root mean squared error (RMSE) of up to 12% and the mean displacement (MDISP) of up to 17.5%. The smaller the target domain data amount, the larger the effect. This shows that TL + Ens significantly reduces beforehand acquisition time and improves reconstruction quality, rendering it a key component in making 4D MRI clinically feasible for the first time in the context of 4D organ motion models of the liver and beyond.

Predicting 4D liver MRI for MR-guided interventions

Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.

Time-resolved MRI for off-line treatment robustness evaluation in carbon-ion radiotherapy of pancreatic cancer

PURPOSE

In this study, we investigate the use of magnetic resonance imaging (MRI) for the clinical evaluation of gating treatment robustness in carbon-ion radiotherapy (CIRT) of pancreatic cancer. Indeed, MRI allows radiation-free repeated scans and fast dynamic sequences for time-resolved (TR) imaging (cine-MRI), providing information on inter- and intra-fraction cycle-to-cycle variations of respiratory motion. MRI can therefore support treatment planning and verification, overcoming the limitations of the current clinical standard, that is, four-dimensional computed tomography (4DCT), which describes an "average" breathing cycle neglecting breathing motion variability.

METHODS

We integrated a technique to generate a virtual CT (vCT) from 3D MRI with a method for 3D reconstruction from 2D cine-MRI, to produce TR vCTs for dose recalculations. For eight patients, the method allowed evaluating inter-fraction variations at end-exhale and intra-fraction cycle-to-cycle variability within the gating window in terms of tumor displacement and dose to the target and organs at risk.

RESULTS

The median inter-fraction tumor motion was in the range 3.33-12.16 mm, but the target coverage was robust (-0.4% median D95% variation). Concerning cycle-to-cycle variations, the gating technique was effective in limiting tumor displacement (1.35 mm median gating motion) and corresponding dose variations (-3.9% median D95% variation). The larger exposure of organs at risk (duodenum and stomach) was caused by inter-fraction motion, whereas intra-fraction cycle-to-cycle dose variations were limited.

CONCLUSIONS

This study proposed a method for the generation of TR vCTs from MRI, which enabled an off-line evaluation of gating treatment robustness and suggested its feasibility to support treatment planning of pancreatic tumors in CIRT.

Stability and Reliability of Enhanced External-Internal Motion Correlation via Dynamic Phase-Shift Corrections Over 30-min Timeframe for Respiratory-Gated Radiotherapy

To assess the stability of patient-specific phase shifts between external- and internal-respiratory motion waveforms, the reliability of enhanced external–internal correlation with phase-shift correction, and the feasibility of guiding respiratory-gated radiotherapy (RGRT) over 30 min. In this clinical feasibility investigation, external bellows and internal-navigator waveforms were simultaneously and prospectively acquired along with two four-dimensional magnetic resonance imaging (4DMRI) scans (6–15 m each) with 15–20 m intervals in 10 volunteers. A bellows was placed 5 cm inferior to the xiphoid to monitor abdominal motion, and an MR navigator was used to track the diaphragmatic motion. The mean phase-domain (MPD) method was applied, which combines three individual phase-calculating methods: phase-space oval fitting, principal component analysis, and analytic signal analysis, weighted by the reciprocal of their residual errors (RE) excluding outliers (RE >2σ). The time-domain cross-correlation (TCC) analysis was applied for comparison. Dynamic phase-shift correction was performed based on the phase shift detected on the fly within two 10 s moving datasets. Simulating bellows-triggered gating, the median and 95% confidence interval for the navigator's position at beam-on/beam-off and %harm (percentage of beam-on time outside the safety margin) were calculated. Averaged across all subjects, the mean phase shifts are found indistinguishable (p > .05) between scan 1 (55˚ ± 9˚) and scan 2 (59˚ ± 11˚). Using the MPD method the averaged correlation increases from 0.56 ± 0.22 to 0.85 ± 0.11 for scan 1 and from 0.47 ± 0.30 to 0.84 ± 0.08 for scan 2. The TCC correction results in similar results. After phase-shift correction, the number of cases that were suitable for amplitude gating (with <10%harm) increased from 2 to 17 out of 20 cases. A patient-specific, stable phase-shift between the external and internal motions was observed and corrected using the MPD and TCC methods, producing long-lasting enhanced motion correlation over 30m. Phase-shift correction offers a feasible strategy for improving the accuracy of tumor-motion prediction during RGRT.

An MRI framework for respiratory motion modelling validation

Introduction

Respiratory motion models establish a correspondence between respiratory‐correlated (RC) 4‐dimensional (4D) imaging and respiratory surrogates, to estimate time‐resolved (TR) 3D breathing motion. To evaluate the performance of motion models on real patient data, a validation framework based on magnetic resonance imaging (MRI) is proposed, entailing the use of RC 4DMRI to build the model, and on both (i) TR 2D cine‐MRI and (ii) additional 4DMRI data for testing intra‐/inter‐fraction breathing motion variability.

Methods

Repeated MRI data were acquired in 7 patients with abdominal lesions. The considered model relied on deformable image registration (DIR) for building the model and compensating for inter‐fraction baseline variations. Both 2D and 3D validation were performed, by comparing model estimations with the ground truth 2D cine‐MRI and 4DMRI respiratory phases, respectively.

Results

The median DIR error was comparable to the voxel size (1.33 × 1.33 × 5 mm³), with higher values in the presence of large inter‐fraction motion (median value: 2.97 mm). In the 2D validation, the median estimation error on anatomical landmarks’ position resulted below 4 mm in every scenario, whereas in the 3D validation it was 1.33 mm and 4.21 mm when testing intra‐ and inter‐fraction motion, respectively. The range of motion described in the cine‐MRI was comparable to the motion of the building 4DMRI, being always above the estimation error. Overall, the model performance was dependent on DIR error, presenting reduced accuracy when inter‐fraction baseline variations occurred.

Conclusions

Results suggest the potential of the proposed framework in evaluating global motion models for organ motion management in MRI‐guided radiotherapy.

Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences

Motion compensation can eliminate inconsistencies of respiratory movement during image acquisitions for precise vascular reconstruction in the clinical diagnosis of vascular disease from x-ray angiographic image sequences. In x-ray-based vascular interventional therapy, motion modeling can simulate the process of organ deformation driven by motion signals to display a dynamic organ on angiograms without contrast agent injection. Automatic respiratory signal estimation from x-ray angiographic image sequences is essential for motion compensation and modeling. The effects of respiratory motion, cardiac impulses, and tremors on structures in the chest and abdomen bring difficulty in extracting accurate respiratory signals individually. In this study, an end-to-end deep learning framework based on a motion-flow-guided recurrent network is proposed to address the aforementioned problem. The proposed method utilizes a convolutional neural network to learn the spatial features of every single frame, and a recurrent neural network to learn the temporal features of the entire sequence. The combination of the two networks can effectively analyze the image sequence to realize respiratory signal estimation. In addition, the motion-flow between consecutive frames is introduced to provide a dynamic constraint of spatial features, which enables the recurrent network to learn better temporal features from dynamic spatial features than from static spatial features. We demonstrate the advantages of our approach on designed datasets which contain coronary and hepatic angiographic sequences with diaphragm structures, and coronary angiographic sequences without diaphragm structures. Our method improves over state-of-the-art manifold-learning-based methods by 85.7%, 81.5% and 75.3% in respiratory signal accuracy metric on these datasets. The results demonstrate that the proposed method can effectively estimate respiratory signals from multiple motion patterns.

Investigating the use of virtual 4DCT from 4DMRI in gated carbon ion radiation therapy of abdominal tumors

PURPOSE

To generate virtual 4DCT from 4DMRI with field of view (FOV) extended to the entire involved patient anatomy, in order to evaluate its use in carbon ion radiation therapy (CIRT) of the abdominal site in a clinical scenario.

MATERIALS AND METHODS

The virtual 4DCT was generated by deforming a reference CT in order to (1) match the anatomy depicted in the 4DMRI within its FOV, by calculating deformation fields with deformable image registration to describe inter-fractional and breathing motion, and (2) obtain physically plausible deformation outside of the 4DMRI FOV, by propagating and modulating the previously obtained deformation fields. The implemented method was validated on a digital anthropomorphic phantom, for which a ground truth (GT) 4DCT was available. A CIRT treatment plan was optimized at the end-exhale reference CT and the RBE-weighted dose distribution was recalculated on both the virtual and GT 4DCTs. The method estimation error was quantified by comparing the virtual and GT 4DCTs and the corresponding recomputed doses. The method was then evaluated on 8 patients with pancreas or liver tumors treated with CIRT using respiratory gating at end-exhale. The clinical treatment plans adopted at the National Center for Oncological Hadrontherapy (CNAO, Pavia, Italy) were considered and the dose distribution was recomputed on all respiratory phases of the planning and virtual 4DCTs. By comparing the two datasets and the corresponding dose distributions, the geometrical and dosimetric impact of organ motion was assessed.

RESULTS

For the phantom, the error outside of the 4DMRI FOV was up to 4.5mm, but it remained sub-millimetric in correspondence to the target within the 4DMRI FOV. Although the impact of motion on the target D_95% resulted in variations ranging from 22% to 90% between the planned dose and the doses recomputed on the GT 4DCT phases, the corresponding estimation error was ≤2.2%. In the patient cases, the variation of the baseline tumor position between the planning and the virtual end-exhale CTs presented a median (interquartile range) value of 6.0 (4.9) mm. For baseline variations larger than 5mm, the tumor D_95% variation between the plan and the dose recomputed on the end-exhale virtual CT resulted larger than 10%. Median variations higher than 10% in the target D_95% and gastro-intestinal OARs D_2% were quantified at the end-inhale, whereas close to the end-exhale phase, limited variations of relevant dose metrics were found for both tumor and OARs.

CONCLUSIONS

The negligible impact of the geometrical inaccuracy in the estimated anatomy outside of the 4DMRI FOV on the overall dosimetric accuracy suggests the feasibility of virtual 4DCT with extended FOV in CIRT of the abdominal site. In the analyzed patient group, inter-fractional variations such as baseline variation and breathing variability were quantified, demonstrating the method capability to support treatment planning in gated CIRT of the abdominal site.

Organ motion quantification and margins evaluation in carbon ion therapy of abdominal lesions

PURPOSE

In image-guided particle radiotherapy of abdominal lesions, respiratory motion hinders treatment accuracy. In this study, 2D cineMRI data were used to quantify the tumor (GTV) motion and to evaluate the clinical approach based on deriving an internal target volume (ITV) from a planning 4DCT for gating treatments.

METHODS

Seven patients with abdominal lesions were treated with carbon-ion therapy at the National Centre of Oncological Hadron-therapy (Italy). The MR scan was performed on the same day of the 4DCT acquisition. For four patients, an additional MR was acquired approximately after 1 week. The cineMRI combined with deformable image registration algorithm was used to quantify tumor motion. Afterwards, two ITVs were defined considering (1) all phases (ITVFB) and (2) only phases within the gating window (ITVG), and then compared with the clinical (4DCT-derived) ITVs (ITVCG and ITVCFB).

RESULTS

Tumor residual motion estimated by cineMRI data in the two MRI sessions resulted not significantly different from 4DCT, although cineMRI accounted for cycle-to-cycle variations. The ITV normalized for the GTV median values were higher for ITVFB with respect to ITVG, ITVCFB and ITVCG. The Hausdorff distances with respect to the GTV were up to 10.55 mm, 3.13 mm, 5.56 mm and 2.51 mm, for ITVFB, ITVG, ITVCFB and ITVCG, respectively. According to both metrics, ITVCG and ITVG were not found significantly different.

CONCLUSIONS

CineMRI acquisitions allowed to quantify organ motion without delivering additional dose to the patient and to verify treatment margins in gated carbon-ion therapy of abdominal lesions.

Virtual 4DCT from 4DMRI for the management of respiratory motion in carbon ion therapy of abdominal tumors

PURPOSE

To evaluate a method for generating virtual four-dimensional computed tomography (4DCT) from four-dimensional magnetic resonance imaging (4DMRI) data in carbon ion radiotherapy with pencil beam scanning for abdominal tumors.

METHODS

Deformable image registration is used to: (a) register each respiratory phase of the 4DMRI to the end-exhale MRI; (b) register the reference end-exhale CT to the end-exhale MRI volume; (c) generate the virtual 4DCT by warping the registered CT according to the obtained deformation fields. A respiratory-gated carbon ion treatment plan is optimized on the planning 4DCT and the corresponding dose distribution is recalculated on the virtual 4DCT. The method was validated on a digital anthropomorphic phantom and tested on eight patients (18 acquisitions). For the phantom, a ground truth dataset was available to assess the method performances from the geometrical and dosimetric standpoints. For the patients, the virtual 4DCT was compared with the planning 4DCT.

RESULTS

In the phantom, the method exhibits a geometrical accuracy within the voxel size and Dose Volume Histograms deviations up to 3.3% for target V_95% (mean dose difference ≤ 0.2% of the prescription dose, gamma pass rate > 98%). For patients, the virtual and the planning 4DCTs show good agreement at end-exhale (3% median D_95% difference), whereas other respiratory phases exhibit moderate motion variability with consequent dose discrepancies, confirming the need for motion mitigation strategies during treatment.

CONCLUSIONS

The virtual 4DCT approach is feasible to evaluate treatment plan robustness against intra- and interfraction motion in carbon ion therapy delivered at the abdominal site.

The noise navigator: a surrogate for respiratory-correlated 4D-MRI for motion characterization in radiotherapy

Respiratory-correlated 4D-MRI can characterize respiratory-induced motion of tumors and organs-at-risk for radiotherapy treatment planning and is a necessity for image guidance of moving tumors treated on an MRI-linac. Essential for 4D-MRI generation is a robust respiratory surrogate signal. We investigated the feasibility of the noise navigator as respiratory surrogate signal for 4D-MRI generation. The noise navigator is based on the respiratory-induced modulation of the thermal noise variance measured by the receive coils during MR acquisition and thus is inherently present and synchronized with MRI data acquisition. Additionally, the noise navigator can be combined with any rectilinear readout strategy (e.g. radial and cartesian) and is independent of MR image contrast and imaging orientation. For radiotherapy applications, the noise navigator provides a robust respiratory signal for patients scanned with an elevated coil setup. This is particularly attractive for widely used cartesian sequences where currently a non-interfering self-navigation means is lacking for MRI-based simulation and MRI-guided radiotherapy. The feasibility of 4D-MRI generation with the noise navigator as respiratory surrogate signal was demonstrated for both cartesian and radial readout strategies in radiotherapy setup on four healthy volunteers and two radiotherapy patients on a dedicated 1.5 T MRI scanner and two radiotherapy patients on a 1.5 T MRI-linac system. Moreover, the respiratory-correlated 4D-MR images showed liver motion comparable to a reference 2D cine MRI series for the volunteers. For 2D cartesian cine MRI acquisitions, both the noise navigator and respiratory bellows were benchmarked against an image navigator. Respiratory phase detection based on the noise navigator agreed 1.4 times better with the image navigator than the respiratory bellows did. For a 3D Stack-of-Stars acquisitions, the noise navigator was compared to radial self-navigation and a 1.7 times higher respiratory phase detection agreement was observed than for the respiratory bellows compared to radial self-navigation.

Real-time control of respiratory motion: Beyond radiation therapy

Motion management in radiation oncology is an important aspect of modern treatment planning and delivery. Special attention has been paid to control respiratory motion in recent years. However, other medical procedures related to both diagnosis and treatment are likely to benefit from the explicit control of breathing motion. Quantitative imaging - including increasingly important tools in radiology and nuclear medicine - is among the fields where a rapid development of motion control is most likely, due to the need for quantification accuracy. Emerging treatment modalities like focussed-ultrasound tumor ablation are also likely to benefit from a significant evolution of motion control in the near future. In the present article an overview of available respiratory motion systems along with ongoing research in this area is provided. Furthermore, an attempt is made to envision some of the most expected developments in this field in the near future.

Validation of a model for physical dose variations in irregularly moving targets treated with carbon ion beams

PURPOSE

In particle therapy, conventional treatment planning systems rely on an imaging representation of the irradiated region to compute the dose. For irregular breathing, when an imaging dataset describing the actual motion is not available, a different approach for dose estimation is needed. To this aim, we validate a method for the estimation of physical dose variations in gated carbon ion treatments, providing also a demonstration of the feasibility of physical dose metrics to assess the method performance. Finally, we describe a sample use case, in which this method is used to assess plan robustness with respect to undetected irregular tumor motion.

METHODS

The method entails the definition of a patient- and beam-specific water equivalent depth (WED) space, the simulation of motion as a translation equal to tumor displacement, and the reconstruction of the altered dose. We validated the approach using four-dimensional computed tomographies (4DCTs) and clinical plans in 12 patients, treated with respiratory gated carbon ion beams at the National Centre for Oncological Hadrontherapy (Pavia, Italy). Using the end-exhale CT and dose distribution as a reference, the physical dose delivered at the end-inhale tumor position was estimated and compared to the ground-truth dose recalculation on the end-inhale CT. Biologically effective and physical dose variations between the plan and the recalculation were compared as well. As a use case, we evaluated dose changes caused by simulated irregular tumor motion, that is, linear and nonlinear baseline shifts and/or amplitude variations with hysteresis.

RESULTS

The ratio between biologically effective and physical equivalent uniform dose (EUD) variations due to end-exhale to end-inhale motion was less than one for 96% of investigated structures. In the validation study, we found a median error corresponding to a 14% EUD overestimation for the tumor and 4% EUD underestimation for a subgroup of organs at risk, together with a high EUD variation due to motion [median 352% EUD variation between end-exhale and end-inhale doses in the planning tumor volume (PTV)]. Considering relevant dose-volume histogram (DVH) metrics, the median difference between estimated and ground truth doses was ≤ 4%. Gamma analysis between estimated and recalculated dose distributions resulted in a pass rate > 80% for 83% of the target volumes. For the two patients selected for the sample use case, a patient-specific assessment of the method performance was performed on the 4DCT and it was possible to relate EUD variations of both tumor and organs at risk to the simulated target motion.

CONCLUSIONS

The physical dose distribution was found to be more sensitive to motion with respect to the biologically effective one, suggesting the suitability of the physical dose metrics for the WED-space method validation. We showed that the method can compensate for intra-fractional tumor motion with proper accuracy in the selected patient group, although its use is recommended when limited deformations are expected. In conclusion, the WED-space method can provide simulations of dose alteration due to irregular breathing when imaging data are lacking, and, once integrated with relative biological effectiveness (RBE) modeling, it would be useful in evaluating the robustness of carbon ion treatment plans.