Evaluation of real-time tumor contour prediction using LSTM networks for MR-guided radiotherapy

Intra-frame motion deterioration effects and deep-learning-based compensation in MR-guided radiotherapy

BACKGROUND

Current commercially available hybrid magnetic resonance linear accelerators (MR-Linac) use 2D+t cine MR imaging to provide intra-fractional motion monitoring. However, given the limited temporal resolution of cine MR imaging, target intra-frame motion deterioration effects, resulting in effective time latency and motion artifacts in the image domain, can be appreciable, especially in the case of fast breathing.

PURPOSE

The aim of this work is to investigate intra-frame motion deterioration effects in MR-guided radiotherapy (MRgRT) by simulating the motion-corrupted image acquisition, and to explore the feasibility of deep-learning-based compensation approaches, relying on the intra-frame motion information which is spatially and temporally encoded in the raw data (k-space).

METHODS

An intra-frame motion model was defined to simulate motion-corrupted MR images, with 4D anthropomorphic digital phantoms being exploited to provide ground truth 2D+t cine MR sequences. A total number of 10 digital phantoms were generated for lung cancer patients, with randomly selected eight patients for training or validation and the remaining two for testing. The simulation code served as the data generator, and a dedicated motion pattern perturbation scheme was proposed to build the intra-frame motion database, where three degrees of freedom were designed to guarantee the diversity of intra-frame motion trajectories, enabling a thorough exploration in the domain of the potential anatomical structure positions. U-Nets with three types of loss functions: L1 or L2 loss defined in image or Fourier domain, referred to as NNImgLoss-L1 , NNFloss-L1 and NNL2-Loss were trained to extract information from the motion-corrupted image and used to estimate the ground truth final-position image, corresponding to the end of the acquisition. Images before and after compensation were evaluated in terms of (i) image mean-squared error (MSE) and mean absolute error (MAE), and (ii) accuracy of gross tumor volume (GTV) contouring, based on optical-flow image registration.

RESULTS

Image degradation caused by intra-frame motion was observed: for a linearly and fully acquired Cartesian readout k-space trajectory, intra-frame motion resulted in an imaging latency of approximately 50% of the acquisition time; in comparison, the motion artifacts exhibited only a negligible contribution to the overall geometric errors. All three compensation models led to a decrease in image MSE/MAE and GTV position offset compared to the motion-corrupted image. In the investigated testing dataset for GTV contouring, the average dice similarity coefficients (DSC) improved from 88% to 96%, and the 95th percentile Hausdorff distance (HD95 ) dropped from 4.8 mm to 2.1 mm. Different models showed slight performance variations across different intra-frame motion amplitude categories: NNImgLoss-L1 excelled for small/medium amplitudes, whereas NNFloss-L1 demonstrated higher DSC median values at larger amplitudes. The saliency maps of the motion-corrupted image highlighted the major contribution of the later acquired k-space data, as well as the edges of the moving anatomical structures at their final positions, during the model inference stage.

CONCLUSIONS

Our results demonstrate the deep-learning-based approaches have the potential to compensate for intra-frame motion by utilizing the later acquired data to drive the convergence of the earlier acquired k-space components.

Impact of technological advances in treatment planning, image guidance, and treatment delivery on target margin design for prostate cancer radiotherapy: an updated review

Recent innovations in image guidance, treatment delivery, and adaptive radiotherapy (RT) have created a new paradigm for planning target volume (PTV) margin design for patients with prostate cancer. We performed a review of the recent literature on PTV margin selection and design for intact prostate RT, excluding post-operative RT, brachytherapy, and proton therapy. Our review describes the increased focus on prostate and seminal vesicles as heterogenous deforming structures with further emergence of intra-prostatic GTV boost and concurrent pelvic lymph node treatment. To capture recent innovations, we highlight the evolution in cone beam CT guidance, and increasing use of MRI for improved target delineation and image registration and supporting online adaptive RT. Moreover, we summarize new and evolving image-guidance treatment platforms as well as recent reports of novel immobilization strategies and motion tracking. Our report also captures recent implementations of artificial intelligence to support image guidance and adaptive RT. To characterize the clinical impact of PTV margin changes via model-based risk estimates and clinical trials, we highlight recent high impact reports. Our report focusses on topics in the context of PTV margins but also showcase studies attempting to move beyond the PTV margin recipes with robust optimization and probabilistic planning approaches. Although guidelines exist for target margins conventional using CT-based image guidance, further validation is required to understand the optimal margins for online adaptation either alone or combined with real-time motion compensation to minimize systematic and random uncertainties in the treatment of patients with prostate cancer.

Real-time motion management in MRI-guided radiotherapy: Current status and AI-enabled prospects

MRI-guided radiotherapy (MRIgRT) is a highly complex treatment modality, allowing adaptation to anatomical changes occurring from one treatment day to the other (inter-fractional), but also to motion occurring during a treatment fraction (intra-fractional). In this vision paper, we describe the different steps of intra-fractional motion management during MRIgRT, from imaging to beam adaptation, and the solutions currently available both clinically and at a research level. Furthermore, considering the latest developments in the literature, a workflow is foreseen in which motion-induced over- and/or under-dosage is compensated in 3D, with minimal impact to the radiotherapy treatment time. Considering the time constraints of real-time adaptation, a particular focus is put on artificial intelligence (AI) solutions as a fast and accurate alternative to conventional algorithms.

Experimental comparison of linear regression and LSTM motion prediction models for MLC-tracking on an MRI-linac

BACKGROUND

Magnetic resonance imaging (MRI)-guided radiotherapy with multileaf collimator (MLC)-tracking is a promising technique for intra-fractional motion management, achieving high dose conformality without prolonging treatment times. To improve beam-target alignment, the geometric error due to system latency should be reduced by using temporal prediction.

PURPOSE

To experimentally compare linear regression (LR) and long-short-term memory (LSTM) motion prediction models for MLC-tracking on an MRI-linac using multiple patient-derived traces with different complexities.

METHODS

Experiments were performed on a prototype 1.0 T MRI-linac capable of MLC-tracking. A motion phantom was programmed to move a target in superior-inferior (SI) direction according to eight lung cancer patient respiratory motion traces. Target centroid positions were localized from sagittal 2D cine MRIs acquired at 4 Hz using a template matching algorithm. The centroid positions were input to one of four motion prediction models. We used (1) a LSTM network which had been optimized in a previous study on patient data from another cohort (offline LSTM). We also used (2) the same LSTM model as a starting point for continuous re-optimization of its weights during the experiment based on recent motion (offline+online LSTM). Furthermore, we implemented (3) a continuously updated LR model, which was solely based on recent motion (online LR). Finally, we used (4) the last available target centroid without any changes as a baseline (no-predictor). The predictions of the models were used to shift the MLC aperture in real-time. An electronic portal imaging device (EPID) was used to visualize the target and MLC aperture during the experiments. Based on the EPID frames, the root-mean-square error (RMSE) between the target and the MLC aperture positions was used to assess the performance of the different motion predictors. Each combination of motion trace and prediction model was repeated twice to test stability, for a total of 64 experiments.

RESULTS

The end-to-end latency of the system was measured to be (389 ± 15) ms and was successfully mitigated by both LR and LSTM models. The offline+online LSTM was found to outperform the other models for all investigated motion traces. It obtained a median RMSE over all traces of (2.8 ± 1.3) mm, compared to the (3.2 ± 1.9) mm of the offline LSTM, the (3.3 ± 1.4) mm of the online LR and the (4.4 ± 2.4) mm when using the no-predictor. According to statistical tests, differences were significant (p-value <0.05) among all models in a pair-wise comparison, but for the offline LSTM and online LR pair. The offline+online LSTM was found to be more reproducible than the offline LSTM and the online LR with a maximum deviation in RMSE between two measurements of 10%.

CONCLUSIONS

This study represents the first experimental comparison of different prediction models for MRI-guided MLC-tracking using several patient-derived respiratory motion traces. We have shown that among the investigated models, continuously re-optimized LSTM networks are the most promising to account for the end-to-end system latency in MRI-guided radiotherapy with MLC-tracking.

The role of artificial intelligence in radiotherapy clinical practice

This review article visits the current state of artificial intelligence (AI) in radiotherapy clinical practice. We will discuss how AI has a place in the modern radiotherapy workflow at the level of automatic segmentation and planning, two applications which have seen real-work implementation. A special emphasis will be placed on the role AI can play in online adaptive radiotherapy, such as performed at MR-linacs, where online plan adaptation is a procedure which could benefit from automation to reduce on-couch time for patients. Pseudo-CT generation and AI for motion tracking will be introduced in the scope of online adaptive radiotherapy as well. We further discuss the use of AI for decision-making and response assessment, for example for personalized prescription and treatment selection, risk stratification for outcomes and toxicities, and AI for quantitative imaging and response assessment. Finally, the challenges of generalizability and ethical aspects will be covered. With this, we provide a comprehensive overview of the current and future applications of AI in radiotherapy.