Adaptive prediction of internal target motion using external marker motion: a technical study

Artificial intelligence-based motion tracking in cancer radiotherapy: A review

Radiotherapy aims to deliver a prescribed dose to the tumor while sparing neighboring organs at risk (OARs). Increasingly complex treatment techniques such as volumetric modulated arc therapy (VMAT), stereotactic radiosurgery (SRS), stereotactic body radiotherapy (SBRT), and proton therapy have been developed to deliver doses more precisely to the target. While such technologies have improved dose delivery, the implementation of intra-fraction motion management to verify tumor position at the time of treatment has become increasingly relevant. Artificial intelligence (AI) has recently demonstrated great potential for real-time tracking of tumors during treatment. However, AI-based motion management faces several challenges, including bias in training data, poor transparency, difficult data collection, complex workflows and quality assurance, and limited sample sizes. This review presents the AI algorithms used for chest, abdomen, and pelvic tumor motion management/tracking for radiotherapy and provides a literature summary on the topic. We will also discuss the limitations of these AI-based studies and propose potential improvements.

A review of artificial intelligence applications for motion tracking in radiotherapy

During radiotherapy, the organs and tumour move as a result of the dynamic nature of the body; this is known as intrafraction motion. Intrafraction motion can result in tumour underdose and healthy tissue overdose, thereby reducing the effectiveness of the treatment while increasing toxicity to the patients. There is a growing appreciation of intrafraction target motion management by the radiation oncology community. Real-time image-guided radiation therapy (IGRT) can track the target and account for the motion, improving the radiation dose to the tumour and reducing the dose to healthy tissue. Recently, artificial intelligence (AI)-based approaches have been applied to motion management and have shown great potential. In this review, four main categories of motion management using AI are summarised: marker-based tracking, markerless tracking, full anatomy monitoring and motion prediction. Marker-based and markerless tracking approaches focus on tracking the individual target throughout the treatment. Full anatomy algorithms monitor for intrafraction changes in the full anatomy within the field of view. Motion prediction algorithms can be used to account for the latencies due to the time for the system to localise, process and act.

Potential of a probabilistic framework for target prediction from surrogate respiratory motion during lung radiotherapy

Purpose.Respiration-induced motion introduces significant positioning uncertainties in radiotherapy treatments for thoracic sites. Accounting for this motion is a non-trivial task commonly addressed with surrogate-based strategies and latency compensating techniques. This study investigates the potential of a new unified probabilistic framework to predict both future target motion in real-time from a surrogate signal and associated uncertainty.Method.A Bayesian approach is developed, based on a Kalman filter theory adapted specifically for surrogate measurements. Breathing motions are collected simultaneously from a lung target, two external surrogates (abdominal and thoracic markers) and an internal surrogate (liver structure) for 9 volunteers during 4 min, in which severe breathing changes occur to assess the robustness of the method. A comparison with an artificial non-linear neural network (NN) is performed, although no confidence interval prediction is provided. A static worst-case scenario and a simple static design are investigated.Results.Although the NN can reduce the prediction errors from thoracic surrogate in some cases, the Bayesian framework outperforms in most cases the NN when using the other surrogates: bias on predictions is reduced by 38% and 16% on average when using respectively the liver and the abdomen for the simple scenario, and by respectively 40% and 31% for the worst-case scenario. The standard deviation of residuals is reduced on average by up to 42%. The Bayesian method is also found to be more robust to increasing latencies. The thoracic marker appears to be less reliable to predict the target position, while the liver shows to be a better surrogate. A statistical test confirms the significance of both observations.Conclusion.The proposed framework predicts both the future target position and the associated uncertainty, which can be valuably used to further assist motion management decisions. Further investigation is required to improve the predictions by using an adaptive version of the proposed framework.

Adaptive respiratory signal prediction using dual multi-layer perceptron neural networks

PURPOSE

To improve the prediction accuracy of respiratory signals by adapting the multi-layer perceptron neural network (MLP-NN) model to changing respiratory signals. We have previously developed an MLP-NN to predict respiratory signals obtained from a real-time position management (RPM) device. Preliminary testing results indicated that poor prediction accuracy may be observed after several seconds for irregular breathing patterns as only a set of fixed data was used in one-time training. To improve the prediction accuracy, we introduced a continuous learning technique using the updated training data to replace one-time learning using the fixed training data. We carried on this new prediction using an adaptation approach with dual MLP-NNs rather than single MLP-NN. When one MLP-NN was performing prediction of the respiratory signals, another one was being trained using the updated data and vice versa. The predicted performance was evaluated by root-mean-square-error (RMSE) between the predicted and true signals from 202 patients' respiratory patterns each with 1 min recording length. The effects of adding an additional network, training parameter, and respiratory signal irregularity on the performance of the new predictor were investigated based on four different network configurations: a single MLP-NN, high-computation dual MLP-NNs (U1), two different combinations of high- and low-computation dual MLP-NNs (U2 and U3). The RMSEs using U1 method were reduced by 34%, 19%, and 10% compared to those using MLP-NN, U2 and U3 methods, respectively. Continuous training of an MLP-NN based on a dual-network configuration using updated respiratory signals improved prediction accuracy compared to one-time training of an MLP-NN using fixed signals.

Predicting real-time 3D deformation field maps (DFM) based on volumetric cine MRI (VC-MRI) and artificial neural networks for on-board 4D target tracking: a feasibility study

To predict real-time 3D deformation field maps (DFMs) using Volumetric Cine MRI (VC-MRI) and adaptive boosting and multi-layer perceptron neural network (ADMLP-NN) for 4D target tracking. One phase of a prior 4D-MRI is set as the prior phase, MRIprior. Principal component analysis (PCA) is used to extract three major respiratory deformation modes from the DFMs generated between the prior and remaining phases. VC-MRI at each time-step is considered a deformation of MRIprior, where the DFM is represented as a weighted linear combination of the PCA components. The PCA weightings are solved by minimizing the differences between on-board 2D cine MRI and its corresponding VC-MRI slice. The PCA weightings solved during the initial training period are used to train an ADMLP-NN to predict PCA weightings ahead of time during the prediction period. The predicted PCA weightings are used to build predicted 3D DFM and ultimately, predicted VC-MRIs for 4D target tracking. The method was evaluated using a 4D computerized phantom (XCAT) with patient breathing curves and MRI data from a real liver cancer patient. Effects of breathing amplitude change and ADMLP-NN parameter variations were assessed. The accuracy of the PCA curve prediction was evaluated. The predicted real-time 3D tumor was evaluated against the ground-truth using volume dice coefficient (VDC), center-of-mass-shift (COMS), and target tracking errors. For the XCAT study, the average VDC and COMS for the predicted tumor were 0.92  ±  0.02 and 1.06  ±  0.40 mm, respectively, across all predicted time-steps. The correlation coefficients between predicted and actual PCA curves generated through VC-MRI estimation for the 1st/2nd principal components were 0.98/0.89 and 0.99/0.57 in the SI and AP directions, respectively. The optimal number of input neurons, hidden neurons, and MLP-NN for ADMLP-NN PCA weighting coefficient prediction were determined to be 7, 4, and 10, respectively. The optimal cost function threshold was determined to be 0.05. PCA weighting coefficient and VC-MRI accuracy was reduced for increased prediction-step size. Accurate PCA weighting coefficient prediction correlated with accurate VC-MRI prediction. For the patient study, the predicted 4D tumor tracking errors in superior-inferior, anterior-posterior and lateral directions were 0.50  ±  0.47 mm, 0.40  ±  0.55 mm, and 0.28  ±  0.12 mm, respectively. Preliminary studies demonstrated the feasibility to use VC-MRI and artificial neural networks to predict real-time 3D DFMs of the tumor for 4D target tracking.

Towards real-time respiratory motion prediction based on long short-term memory neural networks

Radiation therapy of thoracic and abdominal tumors requires incorporating the respiratory motion into treatments. To precisely account for the patient's respiratory motions and predict the respiratory signals, a generalized model for predictions of different types of patients' respiratory motions is desired. The aim of this study is to explore the feasibility of developing a long short-term memory (LSTM)-based generalized model for the respiratory signal prediction. To achieve that, 1703 sets of real-time position management (RPM) data were collected from retrospective studies across three clinical institutions. These datasets were separated as the training, internal validity and external validity groups. Among all the datasets, 1187 datasets were used for model development and the remaining 516 datasets were used to test the model's generality power. Furthermore, an exhaustive grid search was implemented to find the optimal hyper-parameters of the LSTM model. The hyper-parameters are the number of LSTM layers, the number of hidden units, the optimizer, the learning rate, the number of epochs, and the length of time lags. The obtained model achieved superior accuracy over conventional artificial neural network (ANN) models: with the prediction window equaling to 500 ms, the LSTM model achieved an average relative mean absolute error (MAE) of 0.037, an average root mean square error (RMSE) of 0.048, and a maximum error (ME) of 1.687 in the internal validity data, and an average relative MAE of 0.112, an average RMSE of 0.139 and an ME of 1.811 in the external validity data. Compared to the LSTM model trained with default hyper-parameters, the MAE of the optimized model results decreased by 20%, indicating the importance of tuning the hyper-parameters of LSTM models to obtain superior accuracies. This study demonstrates the potential of deep LSTM models for the respiratory signal prediction and illustrates the impacts of major hyper-parameters in LSTM models.

Development of a real-time internal and external marker tracking system for particle therapy: a phantom study using patient tumor trajectory data

Target motion-induced uncertainty in particle therapy is more complicated than that in X-ray therapy, requiring more accurate motion management. Therefore, a hybrid motion-tracking system that can track internal tumor motion and as well as an external surrogate of tumor motion was developed. Recently, many correlation tests between internal and external markers in X-ray therapy have been developed; however, the accuracy of such internal/external marker tracking systems, especially in particle therapy, has not yet been sufficiently tested. In this article, the process of installing an in-house hybrid internal/external motion-tracking system is described and the accuracy level of tracking system was acquired. Our results demonstrated that the developed in-house external/internal combined tracking system has submillimeter accuracy, and can be clinically used as a particle therapy system as well as a simulation system for moving tumor treatment.

Respiratory signal prediction based on adaptive boosting and multi-layer perceptron neural network

To improve the prediction accuracy of respiratory signals using adaptive boosting and multi-layer perceptron neural network (ADMLP-NN) for gated treatment of moving target in radiation therapy. The respiratory signals acquired using a real-time position management (RPM) device from 138 previous 4DCT scans were retrospectively used in this study. The ADMLP-NN was composed of several artificial neural networks (ANNs) which were used as weaker predictors to compose a stronger predictor. The respiratory signal was initially smoothed using a Savitzky-Golay finite impulse response smoothing filter (S-G filter). Then, several similar multi-layer perceptron neural networks (MLP-NNs) were configured to estimate future respiratory signal position from its previous positions. Finally, an adaptive boosting (Adaboost) decision algorithm was used to set weights for each MLP-NN based on the sample prediction error of each MLP-NN. Two prediction methods, MLP-NN and ADMLP-NN (MLP-NN plus adaptive boosting), were evaluated by calculating correlation coefficient and root-mean-square-error between true and predicted signals. For predicting 500 ms ahead of prediction, average correlation coefficients were improved from 0.83 (MLP-NN method) to 0.89 (ADMLP-NN method). The average of root-mean-square-error (relative unit) for 500 ms ahead of prediction using ADMLP-NN were reduced by 27.9%, compared to those using MLP-NN. The preliminary results demonstrate that the ADMLP-NN respiratory prediction method is more accurate than the MLP-NN method and can improve the respiration prediction accuracy.

A neural network-based 2D/3D image registration quality evaluator for pediatric patient setup in external beam radiotherapy

Our purpose was to develop a neural network‐based registration quality evaluator (RQE) that can improve the 2D/3D image registration robustness for pediatric patient setup in external beam radiotherapy. Orthogonal daily setup X‐ray images of six pediatric patients with brain tumors receiving proton therapy treatments were retrospectively registered with their treatment planning computed tomography (CT) images. A neural network‐based pattern classifier was used to determine whether a registration solution was successful based on geometric features of the similarity measure values near the point‐of‐solution. Supervised training and test datasets were generated by rigidly registering a pair of orthogonal daily setup X‐ray images to the treatment planning CT. The best solution for each registration task was selected from 50 optimizing attempts that differed only by the randomly generated initial transformation parameters. The distance from each individual solution to the best solution in the normalized parametrical space was compared to a user‐defined error tolerance to determine whether that solution was acceptable. A supervised training was then used to train the RQE. Performance of the RQE was evaluated using test dataset consisting of registration results that were not used in training. The RQE was integrated with our in‐house 2D/3D registration system and its performance was evaluated using the same patient dataset. With an optimized sampling step size (i.e., 5 mm) in the feature space, the RQE has the sensitivity and the specificity in the ranges of 0.865–0.964 and 0.797–0.990, respectively, when used to detect registration error with mean voxel displacement (MVD) greater than 1 mm. The trial‐to‐acceptance ratio of the integrated 2D/3D registration system, for all patients, is equal to 1.48. The final acceptance ratio is 92.4%. The proposed RQE can potentially be used in a 2D/3D rigid image registration system to improve the overall robustness by rejecting unsuccessful registration solutions. The RQE is not patient‐specific, so a single RQE can be constructed and used for a particular application (e.g., the registration for images acquired on the same anatomical site). Implementation of the RQE in a 2D/3D registration system is clinically feasible.

PACS numbers: 87.57.nj, 87.85.dq, 87.55.Qr

Intra- and Inter-Fractional Variation Prediction of Lung Tumors Using Fuzzy Deep Learning

Tumor movements should be accurately predicted to improve delivery accuracy and reduce unnecessary radiation exposure to healthy tissue during radiotherapy. The tumor movements pertaining to respiration are divided into intra-fractional variation occurring in a single treatment session and inter-fractional variation arising between different sessions. Most studies of patients’ respiration movements deal with intra-fractional variation. Previous studies on inter-fractional variation are hardly mathematized and cannot predict movements well due to inconstant variation. Moreover, the computation time of the prediction should be reduced. To overcome these limitations, we propose a new predictor for intra- and inter-fractional data variation, called intra- and inter-fraction fuzzy deep learning (IIFDL), where FDL, equipped with breathing clustering, predicts the movement accurately and decreases the computation time. Through the experimental results, we validated that the IIFDL improved root-mean-square error (RMSE) by 29.98% and prediction overshoot by 70.93%, compared with existing methods. The results also showed that the IIFDL enhanced the average RMSE and overshoot by 59.73% and 83.27%, respectively. In addition, the average computation time of IIFDL was 1.54 ms for both intra- and inter-fractional variation, which was much smaller than the existing methods. Therefore, the proposed IIFDL might achieve real-time estimation as well as better tracking techniques in radiotherapy.

A fast neural network approach to predict lung tumor motion during respiration for radiation therapy applications

During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A real-time image-guided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with real-time retraining can be used. We have investigated real-time prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higher-order neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feed-forward neural architectures are compared when using gradient descent adaptation and primarily the Levenberg-Marquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast real-time retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For one-second prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time.

Comparative performance of linear and nonlinear neural networks to predict irregular breathing

Breathing adaptation during external-beam radiotherapy is a matter of great concern because uncompensated tumour motion requires extended treatment margins that endanger sensitive tissue. Compensation strategies include beam gating, collimator tracking and robotic beam re-alignment. All of these schemes have a system latency of up to several hundred milliseconds, which calls in turn for predictive control loops. Irregularities in breathing make prediction difficult. We have evaluated the performance of two classes of control loop algorithms-the linear adaptive filter and the adaptive nonlinear neural network-for highly irregular patient breathing behaviours. The neural network demonstrated robust adaptability to all of the observed breathing patterns while the linear filter failed in a significant percentage of cases. For those cases where the linear filter could function, it made less accurate predictions than the neural network. Because the neural network presents no additional computational burden in the control loop we conclude that it is the preferred choice among heuristic predictive algorithms.