Automatic liver tumor localization using deep learning-based liver boundary motion estimation and biomechanical modeling (DL-Bio)

Artificial intelligence-based motion tracking in cancer radiotherapy: A review

Radiotherapy aims to deliver a prescribed dose to the tumor while sparing neighboring organs at risk (OARs). Increasingly complex treatment techniques such as volumetric modulated arc therapy (VMAT), stereotactic radiosurgery (SRS), stereotactic body radiotherapy (SBRT), and proton therapy have been developed to deliver doses more precisely to the target. While such technologies have improved dose delivery, the implementation of intra-fraction motion management to verify tumor position at the time of treatment has become increasingly relevant. Artificial intelligence (AI) has recently demonstrated great potential for real-time tracking of tumors during treatment. However, AI-based motion management faces several challenges, including bias in training data, poor transparency, difficult data collection, complex workflows and quality assurance, and limited sample sizes. This review presents the AI algorithms used for chest, abdomen, and pelvic tumor motion management/tracking for radiotherapy and provides a literature summary on the topic. We will also discuss the limitations of these AI-based studies and propose potential improvements.

Multiparametric MRI-Based Interpretable Radiomics Machine Learning Model Differentiates Medulloblastoma and Ependymoma in Children: A Two-Center Study

RATIONALE AND OBJECTIVES

Medulloblastoma (MB) and Ependymoma (EM) in children, share similarities in age group, tumor location, and clinical presentation. Distinguishing between them through clinical diagnosis is challenging. This study aims to explore the effectiveness of using radiomics and machine learning on multiparametric magnetic resonance imaging (MRI) to differentiate between MB and EM and validate its diagnostic ability with an external set.

MATERIALS AND METHODS

Axial T2 weighted image (T2WI) and contrast-enhanced T1weighted image (CE-T1WI) MRI sequences of 135 patients from two centers were collected as train/test sets. Volume of interest (VOI) was manually delineated by an experienced neuroradiologist, supervised by a senior. Feature selection analysis and the least absolute shrinkage and selection operator (LASSO) algorithm identified valuable features, and Shapley additive explanations (SHAP) evaluated their significance. Five machine-learning classifiers-extreme gradient boosting (XGBoost), Bernoulli naive Bayes (Bernoulli NB), Logistic Regression (LR), support vector machine (SVM), linear support vector machine (Linear SVC) classifiers were built based on T2WI (T2 model), CE-T1WI (T1 model), and T1 + T2WI (T1 + T2 model). A human expert diagnosis was developed and corrected by senior radiologists. External validation was performed at Sun Yat-Sen University Cancer Center.

RESULTS

31 valuable features were extracted from T2WI and CE-T1WI. XGBoost demonstrated the highest performance with an area under the curve (AUC) of 0.92 on the test set and maintained an AUC of 0.80 during external validation. For the T1 model, XGBoost achieved the highest AUC of 0.85 on the test set and the highest accuracy of 0.71 on the external validation set. In the T2 model, XGBoost achieved the highest AUC of 0.86 on the test set and the highest accuracy of 0.82 on the external validation set. The human expert diagnosis had an AUC of 0.66 on the test set and 0.69 on the external validation set. The integrated T1 + T2 model achieved an AUC of 0.92 on the test set, 0.80 on the external validation set, achieved the best performance. Overall, XGBoost consistently outperformed in different classification models.

CONCLUSION

The combination of radiomics and machine learning on multiparametric MRI effectively distinguishes between MB and EM in childhood, surpassing human expert diagnosis in training and testing sets.

Deep learning-based target tracking with X-ray images for radiotherapy: a narrative review

Background and Objective

As one of the main treatment modalities, radiotherapy (RT) (also known as radiation therapy) plays an increasingly important role in the treatment of cancer. RT could benefit greatly from the accurate localization of the gross tumor volume and circumambient organs at risk (OARs). Modern linear accelerators (LINACs) are typically equipped with either gantry-mounted or room-mounted X-ray imaging systems, which provide possibilities for marker-less tracking with two-dimensional (2D) kV X-ray images. However, due to organ overlapping and poor soft tissue contrast, it is challenging to track the target directly and precisely with 2D kV X-ray images. With the flourishing development of deep learning in the field of image processing, it is possible to achieve real-time marker-less tracking of targets with 2D kV X-ray images in RT using advanced deep-learning frameworks. This article sought to review the current development of deep learning-based target tracking with 2D kV X-ray images and discuss the existing limitations and potential solutions. Finally, it also discusses some common challenges and potential future developments.

Methods

Manual searches of the Web of Science, and PubMed, and Google Scholar were carried out to retrieve English-language articles. The keywords used in the searches included "radiotherapy, radiation therapy, motion tracking, target tracking, motion estimation, motion monitoring, X-ray images, digitally reconstructed radiographs, deep learning, convolutional neural network, and deep neural network". Only articles that met the predetermined eligibility criteria were included in the review. Ultimately, 23 articles published between March 2019 and December 2023 were included in the review.

Key Content and Findings

In this article, we narratively reviewed deep learning-based target tracking with 2D kV X-ray images in RT. The existing limitations, common challenges, possible solutions, and future directions of deep learning-based target tracking were also discussed. The use of deep learning-based methods has been shown to be feasible in marker-less target tracking and real-time motion management. However, it is still quite challenging to directly locate tumor and OARs in real-time with 2D kV X-ray images, and more technical and clinical efforts are needed.

Conclusions

Deep learning-based target tracking with 2D kV X-ray images is a promising method in motion management during RT. It has the potential to track the target in real time, recognize motion, reduce the extended margin, and better spare the normal tissue. However, it still has many issues that demand prompt attention, and further development before it can be put into clinical practice.

Real-time liver motion estimation via deep learning-based angle-agnostic X-ray imaging

BACKGROUND

Real-time liver imaging is challenged by the short imaging time (within hundreds of milliseconds) to meet the temporal constraint posted by rapid patient breathing, resulting in extreme under-sampling for desired 3D imaging. Deep learning (DL)-based real-time imaging/motion estimation techniques are emerging as promising solutions, which can use a single X-ray projection to estimate 3D moving liver volumes by solved deformable motion. However, such techniques were mostly developed for a specific, fixed X-ray projection angle, thereby impractical to verify and guide arc-based radiotherapy with continuous gantry rotation.

PURPOSE

To enable deformable motion estimation and 3D liver imaging from individual X-ray projections acquired at arbitrary X-ray scan angles, and to further improve the accuracy of single X-ray-driven motion estimation.

METHODS

We developed a DL-based method, X360, to estimate the deformable motion of the liver boundary using an X-ray projection acquired at an arbitrary gantry angle (angle-agnostic). X360 incorporated patient-specific prior information from planning 4D-CTs to address the under-sampling issue, and adopted a deformation-driven approach to deform a prior liver surface mesh to new meshes that reflect real-time motion. The liver mesh motion is solved via motion-related image features encoded in the arbitrary-angle X-ray projection, and through a sequential combination of rigid and deformable registration modules. To achieve the angle agnosticism, a geometry-informed X-ray feature pooling layer was developed to allow X360 to extract angle-dependent image features for motion estimation. As a liver boundary motion solver, X360 was also combined with priorly-developed, DL-based optical surface imaging and biomechanical modeling techniques for intra-liver motion estimation and tumor localization.

RESULTS

With geometry-aware feature pooling, X360 can solve the liver boundary motion from an arbitrary-angle X-ray projection. Evaluated on a set of 10 liver patient cases, the mean (± s.d.) 95-percentile Hausdorff distance between the solved liver boundary and the "ground-truth" decreased from 10.9 (±4.5) mm (before motion estimation) to 5.5 (±1.9) mm (X360). When X360 was further integrated with surface imaging and biomechanical modeling for liver tumor localization, the mean (± s.d.) center-of-mass localization error of the liver tumors decreased from 9.4 (± 5.1) mm to 2.2 (± 1.7) mm.

CONCLUSION

X360 can achieve fast and robust liver boundary motion estimation from arbitrary-angle X-ray projections for real-time imaging guidance. Serving as a surface motion solver, X360 can be integrated into a combined framework to achieve accurate, real-time, and marker-less liver tumor localization.

Online prediction for respiratory movement compensation: a patient-specific gating control for MRI-guided radiotherapy

BACKGROUND

This study aims to validate the effectiveness of linear regression for motion prediction of internal organs or tumors on 2D cine-MR and to present an online gating signal prediction scheme that can improve the accuracy of MR-guided radiotherapy for liver and lung cancer.

MATERIALS AND METHODS

We collected 2D cine-MR sequences of 21 liver cancer patients and 10 lung cancer patients to develop a binary gating signal prediction algorithm that forecasts the crossing-time of tumor motion traces relative to the target threshold. Both 0.4 s and 0.6 s prediction windows were tested using three linear predictors and three recurrent neural networks (RNNs), given the system delay of 0.5 s. Furthermore, an adaptive linear regression model was evaluated using only the first 30 s as the burn-in period, during which the model parameters were adapted during the online prediction process. The accuracy of the predicted traces was measured using amplitude metrics (MAE, RMSE, and R2), and in addition, we proposed three temporal metrics, namely crossing error, gating error, and gating accuracy, which are more relevant to the nature of the gating signals.

RESULTS

In both 0.6 s and 0.4 s prediction cases, linear regression outperformed other methods, demonstrating significantly smaller amplitude errors compared to the RNNs (P < 0.05). The proposed algorithm with adaptive linear regression had the best performance with an average gating accuracy of 98.3% and 98.0%, a gating error of 44 ms and 45 ms, for liver cancer and lung cancer patients, respectively.

CONCLUSION

A functional online gating control scheme was developed with an adaptive linear regression that is both more cost-efficient and accurate than sophisticated RNN based methods in all studied metrics.

Analysis of emerging trends and hot spots in respiratory biomechanics from 2003 to 2022 based on CiteSpace

Introduction: With the global prevalence of coronavirus disease 2019 (COVID-19), an increasing number of people are experiencing respiratory discomfort. Respiratory biomechanics can monitor breathing patterns and respiratory movements and it is easier to prevent, diagnose, treat or rehabilitate. However, there is still a lack of global knowledge structure in the field of respiratory biomechanics. With the help of CiteSpace software, we aim to help researchers identify potential collaborators and collaborating institutions, hotspots and research frontiers in respiratory biomechanics. Methods: Articles on respiratory biomechanics from 2003 to 2022 were retrieved from the Web of Science Core Collection by using a specific strategy, resulting a total of 2,850 publications. We used CiteSpace 6.1.R6 to analyze the year of publication, journal/journals cited, country, institution, author/authors cited, references, keywords and research trends. Co-citation maps were created to visually observe research hot spots and knowledge structures. Results and discussion: The number of annual publications gradually increased over the past 20 years. Medical Physics published the most articles and had the most citations in this study. The United States was the most influential country, with the highest number and centrality of publications. The most productive and influential institution was Harvard University in the United States. Keall PJ was the most productive author and MCCLELLAND JR was the most cited authors The article by Keall PJ (2006) article (cocitation counts: 55) and the article by McClelland JR (2013) were the most representative and symbolic references, with the highest cocitation number and centrality, respectively. The top keywords were "radiotherapy", "volume", and "ventilation". The top Frontier keywords were "organ motion," "deep inspiration," and "deep learning". The keywords were clustered to form seven labels. Currently, the main area of research in respiratory biomechanics is respiratory motion related to imaging techniques. Future research may focus on respiratory assistance techniques and respiratory detection techniques. At the same time, in the future, we will pay attention to personalized medicine and precision medicine, so that people can monitor their health status anytime and anywhere.

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Objective. Real-time imaging, a building block of real-time adaptive radiotherapy, provides instantaneous knowledge of anatomical motion to drive delivery adaptation to improve patient safety and treatment efficacy. The temporal constraint of real-time imaging (<500 milliseconds) significantly limits the imaging signals that can be acquired, rendering volumetric imaging and 3D tumor localization extremely challenging. Real-time liver imaging is particularly difficult, compounded by the low soft tissue contrast within the liver. We proposed a deep learning (DL)-based framework (Surf-X-Bio), to track 3D liver tumor motion in real-time from combined optical surface image and a single on-board x-ray projection.Approach. Surf-X-Bio performs mesh-based deformable registration to track/localize liver tumors volumetrically via three steps. First, a DL model was built to estimate liver boundary motion from an optical surface image, using learnt motion correlations between the respiratory-induced external body surface and liver boundary. Second, the residual liver boundary motion estimation error was further corrected by a graph neural network-based DL model, using information extracted from a single x-ray projection. Finally, a biomechanical modeling-driven DL model was applied to solve the intra-liver motion for tumor localization, using the liver boundary motion derived via prior steps.Main results. Surf-X-Bio demonstrated higher accuracy and better robustness in tumor localization, as compared to surface-image-only and x-ray-only models. By Surf-X-Bio, the mean (±s.d.) 95-percentile Hausdorff distance of the liver boundary from the 'ground-truth' decreased from 9.8 (±4.5) (before motion estimation) to 2.4 (±1.6) mm. The mean (±s.d.) center-of-mass localization error of the liver tumors decreased from 8.3 (±4.8) to 1.9 (±1.6) mm.Significance. Surf-X-Bio can accurately track liver tumors from combined surface imaging and x-ray imaging. The fast computational speed (<250 milliseconds per inference) allows it to be applied clinically for real-time motion management and adaptive radiotherapy.

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

Objective. Real-time imaging is highly desirable in image-guided radiotherapy, as it provides instantaneous knowledge of patients’ anatomy and motion during treatments and enables online treatment adaptation to achieve the highest tumor targeting accuracy. Due to extremely limited acquisition time, only one or few x-ray projections can be acquired for real-time imaging, which poses a substantial challenge to localize the tumor from the scarce projections. For liver radiotherapy, such a challenge is further exacerbated by the diminished contrast between the tumor and the surrounding normal liver tissues. Here, we propose a framework combining graph neural network-based deep learning and biomechanical modeling to track liver tumor in real-time from a single onboard x-ray projection. Approach. Liver tumor tracking is achieved in two steps. First, a deep learning network is developed to predict the liver surface deformation using image features learned from the x-ray projection. Second, the intra-liver deformation is estimated through biomechanical modeling, using the liver surface deformation as the boundary condition to solve tumor motion by finite element analysis. The accuracy of the proposed framework was evaluated using a dataset of 10 patients with liver cancer. Main results. The results show accurate liver surface registration from the graph neural network-based deep learning model, which translates into accurate, fiducial-less liver tumor localization after biomechanical modeling (<1.2 (±1.2) mm average localization error). Significance. The method demonstrates its potentiality towards intra-treatment and real-time 3D liver tumor monitoring and localization. It could be applied to facilitate 4D dose accumulation, multi-leaf collimator tracking and real-time plan adaptation. The method can be adapted to other anatomical sites as well.