A reinforcement learning application of a guided Monte Carlo Tree Search algorithm for beam orientation selection in radiation therapy

Applications of artificial intelligence for machine- and patient-specific quality assurance in radiation therapy: current status and future directions

Machine- and patient-specific quality assurance (QA) is essential to ensure the safety and accuracy of radiotherapy. QA methods have become complex, especially in high-precision radiotherapy such as intensity-modulated radiation therapy (IMRT) and volumetric modulated arc therapy (VMAT), and various recommendations have been reported by AAPM Task Groups. With the widespread use of IMRT and VMAT, there is an emerging demand for increased operational efficiency. Artificial intelligence (AI) technology is quickly growing in various fields owing to advancements in computers and technology. In the radiotherapy treatment process, AI has led to the development of various techniques for automated segmentation and planning, thereby significantly enhancing treatment efficiency. Many new applications using AI have been reported for machine- and patient-specific QA, such as predicting machine beam data or gamma passing rates for IMRT or VMAT plans. Additionally, these applied technologies are being developed for multicenter studies. In the current review article, AI application techniques in machine- and patient-specific QA have been organized and future directions are discussed. This review presents the learning process and the latest knowledge on machine- and patient-specific QA. Moreover, it contributes to the understanding of the current status and discusses the future directions of machine- and patient-specific QA.

Advancing equitable and personalized cancer care: Novel applications and priorities of artificial intelligence for fairness and inclusivity in the patient care workflow

Patient care workflows are highly multimodal and intertwined: the intersection of data outputs provided from different disciplines and in different formats remains one of the main challenges of modern oncology. Artificial Intelligence (AI) has the potential to revolutionize the current clinical practice of oncology owing to advancements in digitalization, database expansion, computational technologies, and algorithmic innovations that facilitate discernment of complex relationships in multimodal data. Within oncology, radiation therapy (RT) represents an increasingly complex working procedure, involving many labor-intensive and operator-dependent tasks. In this context, AI has gained momentum as a powerful tool to standardize treatment performance and reduce inter-observer variability in a time-efficient manner. This review explores the hurdles associated with the development, implementation, and maintenance of AI platforms and highlights current measures in place to address them. In examining AI's role in oncology workflows, we underscore that a thorough and critical consideration of these challenges is the only way to ensure equitable and unbiased care delivery, ultimately serving patients' survival and quality of life.

Deep-learning Method for the Prediction of Three-Dimensional Dose Distribution for Left Breast Cancer Conformal Radiation Therapy

AIMS

An increase in the demand of a new generation of radiotherapy planning systems based on learning approaches has been reported. At this stage, the new approach is able to improve the planning speed while saving a reasonable level of plan quality, compared with available planning systems. We believe that new achievements, such as deep-learning models, will be able to review the issue from a different point of view.

MATERIALS AND METHODS

The data of 120 breast cancer patients were used to train and test the three-dimensional U-Res-Net model. The network input was computed tomography images and patients' contouring, while the patients' dose distribution was addressed as the output of the model proposed. The predicted dose distributions, created by the model for 10 test patients, were then compared with corresponding dose distributions calculated by a reliable treatment planning system. In particular, the dice similarity coefficients for different isodose volumes, dose difference and mean absolute errors (MAE) for all voxels inside the body, Dmean, D98%, D50%, D2%, V95% for planning target volume and organs at risk were calculated and were statistically analysed with the paired-samples t-test.

RESULTS

The average dose difference for all patients and voxels in body was 0.60 ± 2.81%. The MAE varied from 3.85 ± 6.65% to 8.06 ± 10.00%. The average MAE for test cases was 5.71 ± 1.19%. The average dice similarity coefficients for isodose volumes was 0.91 ± 0.03. The three-dimensional gamma passing rates with 3 mm/3% criteria varied from 78.99% to 97.58% for planning target volume and organs at risk, respectively.

CONCLUSIONS

The investigation showed that a deep-learning model can be applied to predict the three-dimensional dose distribution with optimal accuracy and precision for patients with left breast cancer. As further study, the model can be extended to predict dose distribution in other cancers.

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

MOTIVATION

Medical image analysis involves a series of tasks used to assist physicians in qualitative and quantitative analyses of lesions or anatomical structures which can significantly improve the accuracy and reliability of medical diagnoses and prognoses. Traditionally, these tedious tasks were finished by experienced physicians or medical physicists and were marred with two major problems, low efficiency and bias. In the past decade, many machine learning methods have been applied to accelerate and automate the image analysis process. Compared to the enormous deployments of supervised and unsupervised learning models, attempts to use reinforcement learning in medical image analysis are still scarce. We hope that this review article could serve as the stepping stone for related research in the future.

SIGNIFICANCE

We found that although reinforcement learning has gradually gained momentum in recent years, many researchers in the medical analysis field still find it hard to understand and deploy in clinical settings. One possible cause is a lack of well-organized review articles intended for readers without professional computer science backgrounds. Rather than to provide a comprehensive list of all reinforcement learning models applied in medical image analysis, the aim of this review is to help the readers formulate and solve their medical image analysis research through the lens of reinforcement learning.

APPROACH & RESULTS

We selected published articles from Google Scholar and PubMed. Considering the scarcity of related articles, we also included some outstanding newest preprints. The papers were carefully reviewed and categorized according to the type of image analysis task. In this article, we first reviewed the basic concepts and popular models of reinforcement learning. Then, we explored the applications of reinforcement learning models in medical image analysis. Finally, we concluded the article by discussing the reviewed reinforcement learning approaches' limitations and possible future improvements.

Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey

Reinforcement learning takes sequential decision-making approaches by learning the policy through trial and error based on interaction with the environment. Combining deep learning and reinforcement learning can empower the agent to learn the interactions and the distribution of rewards from state-action pairs to achieve effective and efficient solutions in more complex and dynamic environments. Deep reinforcement learning (DRL) has demonstrated astonishing performance in surpassing the human-level performance in the game domain and many other simulated environments. This paper introduces the basics of reinforcement learning and reviews various categories of DRL algorithms and DRL models developed for medical image analysis and radiation treatment planning optimization. We will also discuss the current challenges of DRL and approaches proposed to make DRL more generalizable and robust in a real-world environment. DRL algorithms, by fostering the designs of the reward function, agents interactions and environment models, can resolve the challenges from scarce and heterogeneous annotated medical image data, which has been a major obstacle to implementing deep learning models in the clinic. DRL is an active research area with enormous potential to improve deep learning applications in medical imaging and radiation therapy planning.

Comparing Multi-Objective Local Search Algorithms for the Beam Angle Selection Problem

In intensity-modulated radiation therapy, treatment planners aim to irradiate the tumour according to a medical prescription while sparing surrounding organs at risk as much as possible. Although this problem is inherently a multi-objective optimisation (MO) problem, most of the models in the literature are single-objective ones. For this reason, a large number of single-objective algorithms have been proposed in the literature to solve such single-objective models rather than multi-objective ones. Further, a difficulty that one has to face when solving the MO version of the problem is that the algorithms take too long before converging to a set of (approximately) non-dominated points. In this paper, we propose and compare three different strategies, namely random PLS (rPLS), judgement-function-guided PLS (jPLS) and neighbour-first PLS (nPLS), to accelerate a previously proposed Pareto local search (PLS) algorithm to solve the beam angle selection problem in IMRT. A distinctive feature of these strategies when compared to the PLS algorithms in the literature is that they do not evaluate their entire neighbourhood before performing the dominance analysis. The rPLS algorithm randomly chooses the next non-dominated solution in the archive and it is used as a baseline for the other implemented algorithms. The jPLS algorithm first chooses the non-dominated solution in the archive that has the best objective function value. Finally, the nPLS algorithm first chooses the solutions that are within the neighbourhood of the current solution. All these strategies prevent us from evaluating a large set of BACs, without any major impairment in the obtained solutions’ quality. We apply our algorithms to a prostate case and compare the obtained results to those obtained by the PLS from the literature. The results show that algorithms proposed in this paper reach a similar performance than PLS and require fewer function evaluations.