Introduction to machine and deep learning for medical physicists

Evaluation of MRI anatomy in machine learning predictive models to assess hydrogel spacer benefit for prostate cancer patients

Introduction

Hydrogel spacers (HS) are designed to minimise the radiation doses to the rectum in prostate cancer radiation therapy (RT) by creating a physical gap between the rectum and the target treatment volume inclusive of the prostate and seminal vesicles (SV). This study aims to determine the feasibility of incorporating diagnostic MRI (dMRI) information in statistical machine learning (SML) models developed with planning CT (pCT) anatomy for dose and rectal toxicity prediction. The SML models aim to support HS insertion decision-making prior to RT planning procedures.

Methods

Regions of interest (ROIs) were retrospectively contoured on the pCT and registered dMRI scans for 20 patients. ROI Dice and Hausdorff distance (HD) comparison metrics were calculated. The ROI and patient clinical risk factors (CRFs) variables were inputted into three SML models and then pCT and dMRI-based dose and toxicity model performance compared through confusion matrices, AUC curves, accuracy performance metric results and observed patient outcomes.

Results

Average Dice values comparing dMRI and pCT ROIs were 0.81, 0.47 and 0.71 for the prostate, SV, and rectum respectively. Average Hausdorff distances were 2.15, 2.75 and 2.75 mm for the prostate, SV, and rectum respectively. The average accuracy metric across all models was 0.83 when using dMRI ROIs and 0.85 when using pCT ROIs.

Conclusion

Differences between pCT and dMRI anatomical ROI variables did not impact SML model performance in this study, demonstrating the feasibility of using dMRI images. Due to the limited sample size further training of the predictive models including dMRI anatomy is recommended.

Accuracy of artificial intelligence in detecting tumor bone metastases: a systematic review and meta-analysis

BACKGROUND

Bone metastases (BM) represent a prevalent complication of tumors. Early and accurate diagnosis, however, is a significant hurdle for radiologists. Recently, artificial intelligence (AI) has emerged as a valuable tool to assist radiologists in the detection of BM. This meta-analysis was undertaken to evaluate the AI diagnostic accuracy for BM.

METHODS

Two reviewers performed an exhaustive search of several databases, including Wei Pu (VIP) database, China National Knowledge Infrastructure (CNKI), Web of Science, Cochrane Library, Ovid-Embase, Ovid-Medline, Wan Fang database, and China Biology Medicine (CBM), from their inception to December 2024. This search focused on studies that developed and/or validated AI techniques for detecting BM in magnetic resonance imaging (MRI) or computed tomography (CT). A hierarchical model was used in the meta-analysis to calculate diagnostic odds ratio (DOR), negative likelihood ratio (NLR), positive likelihood ratio (PLR), area under the curve (AUC), specificity (SP), and pooled sensitivity (SE). The risk of bias and applicability were assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST), while the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis-artificial intelligence (TRIPOD-AI) was employed for evaluating the quality of evidence.

RESULT

This review covered 20 articles, among them, 16 studies were included in the meta-analysis. The results revealed a pooled SE of 0.88 (0.82-0.92), a pooled SP of 0.89 (0.84-0.93), a pooled AUC of 0.95 (0.92-0.96), PLR of 8.1 (5.57-11.80), NLR of 0.14 (0.09-0.21) and DOR of 58 (31-109). When focusing on imaging algorithms. Based on ML, a pooled SE of 0.88 (0.77-0.92), SP 0.88 (0.82-0.92), and AUC 0.93 (0.91-0.95). Based on DL, a pooled SE of 0.89 (0.81-0.95), SP 0.89 (0.81-0.94), and AUC 0.95 (0.93-0.97).

CONCLUSION

This meta-analysis underscores the substantial diagnostic value of AI in identifying BM. Nevertheless, in-depth large-scale prospective research should be carried out for confirming AI's clinical utility in BM management.

A deep learning algorithm that aids visualization of femoral neck fractures and improves physician training

PURPOSE

Missed fractures are the most common radiologic error in clinical practice, and erroneous classification could lead to inappropriate treatment and unfavorable prognosis. Here, we developed a fully automated deep learning model to detect and classify femoral neck fractures using plain radiographs, and evaluated its utility for diagnostic assistance and physician training.

METHODS

1527 plain pelvic and hip radiographs obtained between April 2014 and July 2023 at our Hospital were selected for the model training and evaluation. Faster R-CNN was used to locate the femoral neck. DenseNet-121 was used for Garden classification of the femoral neck fracture, while an additional segmentation method used to visualize the probable fracture area. The model was assessed by the area under the receiver operating characteristic curve (AUC). The accuracy, sensitivity, and specificity for clinicians fracture detection in the diagnostic assistance and physician training experiments were determined.

RESULTS

The accuracy of the model for fracture detection was 94.1 %. The model achieved AUCs of 0.99 for no femoral neck fractures, 0.94 for Garden I/II fractures, and 0.99 for Garden III/IV fractures. In the diagnostic assistance study, the emergency physicians had an average accuracy of 86.33 % unaided and 92.03 % aided, sensitivity of 85.94 % unaided and 91.78 % aided, and specificity of 87.88 % unaided and 93.13 % aided in detecting fractures. In the physician training study, the accuracy, sensitivity, and specificity of the trainees for fracture classification were 81.83 %, 77.28 %, and 84.85 %, respectively, before training, compared with 90.65 %, 88.31 %, and 92.21 %, respectively, after training.

CONCLUSIONS

The model represents a valuable tool for physicians to better visualize fractures and improve training outcomes, indicating deep learning algorithms as a promising approach to improve clinical practice and medical education.

Transformative artificial intelligence in gastric cancer: Advancements in diagnostic techniques

Gastric cancer represents a significant global health challenge with elevated incidence and mortality rates, highlighting the need for advancements in diagnostic and therapeutic strategies. This review paper addresses the critical need for a thorough synthesis of the role of artificial intelligence (AI) in the management of gastric cancer. It provides an in-depth analysis of current AI applications, focusing on their contributions to early diagnosis, treatment planning, and outcome prediction. The review identifies key gaps and limitations in the existing literature by examining recent studies and technological developments. It aims to clarify the evolution of AI-driven methods and their impact on enhancing diagnostic accuracy, personalizing treatment strategies, and improving patient outcomes. The paper emphasizes the transformative potential of AI in overcoming the challenges associated with gastric cancer management and proposes future research directions to further harness AI's capabilities. Through this synthesis, the review underscores the importance of integrating AI technologies into clinical practice to revolutionize gastric cancer management.

Image biomarkers and explainable AI: handcrafted features versus deep learned features

Feature extraction and selection from medical data are the basis of radiomics and image biomarker discovery for various architectures, including convolutional neural networks (CNNs). We herein describe the typical radiomics steps and the components of a CNN for both deep feature extraction and end-to-end approaches. We discuss the curse of dimensionality, along with dimensionality reduction techniques. Despite the outstanding performance of deep learning (DL) approaches, the use of handcrafted features instead of deep learned features needs to be considered for each specific study. Dataset size is a key factor: large-scale datasets with low sample diversity could lead to overfitting; limited sample sizes can provide unstable models. The dataset must be representative of all the "facets" of the clinical phenomenon/disease investigated. The access to high-performance computational resources from graphics processing units is another key factor, especially for the training phase of deep architectures. The advantages of multi-institutional federated/collaborative learning are described. When large language models are used, high stability is needed to avoid catastrophic forgetting in complex domain-specific tasks. We highlight that non-DL approaches provide model explainability superior to that provided by DL approaches. To implement explainability, the need for explainable AI arises, also through post hoc mechanisms. RELEVANCE STATEMENT: This work aims to provide the key concepts for processing the imaging features to extract reliable and robust image biomarkers. KEY POINTS: The key concepts for processing the imaging features to extract reliable and robust image biomarkers are provided. The main differences between radiomics and representation learning approaches are highlighted. The advantages and disadvantages of handcrafted versus learned features are given without losing sight of the clinical purpose of artificial intelligence models.

Robust machine learning challenge: An AIFM multicentric competition to spread knowledge, identify common pitfalls and recommend best practice

PURPOSE

A novel and unconventional approach to a machine learning challenge was designed to spread knowledge, identify robust methods and highlight potential pitfalls about machine learning within the Medical Physics community.

METHODS

A public dataset comprising 41 radiomic features and 535 patients was employed to assess the potential of radiomics in distinguishing between primary lung tumors and metastases. Each participant developed two classification models using: (i) all features (base model); (ii) only robust features (robust model). Both models were validated with cross-validation and on unseen data. The population stability index (PSI) was used as diagnostic metric for implementation issues. Performance was compared to reference. Base and robust models were compared in terms of performance and stability (coefficient of variation (CoV) of prediction probabilities).

RESULTS

PSI detected potential implementation errors in 70 % of models. The dataset exhibited strong imbalance. The average Gmean (i.e. an appropriate metric for imbalance) among all participants was 0.67 ± 0.01, significantly higher than reference Gmean of 0.50 ± 0.04. Robust models performances were slightly worse than base models (p < 0.05). Regarding stability, robust models exhibited lower median CoV on training set only.

CONCLUSION

AI4MP-Challenge models overperformed the reference, significantly improving the Gmean. Exclusion of less-robust features did not improve model robustness and it should be avoided when confounding effects are absent. Other methods, like harmonization or data augmentation, should be evaluated. This study demonstrated how the collaborative effort to foster knowledge on machine learning among medical physicists, through interactive sessions and exchange of information among participants, can result in improved models.

Enhanced IDOL segmentation framework using personalized hyperspace learning IDOL

BACKGROUND

Adaptive radiotherapy (ART) workflows have been increasingly adopted to achieve dose escalation and tissue sparing under shifting anatomic conditions, but the necessity of recontouring and the associated time burden hinders a real-time or online ART workflow. In response to this challenge, approaches to auto-segmentation involving deformable image registration, atlas-based segmentation, and deep learning-based segmentation (DLS) have been developed. Despite the particular promise shown by DLS methods, implementing these approaches in a clinical setting remains a challenge, namely due to the difficulty of curating a data set of sufficient size and quality so as to achieve generalizability in a trained model.

PURPOSE

To address this challenge, we have developed an intentional deep overfit learning (IDOL) framework tailored to the auto-segmentation task. However, certain limitations were identified, particularly the insufficiency of the personalized dataset to effectively overfit the model. In this study, we introduce a personalized hyperspace learning (PHL)-IDOL segmentation framework capable of generating datasets that induce the model to overfit specific patient characteristics for medical image segmentation.

METHODS

The PHL-IDOL model is trained in two stages. In the first, a conventional, general model is trained with a diverse set of patient data (n = 100 patients) consisting of CT images and clinical contours. Following this, the general model is tuned with a data set consisting of two components: (a) selection of a subset of the patient data (m < n) using the similarity metrics (mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and the universal quality image index (UQI) values); (b) adjust the CT and the clinical contours using a deformed vector generated from the reference patient and the selected patients using (a). After training, the general model, the continual model, the conventional IDOL model, and the proposed PHL-IDOL model were evaluated using the volumetric dice similarity coefficient (VDSC) and the Hausdorff distance 95% (HD95%) computed for 18 structures in 20 test patients.

RESULTS

Implementing the PHL-IDOL framework resulted in improved segmentation performance for each patient. The Dice scores increased from 0.81 ± $ \pm $ 0.05 with the general model, 0.83± 0.04 $ \pm 0.04$ for the continual model, 0.83± 0.04 $ \pm 0.04$ for the conventional IDOL model to an average of 0.87± 0.03 $ \pm 0.03$ with the PHL-IDOL model. Similarly, the Hausdorff distance decreased from 3.06± 0.99 $ \pm 0.99$ with the general model, 2.84± 0.69 $ \pm 0.69$ for the continual model, 2.79± 0.79 $ \pm 0.79$ for the conventional IDOL model and 2.36± 0.52 $ \pm 0.52$ for the PHL-IDOL model. All the standard deviations were decreased by nearly half of the values comparing the general model and the PHL-IDOL model.

CONCLUSION

The PHL-IDOL framework applied to the auto-segmentation task achieves improved performance compared to the general DLS approach, demonstrating the promise of leveraging patient-specific prior information in a task central to online ART workflows.

Impact of annotation imperfections and auto-curation for deep learning-based organ-at-risk segmentation

Background and purpose

Segmentation imperfections (noise) in radiotherapy organ-at-risk segmentation naturally arise from specialist experience and image quality. Using clinical contours can result in sub-optimal convolutional neural network (CNN) training and performance, but manual curation is costly. We address the impact of simulated and clinical segmentation noise on CNN parotid gland (PG) segmentation performance and provide proof-of-concept for an easily implemented auto-curation countermeasure.

Methods and Materials

The impact of segmentation imperfections was investigated by simulating noise in clean, high-quality segmentations. Curation efficacy was tested by removing lowest-scoring Dice similarity coefficient (DSC) cases early during CNN training, both in simulated (5-fold) and clinical (10-fold) settings, using our full radiotherapy clinical cohort (RTCC; N = 1750 individual PGs). Statistical significance was assessed using Bonferroni-corrected Wilcoxon signed-rank tests. Curation efficacies were evaluated using DSC and mean surface distance (MSD) on in-distribution and out-of-distribution data and visual inspection.

Results

The curation step correctly removed median(range) 98(90-100)% of corrupted segmentations and restored the majority (1.2 %/1.3 %) of DSC lost from training with 30 % corrupted segmentations. This effect was masked when using typical (non-curated) validation data. In RTCC, 20 % curation showed improved model generalizability which significantly improved out-of-distribution DSC and MSD (p < 1.0e-12, p < 1.0e-6). Improved consistency was observed in particularly the medial and anterior lobes.

Conclusions

Up to 30% case removal, the curation benefit outweighed the training variance lost through curation. Considering the notable ease of implementation, high sensitivity in simulations and performance gains already at lower curation fractions, as a conservative middle ground, we recommend 15% curation of training cases when training CNNs using clinical PG contours.

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy

BACKGROUND AND PURPOSE

Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap.

METHODS AND MATERIALS

A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended.

RESULTS

The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated.

CONCLUSION

A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions

Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.

Developments and future prospects of personalized medicine in head and neck squamous cell carcinoma diagnoses and treatments

BACKGROUND

Precision healthcare has entered a new era because of the developments in personalized medicine, especially in the diagnosis and treatment of head and neck squamous cell carcinoma (HNSCC). This paper explores the dynamic landscape of personalized medicine as applied to HNSCC, encompassing both current developments and future prospects.

RECENT FINDINGS

The integration of personalized medicine strategies into HNSCC diagnosis is driven by the utilization of genetic data and biomarkers. Epigenetic biomarkers, which reflect modifications to DNA that can influence gene expression, have emerged as valuable indicators for early detection and risk assessment. Treatment approaches within the personalized medicine framework are equally promising. Immunotherapy, gene silencing, and editing techniques, including RNA interference and CRISPR/Cas9, offer innovative means to modulate gene expression and correct genetic aberrations driving HNSCC. The integration of stem cell research with personalized medicine presents opportunities for tailored regenerative approaches. The synergy between personalized medicine and technological advancements is exemplified by artificial intelligence (AI) and machine learning (ML) applications. These tools empower clinicians to analyze vast datasets, predict patient responses, and optimize treatment strategies with unprecedented accuracy.

CONCLUSION

The developments and prospects of personalized medicine in HNSCC diagnosis and treatment offer a transformative approach to managing this complex malignancy. By harnessing genetic insights, biomarkers, immunotherapy, gene editing, stem cell therapies, and advanced technologies like AI and ML, personalized medicine holds the key to enhancing patient outcomes and ushering in a new era of precision oncology.

Construction and optimization of vending machine decision support system based on improved C4.5 decision tree

The intensification of market competition makes refined operation management become the focus of attention of major manufacturers. As an important branch of artificial intelligence (AI), machine learning (ML) plays a key role in it, and has its application prospect in various systems. Based on this situation, this paper takes vending machines as the research object. On the one hand, the product classification model of vending machine is constructed based on decision tree algorithm. On the other hand, based on neural network (NN), the sales forecast model of vending machines is built. Finally, based on the above research, the theoretical framework of decision support system (DSS) for vending machines is constructed. The research shows that: (1) The accuracy of C4.5 algorithm can reach 87 % at the highest and 68 % at the lowest. The accuracy of the improved C4.5 algorithm can reach 87 % at the highest and 67 % at the lowest, with little difference between them. (2) The maximum running time of the improved C4.5 algorithm is about 5500 ms, and the minimum is close to 1 ms. In addition, the running time of all seven datasets is better than that of the unmodified algorithm. (3) When the back propagation neural network (BPNN) is used to forecast the sales of vending machines, the curve of the predicted data basically coincides with the curve of the actual data, which shows that its accuracy is high. This paper aims to build a convenient and secure DSS by taking vending machines as an example. In addition, this paper also uses reinforcement learning to optimize the research methods of this paper. It can further optimize the performance and efficiency of vending machines and provide better service experience for customers. Meanwhile, the use of reinforcement learning can make the whole system more intelligent and adaptive to better cope with the changing market environment.

Machine-learning prediction of treatment response to stereotactic body radiation therapy in oligometastatic gynecological cancer: A multi-institutional study

BACKGROUND AND PURPOSE

We aimed to develop and validate different machine-learning (ML) prediction models for the complete response of oligometastatic gynecological cancer after SBRT.

MATERIAL AND METHODS

One hundred fifty-seven patients with 272 lesions from 14 different institutions and treated with SBRT with radical intent were included. Thirteen datasets including 222 lesions were combined for model training and internal validation purposes, with an 80:20 ratio. The external testing dataset was selected as the fourteenth Institution with 50 lesions. Lesions that achieved complete response (CR) were defined as responders. Prognostic clinical and dosimetric variables were selected using the LASSO algorithm. Six supervised ML models, including logistic regression (LR), classification and regression tree analysis (CART) and support vector machine (SVM) using four different kernels, were trained and tested to predict the complete response of uterine lesions after SBRT. The performance of models was assessed by receiver operating characteristic curves (ROC), area under the curve (AUC) and calibration curves. An explainable approach based on SHapley Additive exPlanations (SHAP) method was deployed to generate individual explanations of the model's decisions.

RESULTS

63.6% of lesions had a complete response and were used as ground truth for the supervised models. LASSO strongly associated complete response with three variables, namely the lesion volume (PTV), the type of lesions (lymph-nodal versus parenchymal), and the biological effective dose (BED10), that were used as input for ML modeling. In the training set, the AUCs for complete response were 0.751 (95% CI: 0.716-0.786), 0.766 (95% CI: 0.729-0.802) and 0.800 (95% CI: 0.742-0.857) for the LR, CART and SVM with a radial basis function kernel, respectively. These models achieve AUC values of 0.727 (95% CI: 0.669-0.795), 0.734 (95% CI: 0.649-0.815) and 0.771 (95% CI: 0.717-0.824) in the external testing set, demonstrating excellent generalizability.

CONCLUSION

ML models enable a reliable prediction of the treatment response of oligometastatic lesions receiving SBRT. This approach may assist radiation oncologists to tailor more individualized treatment plans for oligometastatic patients.

Application of artificial intelligence in the assessment of thyroid eye disease (TED) - a scoping review

Background

There is emerging evidence which suggests the utility of artificial intelligence (AI) in the diagnostic assessment and pre-treatment evaluation of thyroid eye disease (TED). This scoping review aims to (1) identify the extent of the available evidence (2) provide an in-depth analysis of AI research methodology of the studies included in the review (3) Identify knowledge gaps pertaining to research in this area.

Methods

This review was performed according to the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA). We quantify the diagnostic accuracy of AI models in the field of TED assessment and appraise the quality of these studies using the modified QUADAS-2 tool.

Results

A total of 13 studies were included in this review. The most common AI models used in these studies are convolutional neural networks (CNN). The majority of the studies compared algorithm performance against healthcare professionals. The overall risk of bias and applicability using the modified Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool led to most of the studies being classified as low risk, although higher deficiency was noted in the risk of bias in flow and timing.

Conclusions

While the results of the review showed high diagnostic accuracy of the AI models in identifying features of TED relevant to disease assessment, deficiencies in study design causing study bias and compromising study applicability were noted. Moving forward, limitations and challenges inherent to machine learning should be addressed with improved standardized guidance around study design, reporting, and legislative framework.

Generation of fluoroscopy-alike radiographs as alternative datasets for deep learning in interventional radiology

In fluoroscopy-guided interventions (FGIs), obtaining large quantities of labelled data for deep learning (DL) can be difficult. Synthetic labelled data can serve as an alternative, generated via pseudo 2D projections of CT volumetric data. However, contrasted vessels have low visibility in simple 2D projections of contrasted CT data. To overcome this, we propose an alternative method to generate fluoroscopy-like radiographs from contrasted head CT Angiography (CTA) volumetric data. The technique involves segmentation of brain tissue, bone, and contrasted vessels from CTA volumetric data, followed by an algorithm to adjust HU values, and finally, a standard ray-based projection is applied to generate the 2D image. The resulting synthetic images were compared to clinical fluoroscopy images for perceptual similarity and subject contrast measurements. Good perceptual similarity was demonstrated on vessel-enhanced synthetic images as compared to the clinical fluoroscopic images. Statistical tests of equivalence show that enhanced synthetic and clinical images have statistically equivalent mean subject contrast within 25% bounds. Furthermore, validation experiments confirmed that the proposed method for generating synthetic images improved the performance of DL models in certain regression tasks, such as localizing anatomical landmarks in clinical fluoroscopy images. Through enhanced pseudo 2D projection of CTA volume data, synthetic images with similar features to real clinical fluoroscopic images can be generated. The use of synthetic images as an alternative source for DL datasets represents a potential solution to the application of DL in FGIs procedures.

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images

PURPOSE

Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions.

METHODS

We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0 × 1.0× 1.5) mm3 . A volume patch (192 × 192 × 96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data.

RESULTS

SwinAttUNet, DSC values were 86.54 ± 1.21, 94.15 ± 1.17, and 87.15 ± 1.68% and HD95 values were 5.06 ± 1.42, 3.16 ± 0.93, and 5.54 ± 1.63 mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45 ± 0.57, 0.82 ± 0.12, and 1.42 ± 0.38 mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90 ± 0.80, 96.16 ± 0.76, 93.74 ± 2.25, and 89.31 ± 3.87%. Respective HD95 values were: 5.13 ± 4.11, 2.73 ± 1.19, 2.29 ± 1.47, and 5.31 ± 1.25 mm. Respective ASD values were: 1.88 ± 1.45, 1.78 ± 1.21, 0.71 ± 0.43, and 1.21 ± 1.11 mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section.

CONCLUSIONS

We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.

Explainable Machine Learning with Pairwise Interactions for Predicting Conversion from Mild Cognitive Impairment to Alzheimer's Disease Utilizing Multi-Modalities Data

BACKGROUND

Predicting cognition decline in patients with mild cognitive impairment (MCI) is crucial for identifying high-risk individuals and implementing effective management. To improve predicting MCI-to-AD conversion, it is necessary to consider various factors using explainable machine learning (XAI) models which provide interpretability while maintaining predictive accuracy. This study used the Explainable Boosting Machine (EBM) model with multimodal features to predict the conversion of MCI to AD during different follow-up periods while providing interpretability.

METHODS

This retrospective case-control study is conducted with data obtained from the ADNI database, with records of 1042 MCI patients from 2006 to 2022 included. The exposures included in this study were MRI biomarkers, cognitive scores, demographics, and clinical features. The main outcome was AD conversion from aMCI during follow-up. The EBM model was utilized to predict aMCI converting to AD based on three feature combinations, obtaining interpretability while ensuring accuracy. Meanwhile, the interaction effect was considered in the model. The three feature combinations were compared in different follow-up periods with accuracy, sensitivity, specificity, and AUC-ROC. The global and local explanations are displayed by importance ranking and feature interpretability plots.

RESULTS

The five-years prediction accuracy reached 85% (AUC = 0.92) using both cognitive scores and MRI markers. Apart from accuracies, we obtained features' importance in different follow-up periods. In early stage of AD, the MRI markers play a major role, while for middle-term, the cognitive scores are more important. Feature risk scoring plots demonstrated insightful nonlinear interactive associations between selected factors and outcome. In one-year prediction, lower right inferior temporal volume (<9000) is significantly associated with AD conversion. For two-year prediction, low left inferior temporal thickness (<2) is most critical. For three-year prediction, higher FAQ scores (>4) is the most important. During four-year prediction, APOE4 is the most critical. For five-year prediction, lower right entorhinal volume (<1000) is the most critical feature.

CONCLUSIONS

The established glass-box model EBMs with multimodal features demonstrated a superior ability with detailed interpretability in predicting AD conversion from MCI. Multi features with significant importance were identified. Further study may be of significance to determine whether the established prediction tool would improve clinical management for AD patients.

Overview of medical physics education and research programs in a non-academic environment

Northwest Medical Physics Center (NMPC) is a nonprofit organization that provides clinical physics support to over 35 radiation therapy facilities concentrated in the Pacific Northwest. Although clinical service is the primary function of NMPC, the diverse array of clinical sites and physics expertise has allowed for the establishment of structured education and research programs, which are complementary to the organization's clinical mission. Three clinical training programs have been developed at NMPC: a therapy medical physics residency program accredited by the Commission on Accreditation of Medical Physics Education Programs (CAMPEP), an Applied Physics Technologist (APT) program, and a summer undergraduate internship program. A partnership has also been established with a major radiation oncology clinical vendor for the purposes of validating and testing new clinical devices across multiple facilities. These programs are managed by a dedicated education and research team at NMPC, made up of four qualified medical physicists (QMPs). The education and research work has made a significant contribution to the organization's clinical mission, and it has provided new training opportunities for early-career physicists across many different clinical environments. Education and research can be incorporated into nonacademic clinical environments, improving the quality of patient care, and increasing the number and type of training opportunities available for medical physicists.

Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration

Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.

Evaluation of the dataset quality in gamma passing rate predictions using machine learning methods

OBJECTIVE

Gamma passing rate (GPR) predictions using machine learning methods have been explored for treatment verification of radiotherapy plans. However, these methods presented datasets with unbalanced number of plans having different treatment conditions (heterogeneous datasets), such as anatomical sites or dose per fractions, leading to lower model interpretability and prediction performance.

METHODS

We investigated the impact of the dataset composition on GPR binary classification (pass/fail) using random forest (RF), XG-boost, and neural network (NN) models. 945 plans were used to create one reference dataset (randomly assembled) and 24 customized datasets that considered four heterogeneity factors independently (anatomical region, number of arcs, dose per fraction, and treatment unit). 309 predictor features were extracted and calculated from plan parameters, modulation complexity metrics, and radiomic analysis (leave-trajectory maps, 3D dose distributions, and portal dosimetry images). The models' performances were measured using the area under the curve from the receiver operating characteristic (ROC-AUC).

RESULTS

Radiomics features for reference models increased ROC-AUC values up to 13%, 15%, and 5% for RF, XG-Boost, and NN, respectively. The datasets with higher heterogeneous conditions presented the lower ROC-AUC values (RF: 0.72 ± 0.11, XG-Boost: 0.67 ± 0.1, NN: 0.89 ± 0.05) compared to models with less heterogeneous treatment conditions (RF: 0.88 ± 0.06, XG-Boost: 0.89 ± 0.07, NN: 0.98 ± 0.01). The ten most important features for each heterogeneity dataset group demonstrated their correlation with the treatments' physical aspects and GPR prediction.

CONCLUSION

Improvements in data generalization and model performances can be associated with datasets having similar treatment conditions. This analysis might be implemented to evaluate the dataset quality and model consistency of further ML applications in radiotherapy.

ADVANCES IN KNOWLEDGE

Dataset heterogeneities decrease ML model performance and reliability.

Artificial Intelligence in Acute Ischemic Stroke Subtypes According to Toast Classification: A Comprehensive Narrative Review

The correct recognition of the etiology of ischemic stroke (IS) allows tempestive interventions in therapy with the aim of treating the cause and preventing a new cerebral ischemic event. Nevertheless, the identification of the cause is often challenging and is based on clinical features and data obtained by imaging techniques and other diagnostic exams. TOAST classification system describes the different etiologies of ischemic stroke and includes five subtypes: LAAS (large-artery atherosclerosis), CEI (cardio embolism), SVD (small vessel disease), ODE (stroke of other determined etiology), and UDE (stroke of undetermined etiology). AI models, providing computational methodologies for quantitative and objective evaluations, seem to increase the sensitivity of main IS causes, such as tomographic diagnosis of carotid stenosis, electrocardiographic recognition of atrial fibrillation, and identification of small vessel disease in magnetic resonance images. The aim of this review is to provide overall knowledge about the most effective AI models used in the differential diagnosis of ischemic stroke etiology according to the TOAST classification. According to our results, AI has proven to be a useful tool for identifying predictive factors capable of subtyping acute stroke patients in large heterogeneous populations and, in particular, clarifying the etiology of UDE IS especially detecting cardioembolic sources.

Evaluation of deep learning based implanted fiducial markers tracking in pancreatic cancer patients

Real-time target position verification during pancreas stereotactic body radiation therapy (SBRT) is important for the detection of unplanned tumour motions. Fast and accurate fiducial marker segmentation is a Requirement of real-time marker-based verification. Deep learning (DL) segmentation techniques are ideal because they don't require additional learning imaging or prior marker information (e.g., shape, orientation). In this study, we evaluated three DL frameworks for marker tracking applied to pancreatic cancer patient data. The DL frameworks evaluated were (1) a convolutional neural network (CNN) classifier with sliding window, (2) a pretrained you-only-look-once (YOLO) version-4 architecture, and (3) a hybrid CNN-YOLO. Intrafraction kV images collected during pancreas SBRT treatments were used as training data (44 fractions, 2017 frames). All patients had 1-4 implanted fiducial markers. Each model was evaluated on unseen kV images (42 fractions, 2517 frames). The ground truth was calculated from manual segmentation and triangulation of markers in orthogonal paired kV/MV images. The sensitivity, specificity, and area under the precision-recall curve (AUC) were calculated. In addition, the mean-absolute-error (MAE), root-mean-square-error (RMSE) and standard-error-of-mean (SEM) were calculated for the centroid of the markers predicted by the models, relative to the ground truth. The sensitivity and specificity of the CNN model were 99.41% and 99.69%, respectively. The AUC was 0.9998. The average precision of the YOLO model for different values of recall was 96.49%. The MAE of the three models in the left-right, superior-inferior, and anterior-posterior directions were under 0.88 ± 0.11 mm, and the RMSE were under 1.09 ± 0.12 mm. The detection times per frame on a GPU were 48.3, 22.9, and 17.1 milliseconds for the CNN, YOLO, and CNN-YOLO, respectively. The results demonstrate submillimeter accuracy of marker position predicted by DL models compared to the ground truth. The marker detection time was fast enough to meet the requirements for real-time application.

HaN-Seg: The head and neck organ-at-risk CT and MR segmentation dataset

PURPOSE

For the cancer in the head and neck (HaN), radiotherapy (RT) represents an important treatment modality. Segmentation of organs-at-risk (OARs) is the starting point of RT planning, however, existing approaches are focused on either computed tomography (CT) or magnetic resonance (MR) images, while multimodal segmentation has not been thoroughly explored yet. We present a dataset of CT and MR images of the same patients with curated reference HaN OAR segmentations for an objective evaluation of segmentation methods.

ACQUISITION AND VALIDATION METHODS

The cohort consists of HaN images of 56 patients that underwent both CT and T1-weighted MR imaging for image-guided RT. For each patient, reference segmentations of up to 30 OARs were obtained by experts performing manual pixel-wise image annotation. By maintaining the distribution of patient age and gender, and annotation type, the patients were randomly split into training Set 1 (42 cases or 75%) and test Set 2 (14 cases or 25%). Baseline auto-segmentation results are also provided by training the publicly available deep nnU-Net architecture on Set 1, and evaluating its performance on Set 2.

DATA FORMAT AND USAGE NOTES

The data are publicly available through an open-access repository under the name HaN-Seg: The Head and Neck Organ-at-Risk CT & MR Segmentation Dataset. Images and reference segmentations are stored in the NRRD file format, where the OAR filenames correspond to the nomenclature recommended by the American Association of Physicists in Medicine, and OAR and demographics information is stored in separate comma-separated value  files.

POTENTIAL APPLICATIONS

The HaN-Seg: The Head and Neck Organ-at-Risk CT & MR Segmentation Challenge is launched in parallel with the dataset release to promote the development of automated techniques for OAR segmentation in the HaN. Other potential applications include out-of-challenge algorithm development and benchmarking, as well as external validation of the developed algorithms.

Machine learning for predicting accuracy of lung and liver tumor motion tracking using radiomic features

Background

Internal tumor motion is commonly predicted using external respiratory signals. However, the internal/external correlation is complex and patient-specific. The purpose of this study was to develop various models based on the radiomic features of computed tomography (CT) images to predict the accuracy of tumor motion tracking using external surrogates and to find accurate and reliable tracking algorithms.

Methods

Images obtained from a total of 108 and 71 patients pathologically diagnosed with lung and liver cancers, respectively, were examined. Real-time position monitoring motion was fitted to tumor motion, and samples with fitting errors greater than 2 mm were considered positive. Radiomic features were extracted from internal target volumes of average intensity projections, and cross-validation least absolute shrinkage and selection operator (LassoCV) was used to conduct feature selection. Based on the radiomic features, a total of 26 separate models (13 for the lung and 13 for the liver) were trained and tested. Area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were used to assess performance. Relative standard deviation was used to assess stability.

Results

Thirty-three and 22 radiomic features were selected for the lung and liver, respectively. For the lung, the AUC varied from 0.848 (decision tree) to 0.941 [support vector classifier (SVC), logistic regression]; sensitivity varied from 0.723 (extreme gradient boosting) to 0.848 [linear support vector classifier (linearSVC)]; specificity varied from 0.834 (gaussian naive bayes) to 0.936 [multilayer perceptron (MLP), wide and deep (W&D)]; and MLP and W&D had better performance and stability than the median. For the liver, the AUC varied from 0.677 [light gradient boosting machine (Light)] to 0.892 (logistic regression); sensitivity varied from 0.717 (W&D) to 0.862 (MLP); specificity varied from 0.566 (Light) to 0.829 (linearSVC); and logistic regression, MLP, and SVC had better performance and stability than the median.

Conclusions

Respiratory-sensitive radiomic features extracted from CT images of lung and liver tumors were proved to contain sufficient information to establish an external/internal motion relationship. We developed a rapid and accurate method based on radiomics to classify the accuracy of monitoring a patient's external surface for lung and liver tumor tracking. Several machine learning algorithms-in particular, MLP-demonstrated excellent classification performance and stability.

Six degrees of freedom intrafraction cranial motion detection using a novel capacitive monitoring technique: evaluation with human subjects

The purpose of this work is to introduce and evaluate a capacitive monitoring array capable of continuous 6DOF cranial motion detection during high precision radiotherapy. The ring-shaped capacitive array consists of four equally sized conductive sensors positioned at the cranial vertex. The system is modular, non-contact, and provides continuous motion information through the thermoplastic immobilization mask without relying on skin monitoring or use of ionizing radiation. The array performance was evaluated through a volunteer study with a cohort of twenty-five individuals. The study was conducted in a linac suite and the volunteers were fitted with an S-frame thermoplastic mask. Each volunteer took part in one data acquisition session per day for three consecutive days. During the data acquisition, the conductive array was translated and rotated relative to their immobilized cranium in 1-millimetre and 1-degree steps to simulate cranial motion. Capacitive signals were collected at each position at a frequency of 20 Hz. The data from the first acquisition session was then used to train a classifier model and establish calibration equations. The classifier and calibration equations were then applied to data from the subsequent acquisition sessions to evaluate the system performance. The trained classifiers had an average success rate of 92.6% over the volunteer cohort. The average error associated with calibration had a mean value below 0.1 mm or 0.1 deg for all six motions. The capacitive array system provides a novel method to detect translational and rotational cranial motion through a thermoplastic mask.

The role of artificial intelligence in veterinary radiation oncology

Veterinary radiation oncology regularly deploys sophisticated contouring, image registration, and treatment planning optimization software for patient care. Over the past decade, advances in computing power and the rapid development of neural networks, open-source software packages, and data science have been realized and resulted in new research and clinical applications of artificial intelligent (AI) systems in radiation oncology. These technologies differ from conventional software in their level of complexity and ability to learn from representative and local data. We provide clinical and research application examples of AI in human radiation oncology and their potential applications in veterinary medicine throughout the patient's care-path: from treatment simulation, deformable registration, auto-segmentation, automated treatment planning and plan selection, quality assurance, adaptive radiotherapy, and outcomes modeling. These technologies have the potential to offer significant time and cost savings in the veterinary setting; however, since the range of usefulness of these technologies have not been well studied nor understood, care must be taken if adopting AI technologies in clinical practice. Over the next several years, some practical and realizable applications of AI in veterinary radiation oncology include automated segmentation of normal tissues and tumor volumes, deformable registration, multi-criteria plan optimization, and adaptive radiotherapy. Keys in achieving success in adopting AI in veterinary radiation oncology include: establishing "truth-data"; data harmonization; multi-institutional data and collaborations; standardized dose reporting and taxonomy; adopting an open access philosophy, data collection and curation; open-source algorithm development; and transparent and platform-independent code development.

Current status and future developments in predicting outcomes in radiation oncology

Advancements in data-driven technologies and the inclusion of information-rich multiomics features have significantly improved the performance of outcomes modeling in radiation oncology. For this current trend to be sustainable, challenges related to robust data modeling such as small sample size, low size to feature ratio, noisy data, as well as issues related to algorithmic modeling such as complexity, uncertainty, and interpretability, need to be mitigated if not resolved. Emerging computational technologies and new paradigms such as federated learning, human-in-the-loop, quantum computing, and novel interpretability methods show great potential in overcoming these challenges and bridging the gap towards precision outcome modeling in radiotherapy. Examples of these promising technologies will be presented and their potential role in improving outcome modeling will be discussed.

Scalable radiotherapy data curation infrastructure for deep-learning based autosegmentation of organs-at-risk: A case study in head and neck cancer

In this era of patient-centered, outcomes-driven and adaptive radiotherapy, deep learning is now being successfully applied to tackle imaging-related workflow bottlenecks such as autosegmentation and dose planning. These applications typically require supervised learning approaches enabled by relatively large, curated radiotherapy datasets which are highly reflective of the contemporary standard of care. However, little has been previously published describing technical infrastructure, recommendations, methods or standards for radiotherapy dataset curation in a holistic fashion. Our radiation oncology department has recently embarked on a large-scale project in partnership with an external partner to develop deep-learning-based tools to assist with our radiotherapy workflow, beginning with autosegmentation of organs-at-risk. This project will require thousands of carefully curated radiotherapy datasets comprising all body sites we routinely treat with radiotherapy. Given such a large project scope, we have approached the need for dataset curation rigorously, with an aim towards building infrastructure that is compatible with efficiency, automation and scalability. Focusing on our first use-case pertaining to head and neck cancer, we describe our developed infrastructure and novel methods applied to radiotherapy dataset curation, inclusive of personnel and workflow organization, dataset selection, expert organ-at-risk segmentation, quality assurance, patient de-identification, data archival and transfer. Over the course of approximately 13 months, our expert multidisciplinary team generated 490 curated head and neck radiotherapy datasets. This task required approximately 6000 human-expert hours in total (not including planning and infrastructure development time). This infrastructure continues to evolve and will support ongoing and future project efforts.

Early Prediction of Planning Adaptation Requirement Indication Due to Volumetric Alterations in Head and Neck Cancer Radiotherapy: A Machine Learning Approach

Background: During RT cycles, the tumor response pattern could affect tumor coverage and may lead to organs at risk of overdose. As such, early prediction of significant volumetric changes could therefore reduce potential radiation-related adverse effects. Nevertheless, effective machine learning approaches based on the radiomic features of the clinically used CBCT images to determine the tumor volume variations due to RT not having been implemented so far. Methods: CBCT images from 40 HN cancer patients were collected weekly during RT treatment. From the obtained images, the Clinical Target Volume (CTV) and Parotid Glands (PG) regions of interest were utilized to calculate 104 delta-radiomics features. These features were fed on a feature selection and classification procedure for the early prediction of significant volumetric alterations. Results: The proposed framework was able to achieve 0.90 classification performance accuracy while detecting a small subset of discriminative characteristics from the 1st week of RT. The selected features were further analyzed regarding their effects on temporal changes in anatomy and tumor response modeling. Conclusion: The use of machine learning algorithms offers promising perspectives for fast and reliable early prediction of large volumetric deviations as a result of RT treatment, exploiting hidden patterns in the overall anatomical characteristics.

Webinar and survey on quality management principles within the Australian and New Zealand ACPSEM Workforce

Healthcare relies upon the accurate and safe delivery of patient care. This is only achievable when systems are developed to ensure high quality, robust outcomes, for instance quality management systems. The concept of quality management can take on a different meaning depending on the context in which it is found. To add complication, the amount of education required for quality management will vary depending on one's exposure to the implementation of quality systems. In part to address these issues, the Australasian College of Physical Scientists and Engineers in Medicine (ACPSEM) Queensland Branch held a quality management webinar for members and non-members across Australia and New Zealand. The purpose of the webinar was to educate and facilitate discussion regarding the application of quality management principles for the ACPSEM profession. In conjunction, a pre- and post-webinar survey was conducted to gain an insight into existing knowledge and attitudes within the professions governed by the ACPSEM and students undertaking related studies. This paper authored by the webinar speakers reintroduces the quality management principles that were discussed in webinar, exemplifies the importance of quality management skills within the ACPSEM professions and presents the results of the surveys, promoting the need for more educational resources on quality management tools.

Image guidance for FLASH radiotherapy

FLASH radiotherapy (FLASH-RT) is an emerging ultra-high dose (>40 Gy/s) delivery that promises to improve the therapeutic potential by limiting toxicities compared to conventional RT while maintaining similar tumor eradication efficacy. Image guidance is an essential component of modern RT that should be harnessed to meet the special emerging needs of FLASH-RT and its associated high risks in planning and delivering of such ultra-high doses in short period of times. Hence, this contribution will elaborate on the imaging requirements and possible solutions in the entire chain of FLASH-RT treatment, from the planning, through the setup and delivery with online in vivo imaging and dosimetry, up to the assessment of biological mechanisms and treatment response. In patient setup and delivery, higher temporal sampling than in conventional RT should ensure that the short treatment is delivered precisely to the targeted region. Additionally, conventional imaging tools such as cone-beam computed tomography will continue to play an important role in improving patient setup prior to delivery, while techniques based on magnetic resonance imaging or positron emission tomography may be extremely valuable for either linear accelerator (Linac) or particle FLASH therapy, to monitor and track anatomical changes during delivery. In either planning or assessing outcomes, quantitative functional imaging could supplement conventional imaging for more accurate utilization of the biological window of the FLASH effect, selecting for or verifying things such as tissue oxygen and existing or transient hypoxia on the relevant timescales of FLASH-RT delivery. Perhaps most importantly at this time, these tools might help improve the understanding of the biological mechanisms of FLASH-RT response in tumor and normal tissues. The high dose deposition of FLASH provides an opportunity to utilize pulse-to-pulse imaging tools such as Cherenkov or radiation acoustic emission imaging. These could provide individual pulse mapping or assessing the 3D dose delivery superficially or at tissue depth, respectively. In summary, the most promising components of modern RT should be used for safer application of FLASH-RT, and new promising developments could be advanced to cope with its novel demands but also exploit new opportunities in connection with the unique nature of pulsed delivery at unprecedented dose rates, opening a new era of biological image guidance and ultrafast, pulse-based in vivo dosimetry.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Cluster model incorporating heterogeneous dose distribution of partial parotid irradiation for radiotherapy induced xerostomia prediction with machine learning methods

PURPOSE

A cluster model incorporating heterogeneous dose distribution within the parotid gland was developed and validated retrospectively for radiotherapy (RT) induced xerostomia prediction with machine learning (ML) techniques.

METHODS

Sixty clusters were obtained at 1 Gy step size with threshold doses ranging from 1 to 60 Gy, for each of the enrolled 155 patients with HNC from three institutions. Feature clusters were selected with the neighborhood component analysis (NCA) and subsequently fed into four supervised ML models for xerostomia prediction comparison: support vector machines (SVM), k-nearest neighbor (kNN), naïve Bayes (NB), and random forest (RF). The predictive performance of each model was evaluated using cross validation resampling with the area-under-the-curves (AUC) of the receiver-operating-characteristic (ROC). The xerostomia predicting capacity using testing data was assessed with accuracy, sensitivity, and specificity for these models and three cluster connectivity choices. Mean dose based logistic regression served as the benchmark for evaluation.

RESULTS

Feature clusters identified by NCA fell in three threshold dose ranges: 5-15Gy, 25-35Gy, and 45-50Gy. Mean dose predictive power was 15% lower than that of the cluster model using the logistic regression classifier. Model validation demonstrated that kNN model outperformed slightly other three models but no substantial difference was observed. Applying the fine-tuned models to testing data yielded that the mean accuracy from SVM, kNN and NB models were between 0.68 and 0.7 while that of RF was ∼0.6. SVM model yielded the best sensitivity (0.76) and kNN model delivered consistent sensitivity and specificity. This is consistent with cross validation. Clusters calculated with three connectivity choices exhibited minimally different predictions.

CONCLUSION

Compared to mean dose, the proposed cluster model has shown its improvement as the xerostomia predictor. When combining with ML techniques, it could provide a clinically useful tool for xerostomia prediction and facilitate decision making during radiotherapy planning for patients with HNC.

The unmet potential of artificial intelligence in veterinary medicine

Veterinary medicine is a broad and growing discipline that includes topics such as companion animal health, population medicine and zoonotic diseases, and agriculture. In this article, we provide insight on how artificial intelligence works and how it is currently applied in veterinary medicine. We also discuss its potential in veterinary medicine. Given the rapid pace of research and commercial product developments in this area, the next several years will pose challenges to understanding, interpreting, and adopting this powerful and evolving technology. Artificial intelligence has the potential to enable veterinarians to perform tasks more efficiently while providing new insights for the management and treatment of disorders. It is our hope that this will translate to better quality of life for animals and those who care for them.

Management of Motion and Anatomical Variations in Charged Particle Therapy: Past, Present, and Into the Future

The major aim of radiation therapy is to provide curative or palliative treatment to cancerous malignancies while minimizing damage to healthy tissues. Charged particle radiotherapy utilizing carbon ions or protons is uniquely suited for this task due to its ability to achieve highly conformal dose distributions around the tumor volume. For these treatment modalities, uncertainties in the localization of patient anatomy due to inter- and intra-fractional motion present a heightened risk of undesired dose delivery. A diverse range of mitigation strategies have been developed and clinically implemented in various disease sites to monitor and correct for patient motion, but much work remains. This review provides an overview of current clinical practices for inter and intra-fractional motion management in charged particle therapy, including motion control, current imaging and motion tracking modalities, as well as treatment planning and delivery techniques. We also cover progress to date on emerging technologies including particle-based radiography imaging, novel treatment delivery methods such as tumor tracking and FLASH, and artificial intelligence and discuss their potential impact towards improving or increasing the challenge of motion mitigation in charged particle therapy.

Development of a deep learning-based patient-specific target contour prediction model for markerless tumor positioning

PURPOSE

For pancreatic cancer patients, image guided radiation therapy and real-time tumor tracking (RTTT) techniques can deliver radiation to the target accurately. Currently, for the radiation therapy machine with kV X-ray imaging systems, internal markers must be implemented to facilitate tumor tracking. The purpose of this study was to develop a markerless deep learning-based pancreatic tumor positioning procedure for real-time tumor tracking with a kV X-ray imaging system.

METHODS AND MATERIALS

Fourteen pancreatic cancer patients treated with intensity-modulated radiation therapy from six fixed gantry angles with a gimbal-head radiotherapy system were included in this study. For a gimbal-head radiotherapy system, the three-dimensional (3D) intrafraction target position can be determined using an orthogonal kV X-ray imaging system. All patients underwent four-dimensional computed tomography (4DCT) simulations for treatment planning, which were divided into 10 respiratory phases. After a patient's 4DCT was acquired, for each X-ray tube angle, 10 digitally reconstructed radiograph (DRR) images were obtained. Then, a data augmentation procedure was conducted. The data augmentation procedure first rotated the CT volume around the superior-inferior and anterior-posterior directions from -3^° to 3^° in 1.5^° intervals. Then, the Super-SloMo model was adapted to interpolate 10 frames between respiratory phases. In total, the data augmentation procedure expanded the data scale 250-fold. In this study, for each patient, 12 datasets containing the DRR images from each specific X-ray tube angle based on the radiation therapy plan were obtained. The augmented dataset was randomly divided into training and testing datasets. The training dataset contained 2000 DRR images with clinical target volume (CTV) contours labeled for fine-tuning the pre-trained target contour prediction model. After the fine-tuning, the patient and X-ray tube angle-specific CTV contour prediction model was acquired. The testing dataset contained the remaining 500 images to evaluate the performance of the CTV contour prediction model. The dice similarity coefficient (DSC) between the area enclosed by the CTV contour and predicted contour was calculated to evaluate the model's contour prediction performance. The 3D position of the CTV was calculated based on the centroid of the contour in the orthogonal DRR images, and the 3D error of the prediction position was calculated to evaluate the CTV positioning performance. For each patient, the DSC results from 12 X-ray tube angles and 3D error from 6 gantry angles were calculated, representing the novelty of this study.

RESULTS

The mean and standard deviation (SD) of all patients' DSCs were 0.98 and 0.015, respectively. The mean and SD of the 3D error were 0.29 mm and 0.14 mm, respectively. The global maximum 3D error was 1.66 mm, and the global minimum DSC was 0.81. The mean calculation time for CTV contour prediction was 55 ms per image. This fulfills the requirement of RTTT.

CONCLUSIONS

Regarding the positioning accuracy and calculation efficiency, the presented procedure can provide a solution for markerless real-time tumor tracking for pancreatic cancer patients. This article is protected by copyright. All rights reserved.

Improving robustness of a deep learning-based lung-nodule classification model of CT images with respect to image noise

Objective. Robustness is an important aspect to consider, when developing methods for medical image analysis. This study investigated robustness properties of deep neural networks (DNNs) for a lung nodule classification problem based on CT images and proposed a solution to improve robustness.Approach. We firstly constructed a class of four DNNs with different widths, each predicting an output label (benign or malignant) for an input CT image cube containing a lung nodule. These networks were trained to achieve Area Under the Curve of 0.891-0.914 on a testing dataset. We then added to the input CT image cubes noise signals generated randomly using a realistic CT image noise model based on a noise power spectrum at 100 mAs, and monitored the DNNs output change. We definedSAR₅(%) to quantify the robustness of the trained DNN model, indicating that for 5% of CT image cubes, the noise can change the prediction results with a chance of at leastSAR₅(%). To understand robustness, we viewed the information processing pipeline by the DNN as a two-step process, with the first step using all but the last layers to extract representations of the input CT image cubes in a latent space, and the second step employing the last fully-connected layer as a linear classifier to determine the position of the sample representations relative to a decision plane. To improve robustness, we proposed to retrain the last layer of the DNN with a Supporting Vector Machine (SVM) hinge loss function to enforce the desired position of the decision plane.Main results.SAR₅ranged in 47.0%-62.0% in different DNNs. The unrobustness behavior may be ascribed to the unfavorable placement of the decision plane in the latent representation space, which made some samples be perturbed to across the decision plane and hence susceptible to noise. The DNN-SVM model improved robustness over the DNN model and reducedSAR₅by 8.8%-21.0%.Significance. This study provided insights about the potential reason for the unrobustness behavior of DNNs and the proposed DNN-SVM model improved model robustness.

Enhancing the impact of Artificial Intelligence in Medicine: A joint AIFM-INFN Italian initiative for a dedicated cloud-based computing infrastructure

Artificial Intelligence (AI) techniques have been implemented in the field of Medical Imaging for more than forty years. Medical Physicists, Clinicians and Computer Scientists have been collaborating since the beginning to realize software solutions to enhance the informative content of medical images, including AI-based support systems for image interpretation. Despite the recent massive progress in this field due to the current emphasis on Radiomics, Machine Learning and Deep Learning, there are still some barriers to overcome before these tools are fully integrated into the clinical workflows to finally enable a precision medicine approach to patients' care. Nowadays, as Medical Imaging has entered the Big Data era, innovative solutions to efficiently deal with huge amounts of data and to exploit large and distributed computing resources are urgently needed. In the framework of a collaboration agreement between the Italian Association of Medical Physicists (AIFM) and the National Institute for Nuclear Physics (INFN), we propose a model of an intensive computing infrastructure, especially suited for training AI models, equipped with secure storage systems, compliant with data protection regulation, which will accelerate the development and extensive validation of AI-based solutions in the Medical Imaging field of research. This solution can be developed and made operational by Physicists and Computer Scientists working on complementary fields of research in Physics, such as High Energy Physics and Medical Physics, who have all the necessary skills to tailor the AI-technology to the needs of the Medical Imaging community and to shorten the pathway towards the clinical applicability of AI-based decision support systems.

Intentional deep overfit learning (IDOL): A novel deep learning strategy for adaptive radiation therapy

PURPOSE

Applications of deep learning (DL) are essential to realizing an effective adaptive radiotherapy (ART) workflow. Despite the promise demonstrated by DL approaches in several critical ART tasks, there remain unsolved challenges to achieve satisfactory generalizability of a trained model in a clinical setting. Foremost among these is the difficulty of collecting a task-specific training dataset with high-quality, consistent annotations for supervised learning applications. In this study, we propose a tailored DL framework for patient-specific performance that leverages the behavior of a model intentionally overfitted to a patient-specific training dataset augmented from the prior information available in an ART workflow-an approach we term Intentional Deep Overfit Learning (IDOL).

METHODS

Implementing the IDOL framework in any task in radiotherapy consists of two training stages: (1) training a generalized model with a diverse training dataset of N patients, just as in the conventional DL approach, and (2) intentionally overfitting this general model to a small training dataset-specific the patient of interest ( N + 1 ) generated through perturbations and augmentations of the available task- and patient-specific prior information to establish a personalized IDOL model. The IDOL framework itself is task-agnostic and is, thus, widely applicable to many components of the ART workflow, three of which we use as a proof of concept here: the autocontouring task on replanning CTs for traditional ART, the MRI super-resolution (SR) task for MRI-guided ART, and the synthetic CT (sCT) reconstruction task for MRI-only ART.

RESULTS

In the replanning CT autocontouring task, the accuracy measured by the Dice similarity coefficient improves from 0.847 with the general model to 0.935 by adopting the IDOL model. In the case of MRI SR, the mean absolute error (MAE) is improved by 40% using the IDOL framework over the conventional model. Finally, in the sCT reconstruction task, the MAE is reduced from 68 to 22 HU by utilizing the IDOL framework.

CONCLUSIONS

In this study, we propose a novel IDOL framework for ART and demonstrate its feasibility using three ART tasks. We expect the IDOL framework to be especially useful in creating personally tailored models in situations with limited availability of training data but existing prior information, which is usually true in the medical setting in general and is especially true in ART.

Dynamic stochastic deep learning approaches for predicting geometric changes in head and neck cancer

Objective.Modern radiotherapy stands to benefit from the ability to efficiently adapt plans during treatment in response to setup and geometric variations such as those caused by internal organ deformation or tumor shrinkage. A promising strategy is to develop a framework, which given an initial state defined by patient-attributes, can predict future states based on pre-learned patterns from a well-defined patient population.Approach.Here, we investigate the feasibility of predicting patient anatomical changes, defined as a joint state of volume and daily setup changes, across a fractionated treatment schedule using two approaches. The first is based on a new joint framework employing quantum mechanics in combination with deep recurrent neural networks, denoted QRNN. The second approach is developed based on a classical framework, which models patient changes as a Markov process, denoted MRNN. We evaluated the performance characteristics of these two approaches on a dataset of 125 head and neck cancer patients, which was supplemented by synthetic data generated using a generative adversarial network. Model performance was evaluated using area under the receiver operating characteristic curve (AUC) scores.Main results.The MRNN framework had slightly better performance than the QRNN framework, with MRNN (QRNN) validation AUC scores of 0.742±0.021 (0.675±0.036), 0.709±0.026 (0.656±0.021), 0.724±0.036 (0.652±0.044), and 0.698±0.016 (0.605±0.035) for system state vector sizes of 4, 6, 8, and 10, respectively. Of these, only the results from the two higher order states had statistically significant differences(p<0.05).A similar trend was also observed when the models were applied to an external testing dataset of 20 patients, yielding MRNN (QRNN) AUC scores of 0.707 (0.623), 0.687 (0.608), 0.723 (0.669), and 0.697 (0.609) for states vectors sizes of 4, 6, 8, and 10, respectively.Significance.These results suggest that both stochastic models have potential value in predicting patient changes during the course of adaptive radiotherapy.

Artificial Intelligence in Thyroid Field-A Comprehensive Review

Simple Summary

The incidence of thyroid pathologies has been increasing worldwide. Historically, the detection of thyroid neoplasms relies on medical imaging analysis, depending mainly on the experience of clinicians. The advent of artificial intelligence (AI) techniques led to a remarkable progress in image-recognition tasks. AI represents a powerful tool that may facilitate understanding of thyroid pathologies, but actually, the diagnostic accuracy is uncertain. This article aims to provide an overview of the basic aspects, limitations and open issues of the AI methods applied to thyroid images. Medical experts should be familiar with the workflow of AI techniques in order to avoid misleading outcomes.

Abstract

Artificial intelligence (AI) uses mathematical algorithms to perform tasks that require human cognitive abilities. AI-based methodologies, e.g., machine learning and deep learning, as well as the recently developed research field of radiomics have noticeable potential to transform medical diagnostics. AI-based techniques applied to medical imaging allow to detect biological abnormalities, to diagnostic neoplasms or to predict the response to treatment. Nonetheless, the diagnostic accuracy of these methods is still a matter of debate. In this article, we first illustrate the key concepts and workflow characteristics of machine learning, deep learning and radiomics. We outline considerations regarding data input requirements, differences among these methodologies and their limitations. Subsequently, a concise overview is presented regarding the application of AI methods to the evaluation of thyroid images. We developed a critical discussion concerning limits and open challenges that should be addressed before the translation of AI techniques to the broad clinical use. Clarification of the pitfalls of AI-based techniques results crucial in order to ensure the optimal application for each patient.

Artificial intelligence and the medical physics profession - A Swedish perspective

BACKGROUND

There is a continuous and dynamic discussion on artificial intelligence (AI) in present-day society. AI is expected to impact on healthcare processes and could contribute to a more sustainable use of resources allocated to healthcare in the future. The aim for this work was to establish a foundation for a Swedish perspective on the potential effect of AI on the medical physics profession.

MATERIALS AND METHODS

We designed a survey to gauge viewpoints regarding AI in the Swedish medical physics community. Based on the survey results and present-day situation in Sweden, a SWOT analysis was performed on the implications of AI for the medical physics profession.

RESULTS

Out of 411 survey recipients, 163 responded (40%). The Swedish medical physicists with a professional license believed (90%) that AI would change the practice of medical physics but did not foresee (81%) that AI would pose a risk to their practice and career. The respondents were largely positive to the inclusion of AI in educational programmes. According to self-assessment, the respondents' knowledge of and workplace preparedness for AI was generally low.

CONCLUSIONS

From the survey and SWOT analysis we conclude that AI will change the medical physics profession and that there are opportunities for the profession associated with the adoption of AI in healthcare. To overcome the weakness of limited AI knowledge, potentially threatening the role of medical physicists, and build upon the strong position in Swedish healthcare, medical physics education and training should include learning objectives on AI.

The perceptions of medical physicists towards relevance and impact of artificial intelligence

Artificial intelligence (AI) is an innovative tool with the potential to impact medical physicists' clinical practices, research, and the profession. The relevance of AI and its impact on the clinical practice and routine of professionals in medical physics were evaluated by medical physicists and researchers in this field. An online survey questionnaire was designed for distribution to professionals and students in medical physics around the world. In addition to demographics questions, we surveyed opinions on the role of AI in medical physicists' practices, the possibility of AI threatening/disrupting the medical physicists' practices and career, the need for medical physicists to acquire knowledge on AI, and the need for teaching AI in postgraduate medical physics programmes. The level of knowledge of medical physicists on AI was also consulted. A total of 1019 respondents from 94 countries participated. More than 85% of the respondents agreed that AI would play an essential role in medical physicists' practices. AI should be taught in the postgraduate medical physics programmes, and that more applications such as quality control (QC), treatment planning would be performed by AI. Half of the respondents thought AI would not threaten/disrupt the medical physicists' practices. AI knowledge was mainly acquired through self-taught and work-related activities. Nonetheless, many (40%) reported that they have no skill in AI. The general perception of medical physicists was that AI is here to stay, influencing our practices. Medical physicists should be prepared with education and training for this new reality.

Integrating Multiomics Information in Deep Learning Architectures for Joint Actuarial Outcome Prediction in Non-Small Cell Lung Cancer Patients After Radiation Therapy

PURPOSE

Novel actuarial deep learning neural network (ADNN) architectures are proposed for joint prediction of radiation therapy outcomes-radiation pneumonitis (RP) and local control (LC)-in stage III non-small cell lung cancer (NSCLC) patients. Unlike normal tissue complication probability/tumor control probability models that use dosimetric information solely, our proposed models consider complex interactions among multiomics information including positron emission tomography (PET) radiomics, cytokines, and miRNAs. Additional time-to-event information is also used in the actuarial prediction.

METHODS AND MATERIALS

Three architectures were investigated: ADNN-DVH considered dosimetric information only; ADNN-com integrated multiomics information; and ADNN-com-joint combined RP2 (RP grade ≥2) and LC prediction. In these architectures, differential dose-volume histograms (DVHs) were fed into 1D convolutional neural networks (CNN) for extracting reduced representations. Variational encoders were used to learn representations of imaging and biological data. Reduced representations were fed into Surv-Nets to predict time-to-event probabilities for RP2 and LC independently and jointly by incorporating time information into designated loss functions.

RESULTS

Models were evaluated on 117 retrospective patients and were independently tested on 25 newly accrued patients prospectively. A multi-institutional RTOG0617 data set of 327 patients was used for external validation. ADNN-DVH yielded cross-validated c-indexes (95% confidence intervals) of 0.660 (0.630-0.690) for RP2 prediction and 0.727 (0.700-0.753) for LC prediction, outperforming a generalized Lyman model for RP2 (0.613 [0.583-0.643]) and a generalized log-logistic model for LC (0.569 [0.545-0.594]). The independent internal test and external validation yielded similar results. ADNN-com achieved an even better performance than ADNN-DVH on both cross-validation and independent internal test. Furthermore, ADNN-com-joint, which yielded performance similar to ADNN-com, realized joint prediction with c-indexes of 0.705 (0.676-0.734) for RP2 and 0.740 (0.714-0.765) for LC and achieved an area under a free-response receiving operator characteristic curve (AU-FROC) of 0.729 (0.697-0.773) for the joint prediction of RP2 and LC.

CONCLUSION

Novel deep learning architectures that integrate multiomics information outperformed traditional normal tissue complication probability/tumor control probability models in actuarial prediction of RP2 and LC.

Radiomic and radiogenomic modeling for radiotherapy: strategies, pitfalls, and challenges

The power of predictive modeling for radiotherapy outcomes has historically been limited by an inability to adequately capture patient-specific variabilities; however, next-generation platforms together with imaging technologies and powerful bioinformatic tools have facilitated strategies and provided optimism. Integrating clinical, biological, imaging, and treatment-specific data for more accurate prediction of tumor control probabilities or risk of radiation-induced side effects are high-dimensional problems whose solutions could have widespread benefits to a diverse patient population-we discuss technical approaches toward this objective. Increasing interest in the above is specifically reflected by the emergence of two nascent fields, which are distinct but complementary: radiogenomics, which broadly seeks to integrate biological risk factors together with treatment and diagnostic information to generate individualized patient risk profiles, and radiomics, which further leverages large-scale imaging correlates and extracted features for the same purpose. We review classical analytical and data-driven approaches for outcomes prediction that serve as antecedents to both radiomic and radiogenomic strategies. Discussion then focuses on uses of conventional and deep machine learning in radiomics. We further consider promising strategies for the harmonization of high-dimensional, heterogeneous multiomics datasets (panomics) and techniques for nonparametric validation of best-fit models. Strategies to overcome common pitfalls that are unique to data-intensive radiomics are also discussed.