Big Data Readiness in Radiation Oncology: An Efficient Approach for Relabeling Radiation Therapy Structures With Their TG-263 Standard Name in Real-World Data Sets

Order From Chaos: The Benefits of Standardized Nomenclature in Radiation Oncology

Although standardization has been shown to improve patient safety and improve the efficiency of workflows, implementation of standards can take considerable effort and requires the engagement of all clinical stakeholders. Engaging team members includes increasing awareness of the proposed benefit of the standard, a clear implementation plan, monitoring for improvements, and open communication to support successful implementation. The benefits of standardization often focus on large institutions to improve research endeavors, yet all clinics can benefit from standardization to increase quality and implement more efficient or automated workflow. The benefits of nomenclature standardization for all team members and institution sizes, including success stories, are discussed with practical implementation guides to facilitate the adoption of standardized nomenclature in radiation oncology.

Benchmarking a Foundation Large Language Model on its Ability to Relabel Structure Names in Accordance With the American Association of Physicists in Medicine Task Group-263 Report

PURPOSE

To introduce the concept of using large language models (LLMs) to relabel structure names in accordance with the American Association of Physicists in Medicine Task Group-263 standard and to establish a benchmark for future studies to reference.

METHODS AND MATERIALS

Generative Pretrained Transformer (GPT)-4 was implemented within a Digital Imaging and Communications in Medicine server. Upon receiving a structure-set Digital Imaging and Communications in Medicine file, the server prompts GPT-4 to relabel the structure names according to the American Association of Physicists in Medicine Task Group-263 report. The results were evaluated for 3 disease sites: prostate, head and neck, and thorax. For each disease site, 150 patients were randomly selected for manually tuning the instructions prompt (in batches of 50), and 50 patients were randomly selected for evaluation. Structure names considered were those that were most likely to be relevant for studies using structure contours for many patients.

RESULTS

The per-patient accuracy was 97.2%, 98.3%, and 97.1% for prostate, head and neck, and thorax disease sites, respectively. On a per-structure basis, the clinical target volume was relabeled correctly in 100%, 95.3%, and 92.9% of cases, respectively.

CONCLUSIONS

Given the accuracy of GPT-4 in relabeling structure names as presented in this work, LLMs are poised to become an important method for standardizing structure names in radiation oncology, especially considering the rapid advancements in LLM capabilities that are likely to continue.

Perceptions on and roadblocks to implementation of standardized nomenclature in radiation oncology: A survey from TG-263U1

PURPOSE

AAPM Task Group No. 263U1 (Update to Report No. 263 - Standardizing Nomenclatures in Radiation Oncology) disseminated a survey to receive feedback on utilization, gaps, and means to facilitate further adoption.

METHODS

The survey was created by TG-263U1 members to solicit feedback from physicists, dosimetrists, and physicians working in radiation oncology. Questions on the adoption of the TG-263 standard were coupled with demographic information, such as clinical role, place of primary employment (e.g., private hospital, academic center), and size of institution. The survey was emailed to all AAPM, AAMD, and ASTRO members.

RESULTS

The survey received 463 responses with 310 completed survey responses used for analysis, of whom most had the clinical role of medical physicist (73%) and the majority were from the United States (83%). There were 83% of respondents who indicated that they believe that having a nomenclature standard is important or very important and 61% had adopted all or portions of TG-263 in their clinics. For those yet to adopt TG-263, the staffing and implementation efforts were the main cause for delaying adoption. Fewer respondents had trouble adopting TG-263 for organs at risk (29%) versus target (44%) nomenclature. Common themes in written feedback were lack of physician support and available resources, especially in vendor systems, to facilitate adoption.

CONCLUSIONS

While there is strong support and belief in the benefit of standardized nomenclature, the widespread adoption of TG-263 has been hindered by the effort needed by staff for implementation.  Feedback from the survey is being utilized to drive the focus of the update efforts and create tools to facilitate easier adoption of TG-263.

Open RT Structures: A Solution for TG-263 Accessibility

PURPOSE

Consistency of nomenclature within radiation oncology is increasingly important as big data efforts and data sharing become more feasible. Automation of radiation oncology workflows depends on standardized contour nomenclature that enables toxicity and outcomes research, while also reducing medical errors and facilitating quality improvement activities. Recommendations for standardized nomenclature have been published in the American Association of Physicists in Medicine (AAPM) report from Task Group 263 (TG-263). Transitioning to TG-263 requires creation and management of structure template libraries and retraining of staff, which can be a considerable burden on clinical resources. Our aim is to develop a program that allows users to create TG-263-compliant structure templates in English, Spanish, or French to facilitate data sharing.

METHODS AND MATERIALS

Fifty-three premade structure templates were arranged by treated organ based on an American Society for Radiation Oncology (ASTRO) consensus paper. Templates were further customized with common target structures, relevant organs at risk (OARs) (eg, spleen for anatomically relevant sites such as the gastroesophageal junction or stomach), subsite- specific templates (eg, partial breast, whole breast, intact prostate, postoperative prostate, etc) and brachytherapy templates. An informal consensus on OAR and target coloration was also achieved, although color selections are fully customizable within the program.

RESULTS

The resulting program is usable on any Windows system and generates template files in practice-specific Digital Imaging and Communications In Medicine (DICOM) or XML formats, extracting standardized structure nomenclature from an online database maintained by members of the TG-263U1, which ensures continuous access to up-to-date templates.

CONCLUSIONS

We have developed a tool to easily create and name DICOM radiation therapy (DICOM-RT) structures sets that are TG-263-compliant for all planning systems using the DICOM standard. The program and source code are publicly available via GitHub to encourage feedback from community users for improvement and guide further development.

Artificial general intelligence for radiation oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale.

Deep learning-based classification of organs at risk and delineation guideline in pelvic cancer radiation therapy

Deep learning (DL) models for radiation therapy (RT) image segmentation require accurately annotated training data. Multiple organ delineation guidelines exist; however, information on the used guideline is not provided with the delineation. Extraction of training data with coherent guidelines can therefore be challenging. We present a supervised classification method for pelvis structure delineations where bowel cavity, femoral heads, bladder, and rectum data, with two guidelines, were classified. The impact on DL-based segmentation quality using mixed guideline training data was also demonstrated. Bowel cavity was manually delineated on CT images for anal cancer patients (n = 170) according to guidelines Devisetty and RTOG. The DL segmentation quality from using training data with coherent or mixed guidelines was investigated. A supervised 3D squeeze-and-excite SENet-154 model was trained to classify two bowel cavity delineation guidelines. In addition, a pelvis CT dataset with manual delineations from prostate cancer patients (n = 1854) was used where data with an alternative guideline for femoral heads, rectum, and bladder were generated using commercial software. The model was evaluated on internal (n = 200) and external test data (n = 99). By using mixed, compared to coherent, delineation guideline training data mean DICE score decreased 3% units, mean Hausdorff distance (95%) increased 5 mm and mean surface distance (MSD) increased 1 mm. The classification of bowel cavity test data achieved 99.8% unweighted classification accuracy, 99.9% macro average precision, 97.2% macro average recall, and 98.5% macro average F1. Corresponding metrics for the pelvis internal test data were all 99% or above and for the external pelvis test data they were 96.3%, 96.6%, 93.3%, and 94.6%. Impaired segmentation performance was observed for training data with mixed guidelines. The DL delineation classification models achieved excellent results on internal and external test data. This can facilitate automated guideline-specific data extraction while avoiding the need for consistent and correct structure labels.

Educative Impact of Automatic Delineation Applied to Head and Neck Cancer Patients on Radiation Oncology Residents

To evaluate the educational impact on radiation oncology residents in training when introducing an automatic segmentation software in head and neck cancer patients regarding organs at risk (OARs) and prophylactic cervical lymph node level (LNL) volumes. Two cases treated by exclusive intensity-modulated radiotherapy were delineated by an expert radiation oncologist and were considered as reference. Then, these cases were delineated by residents divided into two groups: group 1 (control group), experienced residents delineating manually, group 2 (experimental group), young residents on their first rotation trained with automatic delineation, delineating manually first (M -) and then after using the automatic system (M +). The delineation accuracy was assessed using the Overlap Volume (OV). Regarding the OARs, mean OV was 0.62 (SD = 0.05) for group 1, 0.56 (SD = 0.04) for group 2 M - , and 0.61 (SD = 0.03) for group 2 M + . Mean OV was higher in group 1 compared to group 2 M - (p = 0.01). There was no OV difference between group 1 and group 2 M + (p = 0.67). Mean OV was higher in the group 2 M + compared to group 2 M - (p < 0.003). Regarding LNL, mean OV was 0.53 (SD = 0.06) in group 1, 0.54 (SD = 0.03) in group 2 M - , and 0.58 (SD = 0.04) in group 2 M + . Mean OV was higher in group 2 M + for 11 of the 12 analysed structures compared to group 2 M - (p = 0.016). Prior use of the automatic delineation software reduced the average contouring time per case by 34 to 40%. Prior use of atlas-based automatic segmentation reduces the delineation duration, and provides reliable OARs and LNL delineations.

Standardising Breast Radiotherapy Structure Naming Conventions: A Machine Learning Approach

In progressing the use of big data in health systems, standardised nomenclature is required to enable data pooling and analyses. In many radiotherapy planning systems and their data archives, target volumes (TV) and organ-at-risk (OAR) structure nomenclature has not been standardised. Machine learning (ML) has been utilised to standardise volumes nomenclature in retrospective datasets. However, only subsets of the structures have been targeted. Within this paper, we proposed a new approach for standardising all the structures nomenclature by using multi-modal artificial neural networks. A cohort consisting of 1613 breast cancer patients treated with radiotherapy was identified from Liverpool & Macarthur Cancer Therapy Centres, NSW, Australia. Four types of volume characteristics were generated to represent each target and OAR volume: textual features, geometric features, dosimetry features, and imaging data. Five datasets were created from the original cohort, the first four represented different subsets of volumes and the last one represented the whole list of volumes. For each dataset, 15 sets of combinations of features were generated to investigate the effect of using different characteristics on the standardisation performance. The best model reported 99.416% classification accuracy over the hold-out sample when used to standardise all the nomenclatures in a breast cancer radiotherapy plan into 21 classes. Our results showed that ML based automation methods can be used for standardising naming conventions in a radiotherapy plan taking into consideration the inclusion of multiple modalities to better represent each volume.

Head and Neck Radiation Therapy Patterns of Practice Variability Identified as a Challenge to Real-World Big Data: Results From the Learning from Analysis of Multicentre Big Data Aggregation (LAMBDA) Consortium

Purpose

Outside of randomized clinical trials, it is difficult to develop clinically relevant evidence-based recommendations for radiation therapy (RT) practice guidelines owing to lack of comprehensive real-world data. To address this knowledge gap, we formed the Learning from Analysis of Multicenter Big Data Aggregation consortium to cooperatively implement RT data standardization, develop software solutions for data analysis, and recommend clinical practice change based on real-world data analyzed. The first phase of this "Big Data" study aimed at characterizing variability in clinical practice patterns of dosimetric data for organs at risk (OARs) that would undermine subsequent use of large-scale, electronically aggregated data to characterize associations with outcomes. Evidence from this study was used as the basis for practical recommendations to improve data quality.

Methods and Materials

Dosimetric details of patients with head and neck cancer treated with radiation therapy between 2014 and 2019 were analyzed. Institutional patterns of practice were characterized, including structure nomenclature, volumes, and frequency of contouring. Dose volume histogram (DVH) distributions were characterized and compared with institutional constraints and literature values.

Results

Plans for 4664 patients treated to a mean plan dose of 64.4 ± 13.2 Gy in 32 ± 4 fractions were aggregated. Before implementation of TG-263 guidelines in each institution, there was variability in OAR nomenclature across institutions and structures. With evidence from this study, we identified a targeted and practical set of recommendations aimed at improving the quality of real-world data.

Conclusions

Quantifying similarities and differences among institutions for OAR structures and DVH metrics is the launching point for next steps to investigate potential relationships between DVH parameters and patient outcomes.

Digital skills of therapeutic radiographers/radiation therapists - Document analysis for a European educational curriculum

INTRODUCTION

It is estimated that around 50% of cancer patients require Radiotherapy (RT) at some point during their treatment, hence Therapeutic Radiographers/Radiation Therapists (TR/RTTs) have a key role to play in patient management. It is essential for TR/RTTs to keep abreast with new technologies and continuously develop the digital skills necessary for safe RT practice. The RT profession and education is not regulated at European Union level, which leads to heterogeneity in the skills developed and practised among countries. This study aimed to explore the white and grey literature to collate data on the relevant digital skills required for TR/RTTs practice.

METHODS

An exhaustive systematic search was conducted to identify literature discussing digital skills of TR/RTTs; relevant grey literature was also identified. A thematic analysis was performed to identify and organise these skills into themes and sub-themes.

RESULTS

195 digital skills were identified, organised in 35 sub-themes and grouped into six main themes: (i) Transversal Digital Skills, (ii) RT Planning Image, (iii) RT Treatment Planning, (iv) RT Treatment Administration, (v) Quality, Safety and Risk Management, and (vi) Management, Education and Research.

CONCLUSION

This list can be used as a reference to close current gaps in knowledge or skills of TR/RTTs while anticipating future needs regarding the rapid development of new technologies (such as Artificial Intelligence or Big Data).

IMPLICATIONS FOR PRACTICE

It is imperative to align education with current and future RT practice to ensure that all RT patients receive the best care. Filling the gaps in TR/RTTs skill sets will improve current practice and provide TR/RTTs with the support needed to develop more advanced skills.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Automated Plan Checking Software Demonstrates Continuous and Sustained Improvements in Safety and Quality: A 3-year Longitudinal Analysis

PURPOSE

This study aimed to perform a longitudinal analysis of the performance of our automated plan checking software by retrospectively evaluating the number of errors identified in plans delivered to patients in 3, month-long, data collection periods between 2017 and 2020.

METHODS AND MATERIALS

Eleven automated checks were retrospectively run on 1169 external beam radiation therapy treatment plans identified as meeting the following criteria: planning target volume-based multifield photon plans receiving a status of treatment approved in March 2017, March 2018, or March 2020. The number of passes (true positives) and flags were recorded. Flags were subcategorized into false negatives, false negatives due to naming conventions, and true negatives. In addition, 2 × 2 contingency tables using a 2-tailed Fisher's exact test were used to determine whether there were nonrandom associations between the output of the automated plan checking software and whether the check was manual or automated at the original time of treatment approval.

RESULTS

A statistically significant decrease in flags between the pre- and postautomation data sets was observed for 4 contour-based checks, namely adjacent structures overlap, empty structures and missing slices, overlap between body and couch, and laterality, as well as a check that determined whether the plan's global maximum dose was within the planning target volume. A review of the origins of false negatives was fed back into the design of the checks to improve the reliability of the system and help avoid warning fatigue.

CONCLUSIONS

Periodic and longitudinal review of the performance of automated software was essential for monitoring and understanding its impact on error rates, as well as for optimization of the tool to adapt to regular changes of clinical practice. The automated plan checking software has demonstrated continuous contributions to the safe and effective delivery of external beam radiation therapy to our patient population, an impact that extends beyond its initial implementation and deployment.

Deep learning-based classification and structure name standardization for organ at risk and target delineations in prostate cancer radiotherapy

Radiotherapy (RT) datasets can suffer from variations in annotation of organ at risk (OAR) and target structures. Annotation standards exist, but their description for prostate targets is limited. This restricts the use of such data for supervised machine learning purposes as it requires properly annotated data. The aim of this work was to develop a modality independent deep learning (DL) model for automatic classification and annotation of prostate RT DICOM structures. Delineated prostate organs at risk (OAR), support- and target structures (gross tumor volume [GTV]/clinical target volume [CTV]/planning target volume [PTV]), along with or without separate vesicles and/or lymph nodes, were extracted as binary masks from 1854 patients. An image modality independent 2D InceptionResNetV2 classification network was trained with varying amounts of training data using four image input channels. Channel 1-3 consisted of orthogonal 2D projections from each individual binary structure. The fourth channel contained a summation of the other available binary structure masks. Structure classification performance was assessed in independent CT (n = 200 pat) and magnetic resonance imaging (MRI) (n = 40 pat) test datasets and an external CT (n = 99 pat) dataset from another clinic. A weighted classification accuracy of 99.4% was achieved during training. The unweighted classification accuracy and the weighted average F1 score among different structures in the CT test dataset were 98.8% and 98.4% and 98.6% and 98.5% for the MRI test dataset, respectively. The external CT dataset yielded the corresponding results 98.4% and 98.7% when analyzed for trained structures only, and results from the full dataset yielded 79.6% and 75.2%. Most misclassifications in the external CT dataset occurred due to multiple CTVs and PTVs being fused together, which was not included in the training data. Our proposed DL-based method for automated renaming and standardization of prostate radiotherapy annotations shows great potential. Clinic specific contouring standards however need to be represented in the training data for successful use. Source code is available at https://github.com/jamtheim/DicomRTStructRenamerPublic.

Multi-View Data Integration Methods for Radiotherapy Structure Name Standardization

Simple Summary

Structure names associated with radiotherapy treatments need standardization to develop data pipelines enabling personalized treatment plans. Automatic classification of structure names based on the currently available TG-263 nomenclature can help with data aggregation from both retrospective and future data sources. The aim of our proposed machine learning-based data integration methods is to achieve highly accurate structure name classification to automate the data aggregation process. Our multi-view models can overcome the challenges of integrating different data types associated with radiotherapy structures, such as the physician-given text labels and geometric or image data. The models exhibited high accuracy when tested on multi-center and multi-institutional lung and prostate cancer patients data and outperformed the models built on any single data type. This highlights the importance of combining different types of data in building generalizable models for structure name standardization.

Abstract

Standardization of radiotherapy structure names is essential for developing data-driven personalized radiotherapy treatment plans. Different types of data are associated with radiotherapy structures, such as the physician-given text labels, geometric (image) data, and Dose-Volume Histograms (DVH). Prior work on structure name standardization used just one type of data. We present novel approaches to integrate complementary types (views) of structure data to build better-performing machine learning models. We present two methods, namely (a) intermediate integration and (b) late integration, to combine physician-given textual structure name features and geometric information of structures. The dataset consisted of 709 prostate cancer and 752 lung cancer patients across 40 radiotherapy centers administered by the U.S. Veterans Health Administration (VA) and the Department of Radiation Oncology, Virginia Commonwealth University (VCU). We used randomly selected data from 30 centers for training and ten centers for testing. We also used the VCU data for testing. We observed that the intermediate integration approach outperformed the models with a single view of the dataset, while late integration showed comparable performance with single-view results. Thus, we demonstrate that combining different views (types of data) helps build better models for structure name standardization to enable big data analytics in radiation oncology.

Mini review: Personalization of the radiation therapy management of prostate cancer using MRI-based radiomics

Decisions on how to treat prostate cancer with radiation therapy are guideline-based but as such guidelines have been developed for populations of patients, this invariably leads to overly aggressive treatment in some patients and insufficient treatment in others. Heterogeneity within prostate tumors and in metastatic sites, even within the same patient, is believed to be a major cause of treatment failure. Radiomics biomarkers, more commonly referred to as radiomics 'features", provide readily available, cost-effective, non-invasive tools for screening, detecting tumors and serial monitoring of patients, including assessments of response to therapy and identification of therapeutic complications. Radiomics offers the potential to analyse whole tumors in 3D, as well as sub-regions or 'habitats' within tumors. Combining quantitative information from imaging with pathology, demographic details and other biomarkers will pave the way for personalised treatment selection and monitoring in prostate cancer. The aim of this review is to consider if MRI-based radiomics can bridge the gap between population-based management and personalised management of prostate cancer.

A Machine Learning method for relabeling arbitrary DICOM structure sets to TG-263 defined labels

PURPOSE

To present a Machine Learning pipeline for automatically relabeling anatomical structure sets in the Digital Imaging and Communications in Medicine (DICOM) format to a standard nomenclature that will enable data abstraction for research and quality improvement.

METHODS

DICOM structure sets from approximately 1200 lung and prostate cancer patients across 40 treatment centers were used to build predictive models to automate the relabeling of clinically specified structure labels to standardized labels as defined by the American Association of Physics in Medicine's (AAPM) Task Group 263 (TG-263). Volumetric bitmaps were created based on the delineated volumes and were combined with associated bony anatomy data to build feature vectors. Feature reduction was performed with singular value decomposition and the resulting vectors were used for predicting the label of each structure using five different classifier algorithms on the Apache Spark platform with 5-fold cross-validation. Undersampling methods were used to deal with underlying class imbalance that hindered the performance of classifiers. Experiments were performed on both a curated version of the data, which included only annotated structures, and the non-curated data that included all structures from the original treatment plans.

RESULTS

Random Forest provided the highest accuracies with F₁ scores of 98.77 for lung and 95.06 for prostate on the curated data sets. Scores were lower with 95.67 for lung and 90.22 for prostate on the non-curated data sets, highlighting some of the challenges of classifying real clinical data. Including bony anatomy data and pooling information from all structures for the same patient both increased accuracies. In some cases, undersampling with k-Means clustering for class balancing improved classifier accuracy but in all experiments it significantly reduced run time compared to random undersampling.

CONCLUSION

This work shows that structure sets can be relabeled using our approach with accuracies over 95% for many structure types when presented with curated data. Although accuracies dropped when using the full non-curated data sets, some structure types were still correctly labeled over 90% of the time. With similar results obtained on an external test data set, we can infer that the proposed models are likely to work on other clinical data sets.

Introduction to machine and deep learning for medical physicists

Recent years have witnessed tremendous growth in the application of machine learning (ML) and deep learning (DL) techniques in medical physics. Embracing the current big data era, medical physicists equipped with these state-of-the-art tools should be able to solve pressing problems in modern radiation oncology. Here, a review of the basic aspects involved in ML/DL model building, including data processing, model training, and validation for medical physics applications is presented and discussed. Machine learning can be categorized based on the underlying task into supervised learning, unsupervised learning, or reinforcement learning; each of these categories has its own input/output dataset characteristics and aims to solve different classes of problems in medical physics ranging from automation of processes to predictive analytics. It is recognized that data size requirements may vary depending on the specific medical physics application and the nature of the algorithms applied. Data processing, which is a crucial step for model stability and precision, should be performed before training the model. Deep learning as a subset of ML is able to learn multilevel representations from raw input data, eliminating the necessity for hand crafted features in classical ML. It can be thought of as an extension of the classical linear models but with multilayer (deep) structures and nonlinear activation functions. The logic of going "deeper" is related to learning complex data structures and its realization has been aided by recent advancements in parallel computing architectures and the development of more robust optimization methods for efficient training of these algorithms. Model validation is an essential part of ML/DL model building. Without it, the model being developed cannot be easily trusted to generalize to unseen data. Whenever applying ML/DL, one should keep in mind, according to Amara's law, that humans may tend to overestimate the ability of a technology in the short term and underestimate its capability in the long term. To establish ML/DL role into standard clinical workflow, models considering balance between accuracy and interpretability should be developed. Machine learning/DL algorithms have potential in numerous radiation oncology applications, including automatizing mundane procedures, improving efficiency and safety of auto-contouring, treatment planning, quality assurance, motion management, and outcome predictions. Medical physicists have been at the frontiers of technology translation into medicine and they ought to be prepared to embrace the inevitable role of ML/DL in the practice of radiation oncology and lead its clinical implementation.

Integrated Natural Language Processing and Machine Learning Models for Standardizing Radiotherapy Structure Names

The lack of standardized structure names in radiotherapy (RT) data limits interoperability, data sharing, and the ability to perform big data analysis. To standardize radiotherapy structure names, we developed an integrated natural language processing (NLP) and machine learning (ML) based system that can map the physician-given structure names to American Association of Physicists in Medicine (AAPM) Task Group 263 (TG-263) standard names. The dataset consist of 794 prostate and 754 lung cancer patients across the 40 different radiation therapy centers managed by the Veterans Health Administration (VA). Additionally, data from the Radiation Oncology department at Virginia Commonwealth University (VCU) was collected to serve as a test set. Domain experts identified as anatomically significant nine prostate and ten lung organs-at-risk (OAR) structures and manually labeled them according to the TG-263 standards, and remaining structures were labeled as Non_OAR. We experimented with six different classification algorithms and three feature vector methods, and the final model was built with fastText algorithm. Multiple validation techniques are used to assess the robustness of the proposed methodology. The macro-averaged F 1 score was used as the main evaluation metric. The model achieved an F 1 score of 0.97 on prostate structures and 0.99 for lung structures from the VA dataset. The model also performed well on the test (VCU) dataset, achieving an F 1 score of 0.93 for prostate structures and 0.95 on lung structures. In this work, we demonstrate that NLP and ML based approaches can used to standardize the physician-given RT structure names with high fidelity. This standardization can help with big data analytics in the radiation therapy domain using population-derived datasets, including standardization of the treatment planning process, clinical decision support systems, treatment quality improvement programs, and hypothesis-driven clinical research.

Technical Note: An open source solution for improving TG-263 compliance

PURPOSE

Compliance with TG-263 nomenclature standards can be challenging. We introduce an open source solution to this problem and evaluate its impact on compliance within our institution.

MATERIALS/METHODS

The TG-236 nomenclature standards were implemented in our clinic in two phases. In phase 1, we deployed TG-263 compliant templates for each disease site. In phase 2, we developed and deployed a script for evaluating compliance which presented errors to the user. After each phase the compliance was recorded.

RESULTS

Mean compliance errors prior to phase 1 was 31.8% ± 17.4%. Error rates dropped to 8.1% ± 12.2% across phase 1 and dropped further to 2.2% ± 6.9% during the automation system deployed in phase 2.

CONCLUSION

Both structure templates and automation scripts are very useful for increasing compliance with structure naming standards. Our software solution is made available on GitHub for other institutions to implement.