Deep learning-based synthetic CT generation for paediatric brain MR-only photon and proton radiotherapy

Magnetic Resonance Imaging Versus Computed Tomography for Three-Dimensional Bone Imaging of Musculoskeletal Pathologies: A Review

Magnetic resonance imaging (MRI) is increasingly utilized as a radiation‐free alternative to computed tomography (CT) for the diagnosis and treatment planning of musculoskeletal pathologies. MR imaging of hard tissues such as cortical bone remains challenging due to their low proton density and short transverse relaxation times, rendering bone tissues as nonspecific low signal structures on MR images obtained from most sequences. Developments in MR image acquisition and post‐processing have opened the path for enhanced MR‐based bone visualization aiming to provide a CT‐like contrast and, as such, ease clinical interpretation. The purpose of this review is to provide an overview of studies comparing MR and CT imaging for diagnostic and treatment planning purposes in orthopedic care, with a special focus on selective bone visualization, bone segmentation, and three‐dimensional (3D) modeling. This review discusses conventional gradient‐echo derived techniques as well as dedicated short echo time acquisition techniques and post‐processing techniques, including the generation of synthetic CT, in the context of 3D and specific bone visualization. Based on the reviewed literature, it may be concluded that the recent developments in MRI‐based bone visualization are promising. MRI alone provides valuable information on both bone and soft tissues for a broad range of applications including diagnostics, 3D modeling, and treatment planning in multiple anatomical regions, including the skull, spine, shoulder, pelvis, and long bones.

Prospective Clinical Feasibility Study for MRI-Only Brain Radiotherapy

OBJECTIVES

MRI-only radiotherapy (RT) provides a workflow to decrease the geometric uncertainty introduced by the image registration process between MRI and CT data and to streamline the RT planning. Despite the recent availability of validated synthetic CT (sCT) methods for the head region, there are no clinical implementations reported for brain tumors. Based on a preceding validation study of sCT, this study aims to investigate MRI-only brain RT through a prospective clinical feasibility study with endpoints for dosimetry and patient setup.

MATERIAL AND METHODS

Twenty-one glioma patients were included. MRI Dixon images were used to generate sCT images using a CE-marked deep learning-based software. RT treatment plans were generated based on MRI delineated anatomical structures and sCT for absorbed dose calculations. CT scans were acquired but strictly used for sCT quality assurance (QA). Prospective QA was performed prior to MRI-only treatment approval, comparing sCT and CT image characteristics and calculated dose distributions. Additional retrospective analysis of patient positioning and dose distribution gamma evaluation was performed.

RESULTS

Twenty out of 21 patients were treated using the MRI-only workflow. A single patient was excluded due to an MRI artifact caused by a hemostatic substance injected near the target during surgery preceding radiotherapy. All other patients fulfilled the acceptance criteria. Dose deviations in target were within ±1% for all patients in the prospective analysis. Retrospective analysis yielded gamma pass rates (2%, 2 mm) above 99%. Patient positioning using CBCT images was within ± 1 mm for registrations with sCT compared to CT.

CONCLUSION

We report a successful clinical study of MRI-only brain radiotherapy, conducted using both prospective and retrospective analysis. Synthetic CT images generated using the CE-marked deep learning-based software were clinically robust based on endpoints for dosimetry and patient positioning.

Feasibility of Synthetic Computed Tomography Images Generated from Magnetic Resonance Imaging Scans Using Various Deep Learning Methods in the Planning of Radiation Therapy for Prostate Cancer

Simple Summary

MRI-only simulation in radiation therapy (RT) planning has received attention because the CT scan can be omitted. For MRI-only simulation, synthetic CT (sCT) is necessary for the dose calculation. Various methodologies have been suggested for the generation of sCT and, recently, methods using the deep learning approaches are actively investigated. GAN and cycle-consistent GAN (CycGAN) have been mainly tested, however, very limited studies compared the qualities of sCTs generated from these methods or suggested other models for sCT generation. We have compared GAN, CycGAN, and, reference-guided GAN (RgGAN), a new model of deep learning method. We found that the performance in the HU conservation for soft tissue was poorest for GAN. All methods could generate sCTs feasible for VMAT planning with the trend that sCT generated from the RgGAN showed best performance in dosimetric conservation D_98% and D_95% than sCTs from other methodologies.

Abstract

We aimed to evaluate and compare the qualities of synthetic computed tomography (sCT) generated by various deep-learning methods in volumetric modulated arc therapy (VMAT) planning for prostate cancer. Simulation computed tomography (CT) and T2-weighted simulation magnetic resonance image from 113 patients were used in the sCT generation by three deep-learning approaches: generative adversarial network (GAN), cycle-consistent GAN (CycGAN), and reference-guided CycGAN (RgGAN), a new model which performed further adjustment of sCTs generated by CycGAN with available paired images. VMAT plans on the original simulation CT images were recalculated on the sCTs and the dosimetric differences were evaluated. For soft tissue, a significant difference in the mean Hounsfield unites (HUs) was observed between the original CT images and only sCTs from GAN (p = 0.03). The mean relative dose differences for planning target volumes or organs at risk were within 2% among the sCTs from the three deep-learning approaches. The differences in dosimetric parameters for D_98% and D_95% from original CT were lowest in sCT from RgGAN. In conclusion, HU conservation for soft tissue was poorest for GAN. There was the trend that sCT generated from the RgGAN showed best performance in dosimetric conservation D_98% and D_95% than sCTs from other methodologies.

Artificial Intelligence Applications in Pediatric Brain Tumor Imaging: A Systematic Review

OBJECTIVE

Artificial intelligence (AI) has facilitated the analysis of medical imaging given increased computational capacity and medical data availability in recent years. Although many applications for AI in the imaging of brain tumors have been proposed, their potential clinical impact remains to be explored. A systematic review was performed to examine the role of AI in the analysis of pediatric brain tumor imaging.

METHODS

PubMed, Embase, and Scopus were searched for relevant articles up to January 27, 2021.

RESULTS

Literature search identified 298 records, of which 22 studies were included. The most commonly studied tumors were posterior fossa tumors including brainstem glioma, ependymoma, medulloblastoma, and pilocytic astrocytoma (15, 68%). Tumor diagnosis was the most frequently performed task (14, 64%), followed by tumor segmentation (3, 14%) and tumor detection (3, 14%). Of the 6 studies comparing AI to clinical experts, 5 demonstrated superiority of AI for tumor diagnosis. Other tasks including tumor segmentation, attenuation correction of positron emission tomography scans, image registration for patient positioning, and dose calculation for radiotherapy were performed with high accuracy comparable with clinical experts. No studies described use of the AI tool in routine clinical practice.

CONCLUSIONS

AI methods for analysis of pediatric brain tumor imaging have increased exponentially in recent years. However, adoption of these methods in clinical practice requires further characterization of validity and utility. Implementation of these methods may streamline clinical workflows by improving diagnostic accuracy and automating basic imaging analysis tasks.

Influence of eye movement on lens dose and optic nerve target coverage during craniospinal irradiation

Purpose

Optic nerves are part of the craniospinal irradiation (CSI) target volume. Modern radiotherapy techniques achieve highly conformal target doses while avoiding organs-at-risk such as the lens. The magnitude of eye movement and its influence on CSI target- and avoidance volumes are unclear. We aimed to evaluate the movement-range of lenses and optic nerves and its influence on dose distribution of several planning techniques.

Methods

Ten volunteers underwent MRI scans in various gaze directions (neutral, left, right, cranial, caudal). Lenses, orbital optic nerves, optic discs and CSI target volumes were delineated. 36-Gy cranial irradiation plans were constructed on synthetic CT images in neutral gaze, with Volumetric Modulated Arc Therapy, pencil-beam scanning proton therapy, and 3D-conventional photons. Movement-amplitudes of lenses and optic discs were analyzed, and influence of gaze direction on lens and orbital optic nerve dose distribution.

Results

Mean eye structures' shift from neutral position was greatest in caudal gaze; -5.8±1.2 mm (±SD) for lenses and 7.0±2.0 mm for optic discs. In 3D-conventional plans, caudal gaze decreased Mean Lens Dose (MLD). In VMAT and proton plans, eye movements mainly increased MLD and diminished D98 orbital optic nerve (D98_OON) coverage; mean MLD increased up to 5.5 Gy [total ΔMLD range -8.1 to 10.0 Gy], and mean D98_OON decreased up to 3.3 Gy [total ΔD98_OON range -13.6 to 1.2 Gy]. VMAT plans optimized for optic disc Internal Target Volume and lens Planning organ-at-Risk Volume resulted in higher MLD over gaze directions. D98_OON became ≥95% of prescribed dose over 95/100 evaluated gaze directions, while all-gaze bilateral D98_OON significantly changed in 1 of 10 volunteers.

Conclusion

With modern CSI techniques, eye movements result in higher lens doses and a mean detriment for orbital optic nerve dose coverage of <10% of prescribed dose.

Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature review

PURPOSE

In radiotherapy, MRI is used for target volume and organs-at-risk delineation for its superior soft-tissue contrast as compared to CT imaging. However, MRI does not provide the electron density of tissue necessary for dose calculation. Several methods of synthetic-CT (sCT) generation from MRI data have been developed for radiotherapy dose calculation. This work reviewed deep learning (DL) sCT generation methods and their associated image and dose evaluation, in the context of MRI-based dose calculation.

METHODS

We searched the PubMed and ScienceDirect electronic databases from January 2010 to March 2021. For each paper, several items were screened and compiled in figures and tables.

RESULTS

This review included 57 studies. The DL methods were either generator-only based (45% of the reviewed studies), or generative adversarial network (GAN) architecture and its variants (55% of the reviewed studies). The brain and pelvis were the most commonly investigated anatomical localizations (39% and 28% of the reviewed studies, respectively), and more rarely, the head-and-neck (H&N) (15%), abdomen (10%), liver (5%) or breast (3%). All the studies performed an image evaluation of sCTs with a diversity of metrics, with only 36 studies performing dosimetric evaluations of sCT.

CONCLUSIONS

The median mean absolute errors were around 76 HU for the brain and H&N sCTs and 40 HU for the pelvis sCTs. For the brain, the mean dose difference between the sCT and the reference CT was <2%. For the H&N and pelvis, the mean dose difference was below 1% in most of the studies. Recent GAN architectures have advantages compared to generator-only, but no superiority was found in term of image or dose sCT uncertainties. Key challenges of DL-based sCT generation methods from MRI in radiotherapy is the management of movement for abdominal and thoracic localizations, the standardization of sCT evaluation, and the investigation of multicenter impacts.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

Deep learning-based synthetic CT generation for MR-only radiotherapy of prostate cancer patients with 0.35T MRI linear accelerator

PURPOSE

To develop a deep learning model to generate synthetic CT for MR-only radiotherapy of prostate cancer patients treated with 0.35 T MRI linear accelerator.

MATERIALS AND METHODS

A U-NET convolutional neural network was developed to translate 0.35 T TRUFI MRI into electron density map using a novel cost function equalizing the contribution of various tissue types including fat, muscle, bone, and background air in training. The impact of training time, dataset size, image standardization, and data augmentation approaches was also quantified. Mean absolute error (MAE) between synthetic and planning CTs was calculated to measure the goodness of the model.

RESULTS

With 20 patients in training, our U-NET model has the potential to generate synthetic CT with a MAE of about 29.68 ± 4.41, 16.34 ± 2.67, 23.36 ± 2.85, and 105.90 ± 22.80 HU over the entire body, fat, muscle, and bone tissues, respectively. As expected, we found that the number of patients used for training and MAE are nonlinearly correlated. Data augmentation and our proposed loss function were effective to improve MAE by ~9% and ~18% in bony voxels, respectively. Increasing the training time and image standardization did not improve the accuracy of the model.

CONCLUSION

A U-NET model has been developed and tested numerically to generate synthetic CT from 0.35T TRUFI MRI for MR-only radiotherapy of prostate cancer patients. Dosimetric evaluation using a large and independent dataset warrants the validity of the proposed model and the actual number of patients needed for the safe usage of the model in routine clinical workflow.

Intra-fractional per-beam adaptive workflow to mitigate the need for a rotating gantry during MRI-guided proton therapy

The integration of real-time magnetic resonance imaging (MRI)-guidance and proton therapy would potentially improve the proton dose steering capability by reducing daily uncertainties due to anatomical variations. The use of a fixed beamline coupled with an axial patient couch rotation would greatly simplify the proton delivery with MRI-guidance. Nonetheless, it is mandatory to assure that the plan quality is not deteriorated by the anatomical deformations due to patient rotation. In this work, an in-house tool allowing for intra-fractional per-beam adaptation of intensity-modulated proton plans (BeamAdapt) was implemented through features available in RayStation. A set of three MRIs was acquired for two healthy volunteers (V1, V2): (1) no rotation/static, (2) rotation to the right and (3) left. V1 was rotated by 15º, to simulate a clinical pediatric abdominal case and V2 by 45º, to simulate an extreme patient rotation case. For each volunteer, a total of four intensity-modulated pencil beam scanning plans were optimized on the static MRI using virtual abdominal targets and 2-3 posterior-oblique beams. Beam angles were defined according to the angulations on the rotated MRIs. With BeamAdapt, each original plan was first converted into separate plans with one beam per plan. In an iterative order, individual beam doses were non-rigidly deformed to the rotated anatomies and re-optimized accounting for the consequent deformations and the beam doses delivered so far. For evaluation, the final adapted dose distribution was propagated back to the static MRI. Planned and adapted dose distributions were compared by computing relative differences between dose-volume histogram (DVH) metrics. Absolute target dose differences were on average below 1% and mean dose organs-at-risk differences were below 3%. With BeamAdapt, not only intra-fractional per-beam proton plan adaptation coupled with axial patient rotation is possible but also the need for a rotating gantry during MRI-guidance might be mitigated.

Facilitating MR-Guided Adaptive Proton Therapy in Children Using Deep Learning-Based Synthetic CT

Purpose

To determine whether self-attention cycle-generative adversarial networks (cycle-GANs), a novel deep-learning method, can generate accurate synthetic computed tomography (sCT) to facilitate adaptive proton therapy in children with brain tumors.

Materials and Methods

Both CT and T1-weighted magnetic resonance imaging (MRI) of 125 children (ages 1-20 years) with brain tumors were included in the training dataset. A model introducing a self-attention mechanism into the conventional cycle-GAN was created to enhance tissue interfaces and reduce noise. The test dataset consisted of 7 patients (ages 2-14 years) who underwent adaptive planning because of changes in anatomy discovered on MRI during proton therapy. The MRI during proton therapy-based sCT was compared with replanning CT (ground truth).

Results

The Hounsfield unit-mean absolute error was significantly reduced with self-attention cycle-GAN, as compared with conventional cycle-GAN (65.3 ± 13.9 versus 88.9 ± 19.3, P &lt; .01). The average 3-dimensional gamma passing rates (2%/2 mm criteria) for the original plan on the anatomy of the day and for the adapted plan were high (97.6% ± 1.2% and 98.9 ± 0.9%, respectively) when using sCT generated by self-attention cycle-GAN. The mean absolute differences in clinical target volume (CTV) receiving 95% of the prescription dose and 80% distal falloff along the beam axis were 1.1% ± 0.8% and 1.1 ± 0.9 mm, respectively. Areas of greatest dose difference were distal to the CTV and corresponded to shifts in distal falloff. Plan adaptation was appropriately triggered in all test patients when using sCT.

Conclusion

The novel cycle-GAN model with self-attention outperforms conventional cycle-GAN for children with brain tumors. Encouraging dosimetric results suggest that sCT generation can be used to identify patients who would benefit from adaptive replanning.

Artificial Intelligence in magnetic Resonance guided Radiotherapy: Medical and physical considerations on state of art and future perspectives

Over the last years, technological innovation in Radiotherapy (RT) led to the introduction of Magnetic Resonance-guided RT (MRgRT) systems. Due to the higher soft tissue contrast compared to on-board CT-based systems, MRgRT is expected to significantly improve the treatment in many situations. MRgRT systems may extend the management of inter- and intra-fraction anatomical changes, offering the possibility of online adaptation of the dose distribution according to daily patient anatomy and to directly monitor tumor motion during treatment delivery by means of a continuous cine MR acquisition. Online adaptive treatments require a multidisciplinary and well-trained team, able to perform a series of operations in a safe, precise and fast manner while the patient is waiting on the treatment couch. Artificial Intelligence (AI) is expected to rapidly contribute to MRgRT, primarily by safely and efficiently automatising the various manual operations characterizing online adaptive treatments. Furthermore, AI is finding relevant applications in MRgRT in the fields of image segmentation, synthetic CT reconstruction, automatic (on-line) planning and the development of predictive models based on daily MRI. This review provides a comprehensive overview of the current AI integration in MRgRT from a medical physicist's perspective. Medical physicists are expected to be major actors in solving new tasks and in taking new responsibilities: their traditional role of guardians of the new technology implementation will change with increasing emphasis on the managing of AI tools, processes and advanced systems for imaging and data analysis, gradually replacing many repetitive manual tasks.

Cone beam computed tomography based image guidance and quality assessment of prostate cancer for magnetic resonance imaging-only radiotherapy in the pelvis

•

MRI-only IGRT accuracy is ≤2 mm as compared to CT but significant differences were observed.

•

MRI-only CBCT-based IGRT seems feasible but caution is advised.

•

The median absolute error (MeAE) for independent verification on the sCT quality is proposed.

•

A MeAE around 0.1 in mass density could call for sCT quality inspection.

Background and purpose

Radiotherapy (RT) based on magentic resonance imaging (MRI) only is currently used clinically in the pelvis. A synthetic computed tomography (sCT) is needed for dose planning. Here, we investigate the accuracy of cone beam CT (CBCT) based MRI-only image guided RT (IGRT) and sCT image quality.

Materials and methods

CT, MRI and CBCT scans of ten prostate cancer patients were included. The MRI was converted to a sCT using a multi-atlas approach. The sCT, CT and MR images were auto-matched with the CBCT on the bony anatomy. Paired sCT-CT and sCT-CBCT data were created. CT numbers were converted to relative electron (RED) and mass densities (DES) using a standard calibration curve for the CT and sCT. For the CBCT RED/DES conversion, a phantom and paired CT-CBCT population based calibration curve was used. For the latter, the CBCT numbers were averaged in 100 HU bins and the known RED/DES of the CT were assigned. The paired sCT-CT and sCT-CBCT data were averaged in bins of 10 HU or 0.01 RED/DES. The median absolute error (MeAE) between the sCT-CT and sCT-CBCT bins was calculated. Wilcoxon rank-sum tests were carried out for the IGRT and MeAE study.

Results

The mean sCT or MR IGRT difference from CT was ≤ 2 mm but significant differences were observed. A CBCT HU or phantom-based RED/DES MeAE did not estimate the sCT quality similar to a CT based MeAE but the CBCT population-based RED/DES MeAE did.

Conclusions

MRI-only CBCT-based IGRT seems feasible but caution is advised. A MeAE around 0.1 DES could call for sCT quality inspection.

A feature invariant generative adversarial network for head and neck MRI/CT image synthesis

With the emergence of online MRI radiotherapy treatments, MR-based workflows have increased in importance in the clinical workflow. However proper dose planning still requires CT images to calculate dose attenuation due to bony structures. In this paper, we present a novel deep image synthesis model that generates in an unsupervised manner CT images from diagnostic MRI for radiotherapy planning. The proposed model based on a generative adversarial network (GAN) consists of learning a new invariant representation to generate synthetic CT (sCT) images based on high frequency and appearance patterns. This new representation encodes each convolutional feature map of the convolutional GAN discriminator, leading the training of the proposed model to be particularly robust in terms of image synthesis quality. Our model includes an analysis of common histogram features in the training process, thus reinforcing the generator such that the output sCT image exhibits a histogram matching that of the ground-truth CT. This CT-matched histogram is embedded then in a multi-resolution framework by assessing the evaluation over all layers of the discriminator network, which then allows the model to robustly classify the output synthetic image. Experiments were conducted on head and neck images of 56 cancer patients with a wide range of shape sizes and spatial image resolutions. The obtained results confirm the efficiency of the proposed model compared to other generative models, where the mean absolute error yielded by our model was 26.44(0.62), with a Hounsfield unit error of 45.3(1.87), and an overall Dice coefficient of 0.74(0.05), demonstrating the potential of the synthesis model for radiotherapy planning applications.

Synthetic CT for single-fraction neoadjuvant partial breast irradiation on an MRI-linac

A synthetic computed tomography (sCT) is required for daily plan optimization on an MRI-linac. Yet, only limited information is available on the accuracy of dose calculations on sCT for breast radiotherapy. This work aimed to (1) evaluate dosimetric accuracy of treatment plans for single-fraction neoadjuvant partial breast irradiation (PBI) on a 1.5 T MRI-linac calculated on a) bulk-density sCT mimicking the current MRI-linac workflow and b) deep learning-generated sCT, and (2) investigate the number of bulk-density levels required. For ten breast cancer patients we created three bulk-density sCTs of increasing complexity from the planning-CT, using bulk-density for: (1) body, lungs, and GTV (sCT_BD1); (2) volumes for sCT_BD1plus chest wall and ipsilateral breast (sCT_BD2); (3) volumes for sCT_BD2plus ribs (sCT_BD3); and a deep learning-generated sCT (sCT_DL) from a 1.5 T MRI in supine position. Single-fraction neoadjuvant PBI treatment plans for a 1.5 T MRI-linac were optimized on each sCT and recalculated on the planning-CT. Image evaluation was performed by assessing mean absolute error (MAE) and mean error (ME) in Hounsfield Units (HU) between the sCTs and the planning-CT. Dosimetric evaluation was performed by assessing dose differences, gamma pass rates, and dose-volume histogram (DVH) differences. The following results were obtained (median across patients for sCT_BD1/sCT_BD2/sCT_BD3/sCT_DLrespectively): MAE inside the body contour was 106/104/104/75 HU and ME was 8/9/6/28 HU, mean dose difference in the PTV_GTVwas 0.15/0.00/0.00/-0.07 Gy, median gamma pass rate (2%/2 mm, 10% dose threshold) was 98.9/98.9/98.7/99.4%, and differences in DVH parameters were well below 2% for all structures except for the skin in the sCT_DL. Accurate dose calculations for single-fraction neoadjuvant PBI on an MRI-linac could be performed on both bulk-density and deep learning sCT, facilitating further implementation of MRI-guided radiotherapy for breast cancer. Balancing simplicity and accuracy, sCT_BD2showed the optimal number of bulk-density levels for a bulk-density approach.

Technical Challenges of Real-Time Adaptive MR-Guided Radiotherapy

In the past few years, radiotherapy (RT) has experienced a major technological innovation with the development of hybrid machines combining magnetic resonance (MR) imaging and linear accelerators. This new technology for MR-guided cancer treatment has the potential to revolutionize the field of adaptive RT due to the opportunity to provide high-resolution, real-time MR imaging before and during treatment application. However, from a technical point of view, several challenges remain which need to be tackled to ensure safe and robust real-time adaptive MR-guided RT delivery. In this manuscript, several technical challenges to MR-guided RT are discussed. Starting with magnetic field strength tradeoffs, the potential and limitations for purely MR-based RT workflows are discussed. Furthermore, the current status of real-time 3D MR imaging and its potential for real-time RT are summarized. Finally, the potential of quantitative MR imaging for future biological RT adaptation is highlighted.