Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch‐based three‐dimensional convolutional neural network

Synthetic CT for gamma knife radiosurgery dose calculation: A feasibility study

PURPOSE

To determine if MRI-based synthetic CTs (sCT), generated with no predefined pulse sequence, can be used for inhomogeneity correction in routine gamma knife radiosurgery (GKRS) treatment planning dose calculation.

METHODS

Two sets of sCTs were generated from T1post and T2 images using cycleGAN. Twenty-eight patients (18 training, 10 validation) were retrospectively selected. The image quality of the generated sCTs was compared with the original CT (oCT) regarding the HU value preservation using histogram comparison, RMSE and MAE, and structural integrity. Dosimetric comparisons were also made among GKRS plans from 3 calculation approaches: TMR10 (oCT), and convolution (oCT and sCT), at four locations: original disease site, bone/tissue interface, air/tissue interface, and mid-brain.

RESULTS

The study showed that sCTs and oCTs' HU were similar, with T2-sCT performing better. TMR10 significantly underdosed the target by a mean of 5.4% compared to the convolution algorithm. There was no significant difference in convolution algorithm shot time between the oCT and sCT generated with T2. The highest and lowest dosimetric differences between the two CTs were observed in the bone and air interface, respectively. Dosimetric differences of 3.3% were observed in sCT predicted from MRI with stereotactic frames, which was not included in the training sets.

CONCLUSIONS

MRI-based sCT can be utilized for GKRS convolution dose calculation without the unnecessary radiation dose, and sCT without metal artifacts could be generated in framed cases. Larger datasets inclusive of all pulse sequences can improve the training set. Further investigation and validation studies are needed before clinical implementation.

Generating Synthetic Data for Medical Imaging

Artificial intelligence (AI) models for medical imaging tasks, such as classification or segmentation, require large and diverse datasets of images. However, due to privacy and ethical issues, as well as data sharing infrastructure barriers, these datasets are scarce and difficult to assemble. Synthetic medical imaging data generated by AI from existing data could address this challenge by augmenting and anonymizing real imaging data. In addition, synthetic data enable new applications, including modality translation, contrast synthesis, and professional training for radiologists. However, the use of synthetic data also poses technical and ethical challenges. These challenges include ensuring the realism and diversity of the synthesized images while keeping data unidentifiable, evaluating the performance and generalizability of models trained on synthetic data, and high computational costs. Since existing regulations are not sufficient to guarantee the safe and ethical use of synthetic images, it becomes evident that updated laws and more rigorous oversight are needed. Regulatory bodies, physicians, and AI developers should collaborate to develop, maintain, and continually refine best practices for synthetic data. This review aims to provide an overview of the current knowledge of synthetic data in medical imaging and highlights current key challenges in the field to guide future research and development.

Challenges and opportunities in the development and clinical implementation of artificial intelligence based synthetic computed tomography for magnetic resonance only radiotherapy

Synthetic computed tomography (sCT) generated from magnetic resonance imaging (MRI) can serve as a substitute for planning CT in radiation therapy (RT), thereby removing registration uncertainties associated with multi-modality imaging pairing, reducing costs and patient radiation exposure. CE/FDA-approved sCT solutions are nowadays available for pelvis, brain, and head and neck, while more complex deep learning (DL) algorithms are under investigation for other anatomic sites. The main challenge in achieving a widespread clinical implementation of sCT lies in the absence of consensus on sCT commissioning and quality assurance (QA), resulting in variation of sCT approaches across different hospitals. To address this issue, a group of experts gathered at the ESTRO Physics Workshop 2022 to discuss the integration of sCT solutions into clinics and report the process and its outcomes. This position paper focuses on aspects of sCT development and commissioning, outlining key elements crucial for the safe implementation of an MRI-only RT workflow.

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

A unified generation-registration framework for improved MR-based CT synthesis in proton therapy

BACKGROUND

The use of magnetic resonance (MR) imaging for proton therapy treatment planning is gaining attention as a highly effective method for guidance. At the core of this approach is the generation of computed tomography (CT) images from MR scans. However, the critical issue in this process is accurately aligning the MR and CT images, a task that becomes particularly challenging in frequently moving body areas, such as the head-and-neck. Misalignments in these images can result in blurred synthetic CT (sCT) images, adversely affecting the precision and effectiveness of the treatment planning.

PURPOSE

This study introduces a novel network that cohesively unifies image generation and registration processes to enhance the quality and anatomical fidelity of sCTs derived from better-aligned MR images.

METHODS

The approach synergizes a generation network (G) with a deformable registration network (R), optimizing them jointly in MR-to-CT synthesis. This goal is achieved by alternately minimizing the discrepancies between the generated/registered CT images and their corresponding reference CT counterparts. The generation network employs a UNet architecture, while the registration network leverages an implicit neural representation (INR) of the displacement vector fields (DVFs). We validated this method on a dataset comprising 60 head-and-neck patients, reserving 12 cases for holdout testing.

RESULTS

Compared to the baseline Pix2Pix method with MAE 124.95 ± $\pm$ 30.74 HU, the proposed technique demonstrated 80.98 ± $\pm$ 7.55 HU. The unified translation-registration network produced sharper and more anatomically congruent outputs, showing superior efficacy in converting MR images to sCTs. Additionally, from a dosimetric perspective, the plan recalculated on the resulting sCTs resulted in a remarkably reduced discrepancy to the reference proton plans.

CONCLUSIONS

This study conclusively demonstrates that a holistic MR-based CT synthesis approach, integrating both image-to-image translation and deformable registration, significantly improves the precision and quality of sCT generation, particularly for the challenging body area with varied anatomic changes between corresponding MR and CT.

Multi-planar dual adversarial network based on dynamic 3D features for MRI-CT head and neck image synthesis

Objective.Head and neck radiotherapy planning requires electron densities from different tissues for dose calculation. Dose calculation from imaging modalities such as MRI remains an unsolved problem since this imaging modality does not provide information about the density of electrons.Approach.We propose a generative adversarial network (GAN) approach that synthesizes CT (sCT) images from T1-weighted MRI acquisitions in head and neck cancer patients. Our contribution is to exploit new features that are relevant for improving multimodal image synthesis, and thus improving the quality of the generated CT images. More precisely, we propose a Dual branch generator based on the U-Net architecture and on an augmented multi-planar branch. The augmented branch learns specific 3D dynamic features, which describe the dynamic image shape variations and are extracted from different view-points of the volumetric input MRI. The architecture of the proposed model relies on an end-to-end convolutional U-Net embedding network.Results.The proposed model achieves a mean absolute error (MAE) of18.76±5.167in the target Hounsfield unit (HU) space on sagittal head and neck patients, with a mean structural similarity (MSSIM) of0.95±0.09and a Frechet inception distance (FID) of145.60±8.38. The model yields a MAE of26.83±8.27to generate specific primary tumor regions on axial patient acquisitions, with a Dice score of0.73±0.06and a FID distance equal to122.58±7.55. The improvement of our model over other state-of-the-art GAN approaches is of 3.8%, on a tumor test set. On both sagittal and axial acquisitions, the model yields the best peak signal-to-noise ratio of27.89±2.22and26.08±2.95to synthesize MRI from CT input.Significance.The proposed model synthesizes both sagittal and axial CT tumor images, used for radiotherapy treatment planning in head and neck cancer cases. The performance analysis across different imaging metrics and under different evaluation strategies demonstrates the effectiveness of our dual CT synthesis model to produce high quality sCT images compared to other state-of-the-art approaches. Our model could improve clinical tumor analysis, in which a further clinical validation remains to be explored.

Synthetic CT generation based on CBCT using improved vision transformer CycleGAN

Cone-beam computed tomography (CBCT) is a crucial component of adaptive radiation therapy; however, it frequently encounters challenges such as artifacts and noise, significantly constraining its clinical utility. While CycleGAN is a widely employed method for CT image synthesis, it has notable limitations regarding the inadequate capture of global features. To tackle these challenges, we introduce a refined unsupervised learning model called improved vision transformer CycleGAN (IViT-CycleGAN). Firstly, we integrate a U-net framework that builds upon ViT. Next, we augment the feed-forward neural network by incorporating deep convolutional networks. Lastly, we enhance the stability of the model training process by introducing gradient penalty and integrating an additional loss term into the generator loss. The experiment demonstrates from multiple perspectives that our model-generated synthesizing CT(sCT) has significant advantages compared to other unsupervised learning models, thereby validating the clinical applicability and robustness of our model. In future clinical practice, our model has the potential to assist clinical practitioners in formulating precise radiotherapy plans.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Incidental findings and safety events from magnetic resonance imaging simulation for head and neck radiation treatment planning: A single institution experience

Purpose

Having dedicated MRI scanners within radiation oncology departments may present unexpected challenges since radiation oncologists and radiation therapists are generally not trained in this modality and there are potential patient safety concerns. This study retrospectively reviews the incidental findings and safety events that were observed at a single institution during introduction of MRI sim for head and neck radiotherapy planning.

Methods

Consecutive patients from March 1, 2020, to May 31, 2022, who were scheduled for MRI sim after having completed CT simulation for head and neck radiotherapy were included for analysis. Patients first underwent a CT simulation with a thermoplastic mask and in most cases with an intraoral stent. The same setup was then reproduced in the MRI simulator. Safety events were instances where scheduled MRI sims were not completed due to the MRI technologist identifying MRI-incompatible devices or objects at the time of sim. Incidental findings were identified during weekly quality assurance rounds as a joint enterprise of head and neck radiation oncology and neuroradiology. Categorical variables between completed and not completed MRI sims were compared using the Chi-Square test and continuous variables were compared using the Mann-Whitney U test with a p-value of < 0.05 considered to be statistically significant.

Results

148 of 169 MRI sims (88 %) were completed as scheduled and 21 (12 %) were not completed (Table 1). Among the 21 aborted MRI sims, the most common reason was due to safety events flagged by the MRI technologist (n = 8, 38 %) because of the presence of metal or a medical device that was not noted at the time of initial screening by the administrative coordinator. Patients who did not complete MRI sim were more likely to be treated for non-squamous head and neck primary tumor (p = 0.016) and were being treated post-operatively (p < 0.001). CT and MRI sim scans each had 17 incidental findings. CT simulation detected 3 cases of new metastases in lungs, which were outside the scan parameters of MRI sim. MRI sim detected one case of dural venous thrombosis and one case of cervical spine epidural abscess, which were not detected by CT simulation.

Conclusions

Radiation oncology departments with dedicated MRI simulation scanners would benefit from diagnostic radiology review for incidental findings and having therapists with MRI safety credentialing to catch near-miss events.

A study of Bayesian deep network uncertainty and its application to synthetic CT generation for MR-only radiotherapy treatment planning

BACKGROUND

The use of synthetic computed tomography (CT) for radiotherapy treatment planning has received considerable attention because of the absence of ionizing radiation and close spatial correspondence to source magnetic resonance (MR) images, which have excellent tissue contrast. However, in an MR-only environment, little effort has been made to examine the quality of synthetic CT images without using the original CT images.

PURPOSE

To estimate synthetic CT quality without referring to original CT images, this study established the relationship between synthetic CT uncertainty and Bayesian uncertainty, and proposed a new Bayesian deep network for generating synthetic CT images and estimating synthetic CT uncertainty for MR-only radiotherapy treatment planning.

METHODS AND MATERIALS

A novel deep Bayesian network was formulated using probabilistic network weights. Two mathematical expressions were proposed to quantify the Bayesian uncertainty of the network and synthetic CT uncertainty, which was closely related to the mean absolute error (MAE) in Hounsfield Unit (HU) of synthetic CT. These uncertainties were examined to demonstrate the accuracy of representing the synthetic CT uncertainty using a Bayesian counterpart. We developed a hybrid Bayesian architecture and a new data normalization scheme, enabling the Bayesian network to generate both accurate synthetic CT and reliable uncertainty information when probabilistic weights were applied. The proposed method was evaluated in 59 patients (13/12/32/2 for training/validation/testing/uncertainty visualization) diagnosed with prostate cancer, who underwent same-day pelvic CT- and MR-acquisitions. To assess the relationship between Bayesian and synthetic CT uncertainties, linear and non-linear correlation coefficients were calculated on per-voxel, per-tissue, and per-patient bases. For accessing the accuracy of the CT number and dosimetric accuracy, the proposed method was compared with a commercially available atlas-based method (MRCAT) and a U-Net conditional-generative adversarial network (UcGAN).

RESULTS

The proposed model exhibited 44.33 MAE, outperforming UcGAN 52.51 and MRCAT 54.87. The gamma rate (2%/2 mm dose difference/distance to agreement) of the proposed model was 98.68%, comparable to that of UcGAN (98.60%) and MRCAT (98.56%). The per-patient and per-tissue linear correlation coefficients between the Bayesian and synthetic CT uncertainties ranged from 0.53 to 0.83, implying a moderate to strong linear correlation. Per-voxel correlation coefficients varied from -0.13 to 0.67 depending on the regions-of-interest evaluated, indicating tissue-dependent correlation. The R2 value for estimating MAE solely using Bayesian uncertainty was 0.98, suggesting that the uncertainty of the proposed model was an ideal candidate for predicting synthetic CT error, without referring to the original CT.

CONCLUSION

This study established a relationship between the Bayesian model uncertainty and synthetic CT uncertainty. A novel Bayesian deep network was proposed to generate a synthetic CT and estimate its uncertainty. Various metrics were used to thoroughly examine the relationship between the uncertainties of the proposed Bayesian model and the generated synthetic CT. Compared with existing approaches, the proposed model showed comparable CT number and dosimetric accuracies. The experiments showed that the proposed Bayesian model was capable of producing accurate synthetic CT, and was an effective indicator of the uncertainty and error associated with synthetic CT in MR-only workflows.

Deep learning in MRI-guided radiation therapy: A systematic review

Recent advances in MRI-guided radiation therapy (MRgRT) and deep learning techniques encourage fully adaptive radiation therapy (ART), real-time MRI monitoring, and the MRI-only treatment planning workflow. Given the rapid growth and emergence of new state-of-the-art methods in these fields, we systematically review 197 studies written on or before December 31, 2022, and categorize the studies into the areas of image segmentation, image synthesis, radiomics, and real time MRI. Building from the underlying deep learning methods, we discuss their clinical importance and current challenges in facilitating small tumor segmentation, accurate x-ray attenuation information from MRI, tumor characterization and prognosis, and tumor motion tracking. In particular, we highlight the recent trends in deep learning such as the emergence of multi-modal, visual transformer, and diffusion models.

Evaluating the Hounsfield unit assignment and dose differences between CT-based standard and deep learning-based synthetic CT images for MRI-only radiation therapy of the head and neck

BACKGROUND

Magnetic resonance image only (MRI-only) simulation for head and neck (H&N) radiotherapy (RT) could allow for single-image modality planning with excellent soft tissue contrast. In the MRI-only simulation workflow, synthetic computed tomography (sCT) is generated from MRI to provide electron density information for dose calculation. Bone/air regions produce little MRI signal which could lead to electron density misclassification in sCT. Establishing the dosimetric impact of this error could inform quality assurance (QA) procedures using MRI-only RT planning or compensatory methods for accurate dosimetric calculation.

PURPOSE

The aim of this study was to investigate if Hounsfield unit (HU) voxel misassignments from sCT images result in dosimetric errors in clinical treatment plans.

METHODS

Fourteen H&N cancer patients undergoing same-day CT and 3T MRI simulation were retrospectively identified. MRI was deformed to the CT using multimodal deformable image registration. sCTs were generated from T1w DIXON MRIs using a commercially available deep learning-based generator (MRIplanner, Spectronic Medical AB, Helsingborg, Sweden). Tissue voxel assignment was quantified by creating a CT-derived HU threshold contour. CT/sCT HU differences for anatomical/target contours and tissue classification regions including air (<250 HU), adipose tissue (-250 HU to -51 HU), soft tissue (-50 HU to 199 HU), spongy (200 HU to 499 HU) and cortical bone (>500 HU) were quantified. t-test was used to determine if sCT/CT HU differences were significant. The frequency of structures that had a HU difference > 80 HU (the CT window-width setting for intra-cranial structures) was computed to establish structure classification accuracy. Clinical intensity modulated radiation therapy (IMRT) treatment plans created on CT were retrospectively recalculated on sCT images and compared using the gamma metric.

RESULTS

The mean ratio of sCT HUs relative to CT for air, adipose tissue, soft tissue, spongy and cortical bone were 1.7 ± 0.3, 1.1 ± 0.1, 1.0 ± 0.1, 0.9 ± 0.1 and 0.8 ± 0.1 (value of 1 indicates perfect agreement). T-tests (significance set at t = 0.05) identified differences in HU values for air, spongy and cortical bone in sCT images compared to CT. The structures with sCT/CT HU differences > 80 HU of note were the left and right (L/R) cochlea and mandible (>79% of the tested cohort), the oral cavity (for 57% of the tested cohort), the epiglottis (for 43% of the tested cohort) and the L/R TM joints (occurring > 29% of the cohort). In the case of the cochlea and TM joints, these structures contain dense bone/air interfaces. In the case of the oral cavity and mandible, these structures suffer the additional challenge of being positionally altered in CT versus MRI simulation (due to a non-MR safe immobilizing bite block requiring absence of bite block in MR). Finally, the epiglottis HU assignment suffers from its small size and unstable positionality. Plans recalculated on sCT yielded global/local gamma pass rates of 95.5% ± 2% (3 mm, 3%) and 92.7% ± 2.1% (2 mm, 2%). The largest mean differences in D95, Dmean , D50 dose volume histogram (DVH) metrics for organ-at-risk (OAR) and planning tumor volumes (PTVs) were 2.3% ± 3.0% and 0.7% ± 1.9% respectively.

CONCLUSIONS

In this cohort, HU differences of CT and sCT were observed but did not translate into a reduction in gamma pass rates or differences in average PTV/OAR dose metrics greater than 3%. For sites such as the H&N where there are many tissue interfaces we did not observe large scale dose deviations but further studies using larger retrospective cohorts are merited to establish the variation in sCT dosimetric accuracy which could help to inform QA limits on clinical sCT usage.

Comprehensive evaluation of similarity between synthetic and real CT images for nasopharyngeal carcinoma

BACKGROUND

Although magnetic resonance imaging (MRI)-to-computed tomography (CT) synthesis studies based on deep learning have significantly progressed, the similarity between synthetic CT (sCT) and real CT (rCT) has only been evaluated in image quality metrics (IQMs). To evaluate the similarity between synthetic CT (sCT) and real CT (rCT) comprehensively, we comprehensively evaluated IQMs and radiomic features for the first time.

METHODS

This study enrolled 127 patients with nasopharyngeal carcinoma who underwent CT and MRI scans. Supervised-learning (Unet) and unsupervised-learning (CycleGAN) methods were applied to build MRI-to-CT synthesis models. The regions of interest (ROIs) included nasopharynx gross tumor volume (GTVnx), brainstem, parotid glands, and temporal lobes. The peak signal-to-noise ratio (PSNR), mean absolute error (MAE), root mean square error (RMSE), and structural similarity (SSIM) were used to evaluate image quality. Additionally, 837 radiomic features were extracted for each ROI, and the correlation was evaluated using the concordance correlation coefficient (CCC).

RESULTS

The MAE, RMSE, SSIM, and PSNR of the body were 91.99, 187.12, 0.97, and 51.15 for Unet and 108.30, 211.63, 0.96, and 49.84 for CycleGAN. For the metrics, Unet was superior to CycleGAN (P < 0.05). For the radiomic features, the percentage of four levels (i.e., excellent, good, moderate, and poor, respectively) were as follows: GTVnx, 8.5%, 14.6%, 26.5%, and 50.4% for Unet and 12.3%, 25%, 38.4%, and 24.4% for CycleGAN; other ROIs, 5.44% ± 3.27%, 5.56% ± 2.92%, 21.38% ± 6.91%, and 67.58% ± 8.96% for Unet and 5.16% ± 1.69%, 3.5% ± 1.52%, 12.68% ± 7.51%, and 78.62% ± 8.57% for CycleGAN.

CONCLUSIONS

Unet-sCT was superior to CycleGAN-sCT for the IQMs. However, neither exhibited absolute superiority in radiomic features, and both were far less similar to rCT. Therefore, further work is required to improve the radiomic similarity for MRI-to-CT synthesis.

TRIAL REGISTRATION

This study was a retrospective study, so it was free from registration.

The three-dimensional coupling mechanism in scoliosis and its consequences for correction

INTRODUCTION

In idiopathic scoliosis, the anterior spinal column has rotated away from the midline and has become longer through unloading and expansion of the intervertebral discs. Theoretically, extension of the spine in the sagittal plane should provide room for this longer anterior spinal column, allowing it to swing back towards the midline in the coronal and axial plane, thus reducing both the Cobb angle and the apical vertebral rotation.

METHODS

In this prospective experimental study, ten patients with primary thoracic adolescent idiopathic scoliosis (AIS) underwent MRI (BoneMRI and cVISTA sequences) in supine as well as in an extended position by placing a broad bolster, supporting both hemi-thoraces, under the scoliotic apex. Differences in T4-T12 kyphosis angle, coronal Cobb angle, vertebral rotation, as well as shape of the intervertebral disc and shape and position of the nucleus pulposus, were analysed and compared between the two positions.

RESULTS

Extension reduced T4-T12 thoracic kyphosis by 10° (p < 0.001), the coronal Cobb angle decreased by 9° (p < 0.001) and vertebral rotation by 4° (p = 0.036). The coronal wedge shape of the disc significantly normalized and the wedged and lateralized nucleus pulposus partially reduced to a more symmetrical position.

CONCLUSION

Simple extension of the scoliotic spine leads to a reduction of the deformity in the coronal and axial plane. The shape of the disc normalizes and the eccentric nucleus pulposus partially moves back to the midline.

Artificial general intelligence for radiation oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale.

CT synthesis from CBCT using a sequence-aware contrastive generative network

Computerized tomography (CT) synthesis from cone-beam computerized tomography (CBCT) is a key step in adaptive radiotherapy. It uses a synthetic CT to calculate the dose to correct and adjust the radiotherapy plan in a timely manner. The cycle-consistent adversarial network (Cycle GAN) is commonly used in CT synthesis tasks but it has some defects: (a) the premise of the cycle consistency loss is that the conversion between domains is bijective, but the CBCT and CT conversion does not fully satisfy the bijective relationship, and (b) it does not take advantage of the complementary information between multiple sets of CBCTs for the same patient. To address these problems, we propose a novel framework named the sequence-aware contrastive generative network (SCGN) that introduces an attention sequence fusion module to improve the CBCT quality. In addition, it not only applies contrastive learning to the generative adversarial networks (GANs) to pay more attention to the anatomical structure of CBCT in feature extraction but also uses a new generator to improve the accuracy of the anatomical details. Experimental results on our datasets show that our method significantly outperforms the existing unsupervised CT synthesis methods.

Machine Learning for Medical Image Translation: A Systematic Review

BACKGROUND

CT scans are often the first and only form of brain imaging that is performed to inform treatment plans for neurological patients due to its time- and cost-effective nature. However, MR images give a more detailed picture of tissue structure and characteristics and are more likely to pick up abnormalities and lesions. The purpose of this paper is to review studies which use deep learning methods to generate synthetic medical images of modalities such as MRI and CT.

METHODS

A literature search was performed in March 2023, and relevant articles were selected and analyzed. The year of publication, dataset size, input modality, synthesized modality, deep learning architecture, motivations, and evaluation methods were analyzed.

RESULTS

A total of 103 studies were included in this review, all of which were published since 2017. Of these, 74% of studies investigated MRI to CT synthesis, and the remaining studies investigated CT to MRI, Cross MRI, PET to CT, and MRI to PET. Additionally, 58% of studies were motivated by synthesizing CT scans from MRI to perform MRI-only radiation therapy. Other motivations included synthesizing scans to aid diagnosis and completing datasets by synthesizing missing scans.

CONCLUSIONS

Considerably more research has been carried out on MRI to CT synthesis, despite CT to MRI synthesis yielding specific benefits. A limitation on medical image synthesis is that medical datasets, especially paired datasets of different modalities, are lacking in size and availability; it is therefore recommended that a global consortium be developed to obtain and make available more datasets for use. Finally, it is recommended that work be carried out to establish all uses of the synthesis of medical scans in clinical practice and discover which evaluation methods are suitable for assessing the synthesized images for these needs.

Compensation cycle consistent generative adversarial networks (Comp-GAN) for synthetic CT generation from MR scans with truncated anatomy

BACKGROUND

MR scans used in radiotherapy can be partially truncated due to the limited field of view (FOV), affecting dose calculation accuracy in MR-based radiation treatment planning.

PURPOSE

We proposed a novel Compensation-cycleGAN (Comp-cycleGAN) by modifying the cycle-consistent generative adversarial network (cycleGAN), to simultaneously create synthetic CT (sCT) images and compensate the missing anatomy from the truncated MR images.

METHODS

Computed tomography (CT) and T1 MR images with complete anatomy of 79 head-and-neck patients were used for this study. The original MR images were manually cropped 10-25 mm off at the posterior head to simulate clinically truncated MR images. Fifteen patients were randomly chosen for testing and the rest of the patients were used for model training and validation. Both the truncated and original MR images were used in the Comp-cycleGAN training stage, which enables the model to compensate for the missing anatomy by learning the relationship between the truncation and known structures. After the model was trained, sCT images with complete anatomy can be generated by feeding only the truncated MR images into the model. In addition, the external body contours acquired from the CT images with full anatomy could be an optional input for the proposed method to leverage the additional information of the actual body shape for each test patient. The mean absolute error (MAE) of Hounsfield units (HU), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) were calculated between sCT and real CT images to quantify the overall sCT performance. To further evaluate the shape accuracy, we generated the external body contours for sCT and original MR images with full anatomy. The Dice similarity coefficient (DSC) and mean surface distance (MSD) were calculated between the body contours of sCT and original MR images for the truncation region to assess the anatomy compensation accuracy.

RESULTS

The average MAE, PSNR, and SSIM calculated over test patients were 93.1 HU/91.3 HU, 26.5 dB/27.4 dB, and 0.94/0.94 for the proposed Comp-cycleGAN models trained without/with body-contour information, respectively. These results were comparable with those obtained from the cycleGAN model which is trained and tested on full-anatomy MR images, indicating the high quality of the sCT generated from truncated MR images by the proposed method. Within the truncated region, the mean DSC and MSD were 0.85/0.89 and 1.3/0.7 mm for the proposed Comp-cycleGAN models trained without/with body contour information, demonstrating good performance in compensating the truncated anatomy.

CONCLUSIONS

We developed a novel Comp-cycleGAN model that can effectively create sCT with complete anatomy compensation from truncated MR images, which could potentially benefit the MRI-based treatment planning.

Synthetic computed tomography for low-field magnetic resonance-only radiotherapy in head-and-neck cancer using residual vision transformers

Background and purpose

Synthetic computed tomography (sCT) scans are necessary for dose calculation in magnetic resonance (MR)-only radiotherapy. While deep learning (DL) has shown remarkable performance in generating sCT scans from MR images, research has predominantly focused on high-field MR images. This study presents the first implementation of a DL model for sCT generation in head-and-neck (HN) cancer using low-field MR images. Specifically, the use of vision transformers (ViTs) was explored.

Materials and methods

The dataset consisted of 31 patients, resulting in 196 pairs of deformably-registered computed tomography (dCT) and MR scans. The latter were obtained using a balanced steady-state precession sequence on a 0.35T scanner. Residual ViTs were trained on 2D axial, sagittal, and coronal slices, respectively, and the final sCTs were generated by averaging the models' outputs. Different image similarity metrics, dose volume histogram (DVH) deviations, and gamma analyses were computed on the test set (n = 6). The overlap between auto-contours on sCT scans and manual contours on MR images was evaluated for different organs-at-risk using the Dice score.

Results

The median [range] value of the test mean absolute error was 57 [37-74] HU. DVH deviations were below 1% for all structures. The median gamma passing rates exceeded 94% in the 2%/2mm analysis (threshold = 90%). The median Dice scores were above 0.7 for all organs-at-risk.

Conclusions

The clinical applicability of DL-based sCT generation from low-field MR images in HN cancer was proved. High sCT-dCT similarity and dose metric accuracy were achieved, and sCT suitability for organs-at-risk auto-delineation was shown.

Evaluation of MRI-only based online adaptive radiotherapy of abdominal region on MR-linac

PURPOSE

A hybrid magnetic resonance linear accelerator (MRL) can perform magnetic resonance imaging (MRI) with high soft-tissue contrast to be used for online adaptive radiotherapy (oART). To obtain electron densities needed for the oART dose calculation, a computed tomography (CT) is often deformably registered to MRI. Our aim was to evaluate an MRI-only based synthetic CT (sCT) generation as an alternative to the deformed CT (dCT)-based oART in the abdominal region.

METHODS

The study data consisted of 57 patients who were treated on a 0.35 T MRL system mainly for abdominal tumors. Simulation MRI-CT pairs of 43 patients were used for training and validation of a prototype convolutional neural network sCT-generation algorithm, based on HighRes3DNet, for the abdominal region. For remaining test patients, sCT images were produced from simulation MRIs and daily MRIs. The dCT-based plans were re-calculated on sCT with identical calculation parameters. The sCT and dCT were compared in terms of geometric agreement and calculated dose.

RESULTS

The mean and one standard deviation of the geometric agreement metrics over dCT-sCT-pairs were: mean error of 8 ± 10 HU, mean absolute error of 49 ± 10 HU, and Dice similarity coefficient of 55 ± 12%, 60 ± 5%, and 82 ± 15% for bone, fat, and lung tissues, respectively. The dose differences between the sCT and dCT-based dose for planning target volumes were 0.5 ± 0.9%, 0.6 ± 0.8%, and 0.5 ± 0.8% at D_2% , D_50% , and D_98% in physical dose and 0.8 ± 1.4%, 0.8 ± 1.2%, and 0.6 ± 1.1% in biologically effective dose (BED). For organs-at-risk, the dose differences of all evaluated dose-volume histogram points were within [-4.5%, 7.8%] and [-1.1 Gy, 3.5 Gy] in both physical dose and BED.

CONCLUSIONS

The geometric agreement metrics were within typically reported values and most average relative dose differences were within 1%. Thus, an MRI-only sCT-based approach is a promising alternative to the current clinical practice of the abdominal oART on MRL.

Synthetic CT in Carbon Ion Radiotherapy of the Abdominal Site

The generation of synthetic CT for carbon ion radiotherapy (CIRT) applications is challenging, since high accuracy is required in treatment planning and delivery, especially in an anatomical site as complex as the abdomen. Thirty-nine abdominal MRI-CT volume pairs were collected and a three-channel cGAN (accounting for air, bones, soft tissues) was used to generate sCTs. The network was tested on five held-out MRI volumes for two scenarios: (i) a CT-based segmentation of the MRI channels, to assess the quality of sCTs and (ii) an MRI manual segmentation, to simulate an MRI-only treatment scenario. The sCTs were evaluated by means of similarity metrics (e.g., mean absolute error, MAE) and geometrical criteria (e.g., dice coefficient). Recalculated CIRT plans were evaluated through dose volume histogram, gamma analysis and range shift analysis. The CT-based test set presented optimal MAE on bones (86.03 ± 10.76 HU), soft tissues (55.39 ± 3.41 HU) and air (54.42 ± 11.48 HU). Higher values were obtained from the MRI-only test set (MAE_BONE = 154.87 ± 22.90 HU). The global gamma pass rate reached 94.88 ± 4.9% with 3%/3 mm, while the range shift reached a median (IQR) of 0.98 (3.64) mm. The three-channel cGAN can generate acceptable abdominal sCTs and allow for CIRT dose recalculations comparable to the clinical plans.

Investigation of a water equivalent depth method for dosimetric accuracy evaluation of synthetic CT

PURPOSE

To provide a metric that reflects the dosimetric utility of the synthetic CT (sCT) and can be rapidly determined.

METHODS

Retrospective CT and atlas-based sCT of 62 (53 IMRT and 9 VMAT) prostate cancer patients were used. For image similarity measurements, the sCT and reference CT (rCT) were aligned using clinical registration parameters. Conventional image similarity metrics including the mean absolute error (MAE) and mean error (ME) were calculated. The water equivalent depth (WED) was automatically determined for each patient on the rCT and sCT as the distance from the skin surface to the treatment plan isocentre at 36 equidistant gantry angles, and the mean WED difference (ΔWED¯) between the two scans was calculated. Doses were calculated on each scan pair for the clinical plan in the treatment planning system. The image similarity measurements and ΔWED¯ were then compared to the isocentre dose difference (ΔD_iso) between the two scans.

RESULTS

While no particular relationship to dose was observed for the other image similarity metrics, the ME results showed a linear trend against ΔD_iso with R² = 0.6, and the 95 % prediction interval for ΔD_iso between -1.2 and 1 %. The ΔWED¯ results showed an improved linear trend (R² = 0.8) with a narrower 95 % prediction interval from -0.8 % to 0.8 %.

CONCLUSION

ΔWED¯ highly correlates with ΔD_iso for the reference and synthetic CT scans. This is easy to calculate automatically and does not require time-consuming dose calculations. Therefore, it can facilitate the process of developing and evaluating new sCT generation algorithms.

SARU: A self-attention ResUNet to generate synthetic CT images for MR-only BNCT treatment planning

PURPOSE

Despite the significant physical differences between magnetic resonance imaging (MRI) and computed tomography (CT), the high entropy of MRI data indicates the existence of a surjective transformation from MRI to CT image. However, there is no specific optimization of the network itself in previous MRI/CT translation works, resulting in mistakes in details such as the skull margin and cavity edge. These errors might have moderate effect on conventional radiotherapy, but for boron neutron capture therapy (BNCT), the skin dose will be a critical part of the dose composition. Thus, the purpose of this work is to create a self-attention network that could directly transfer MRI to synthetical computerized tomography (sCT) images with lower inaccuracy at the skin edge and examine the viability of magnetic resonance (MR)-guided BNCT.

METHODS

A retrospective analysis was undertaken on 104 patients with brain malignancies who had both CT and MRI as part of their radiation treatment plan. The CT images were deformably registered to the MRI. In the U-shaped generation network, we introduced spatial and channel attention modules, as well as a versatile "Attentional ResBlock," which reduce the parameters while maintaining high performance. We employed five-fold cross-validation to test all patients, compared the proposed network to those used in earlier studies, and used Monte Carlo software to simulate the BNCT process for dosimetric evaluation in test set.

RESULTS

Compared with UNet, Pix2Pix, and ResNet, the mean absolute error (MAE) of self-attention ResUNet (SARU) is reduced by 12.91, 17.48, and 9.50 HU, respectively. The "two one-sided tests" show no significant difference in dose-volume histogram (DVH) results. And for all tested cases, the average 2%/2 mm gamma index of UNet, ResNet, Pix2Pix, and SARU were 0.96 ± 0.03, 0.96 ± 0.03, 0.95 ± 0.03, and 0.98 ± 0.01, respectively. The error of skin dose from SARU is much less than the results from other methods.

CONCLUSIONS

We have developed a residual U-shape network with an attention mechanism to generate sCT images from MRI for BNCT treatment planning with lower MAE in six organs. There is no significant difference between the dose distribution calculated by sCT and real CT. This solution may greatly simplify the BNCT treatment planning process, lower the BNCT treatment dose, and minimize image feature mismatch.

Quality assurance for MRI-only radiation therapy: A voxel-wise population-based methodology for image and dose assessment of synthetic CT generation methods

The quality assurance of synthetic CT (sCT) is crucial for safe clinical transfer to an MRI-only radiotherapy planning workflow. The aim of this work is to propose a population-based process assessing local errors in the generation of sCTs and their impact on dose distribution. For the analysis to be anatomically meaningful, a customized interpatient registration method brought the population data to the same coordinate system. Then, the voxel-based process was applied on two sCT generation methods: a bulk-density method and a generative adversarial network. The CT and MRI pairs of 39 patients treated by radiotherapy for prostate cancer were used for sCT generation, and 26 of them with delineated structures were selected for analysis. Voxel-wise errors in sCT compared to CT were assessed for image intensities and dose calculation, and a population-based statistical test was applied to identify the regions where discrepancies were significant. The cumulative histograms of the mean absolute dose error per volume of tissue were computed to give a quantitative indication of the error for each generation method. Accurate interpatient registration was achieved, with mean Dice scores higher than 0.91 for all organs. The proposed method produces three-dimensional maps that precisely show the location of the major discrepancies for both sCT generation methods, highlighting the heterogeneity of image and dose errors for sCT generation methods from MRI across the pelvic anatomy. Hence, this method provides additional information that will assist with both sCT development and quality control for MRI-based planning radiotherapy.

Deep Learning for Synthetic CT from Bone MRI in the Head and Neck

BACKGROUND AND PURPOSE

Bone MR imaging techniques enable visualization of cortical bone without the need for ionizing radiation. Automated conversion of bone MR imaging to synthetic CT is highly desirable for downstream image processing and eventual clinical adoption. Given the complex anatomy and pathology of the head and neck, deep learning models are ideally suited for learning such mapping.

MATERIALS AND METHODS

This was a retrospective study of 39 pediatric and adult patients with bone MR imaging and CT examinations of the head and neck. For each patient, MR imaging and CT data sets were spatially coregistered using multiple-point affine transformation. Paired MR imaging and CT slices were generated for model training, using 4-fold cross-validation. We trained 3 different encoder-decoder models: Light_U-Net (2 million parameters) and VGG-16 U-Net (29 million parameters) without and with transfer learning. Loss functions included mean absolute error, mean squared error, and a weighted average. Performance metrics included Pearson R, mean absolute error, mean squared error, bone precision, and bone recall. We investigated model generalizability by training and validating across different conditions.

RESULTS

The Light_U-Net architecture quantitatively outperformed VGG-16 models. Mean absolute error loss resulted in higher bone precision, while mean squared error yielded higher bone recall. Performance metrics decreased when using training data captured only in a different environment but increased when local training data were augmented with those from different hospitals, vendors, or MR imaging techniques.

CONCLUSIONS

We have optimized a robust deep learning model for conversion of bone MR imaging to synthetic CT, which shows good performance and generalizability when trained on different hospitals, vendors, and MR imaging techniques. This approach shows promise for facilitating downstream image processing and adoption into clinical practice.

Possibilities and challenges when using synthetic computed tomography in an adaptive carbon-ion treatment workflow

BACKGROUND AND PURPOSE

Anatomical surveillance during ion-beam therapy is the basis for an effective tumor treatment and optimal organ at risk (OAR) sparing. Synthetic computed tomography (sCT) based on magnetic resonance imaging (MRI) can replace the X-ray based planning CT (X-rayCT) in photon radiotherapy and improve the workflow efficiency without additional imaging dose. The extension to carbon-ion radiotherapy is highly challenging; complex patient positioning, unique anatomical situations, distinct horizontal and vertical beam incidence directions, and limited training data are only few problems. This study gives insight into the possibilities and challenges of using sCTs in carbon-ion therapy.

MATERIALS AND METHODS

For head and neck patients immobilised with thermoplastic masks 30 clinically applied actively scanned carbon-ion treatment plans on 15 CTs comprising 60 beams were analyzed. Those treatment plans were re-calculated on MRI based sCTs which were created employing a 3D U-Net. Dose differences and carbon-ion spot displacements between sCT and X-rayCT were evaluated on a patient specific basis.

RESULTS

Spot displacement analysis showed a peak displacement by 0.2 cm caused by the immobilisation mask not measurable with the MRI. 95.7% of all spot displacements were located within 1 cm. For the clinical target volume (CTV) the median D_50% agreed within -0.2% (-1.3 to 1.4%), while the median D_0.01cc differed up to 4.2% (-1.3 to 25.3%) comparing the dose distribution on the X-rayCT and the sCT. OAR deviations depended strongly on the position and the dose gradient. For three patients no deterioration of the OAR parameters was observed. Other patients showed large deteriorations, e.g. for one patient D_2% of the chiasm differed by 28.1%.

CONCLUSION

The usage of sCTs opens several new questions, concluding that we are not ready yet for an MR-only workflow in carbon-ion therapy, as envisaged in photon therapy. Although omitting the X-rayCT seems unfavourable in the case of carbon-ion therapy, an sCT could be advantageous for monitoring, re-planning, and adaptation.

Target Definition in MR-Guided Adaptive Radiotherapy for Head and Neck Cancer

In recent years, MRI-guided radiotherapy (MRgRT) has taken an increasingly important position in image-guided radiotherapy (IGRT). Magnetic resonance imaging (MRI) offers superior soft tissue contrast in anatomical imaging compared to computed tomography (CT), but also provides functional and dynamic information with selected sequences. Due to these benefits, in current clinical practice, MRI is already used for target delineation and response assessment in patients with head and neck squamous cell carcinoma (HNSCC). Because of the close proximity of target areas and radiosensitive organs at risk (OARs) during HNSCC treatment, MRgRT could provide a more accurate treatment in which OARs receive less radiation dose. With the introduction of several new radiotherapy techniques (i.e., adaptive MRgRT, proton therapy, adaptive cone beam computed tomography (CBCT) RT, (daily) adaptive radiotherapy ensures radiation dose is accurately delivered to the target areas. With the integration of a daily adaptive workflow, interfraction changes have become visible, which allows regular and fast adaptation of target areas. In proton therapy, adaptation is even more important in order to obtain high quality dosimetry, due to its susceptibility for density differences in relation to the range uncertainty of the protons. The question is which adaptations during radiotherapy treatment are oncology safe and at the same time provide better sparing of OARs. For an optimal use of all these new tools there is an urgent need for an update of the target definitions in case of adaptive treatment for HNSCC. This review will provide current state of evidence regarding adaptive target definition using MR during radiotherapy for HNSCC. Additionally, future perspectives for adaptive MR-guided radiotherapy will be discussed.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Rates of MRI simulator utilisation in a tertiary cancer therapy centre

Magnetic resonance imaging (MRI) is increasingly being integrated into the radiation oncology workflow, due to its improved soft tissue contrast without additional exposure to ionising radiation. A review of MRI utilisation according to evidence based departmental guidelines was performed. Guideline utilisation rates were calculated to be 50% (true utilisation rate was 46%) of all new cancer patients treated with adjuvant or curative intent, excluding simple skin and breast cancer patients. Guideline utilisation rates were highest in the lower gastrointestinal and gynaecological subsites, with the lowest being in the upper gastrointestinal and thorax subsites. Head and neck (38% vs 45%) and CNS (46% vs 67%) cancers had the largest discrepancy between true and guideline utilisation rates due to unnamed reasons and non-contemporaneous diagnostic imaging respectively. This report outlines approximate MRI utilisation rates in a tertiary radiation oncology service and may help guide planning for future departments contemplating installation of an MRI simulator.

Personalized modeling to improve pseudo-CT images for magnetic resonance imaging-guided adaptive radiotherapy

PURPOSE

Magnetic resonance imaging-guided adaptive radiotherapy (MRIgART) greatly improves daily tumor localization and enables online re-planning to obtain maximum dosimetric benefits. However, accurately predicting patient-specific electron density maps for adaptive radiotherapy (ART) planning remains a challenge. Therefore, this study proposes a personalized modeling framework for generating pseudo-computed tomography (pCT) in MRIgART.

METHODS AND MATERIALS

Eighty-three patients who received MRIgART were included and CT simulations were performed on all the patients. Daily T2-weighted 1.5 T MRI was acquired using the Unity MR-linac for adaptive planning. Pairs of co-registered CT and daily MRI images of the randomly selected training set (68 patients) were inputted into a generative adversarial network (GAN) to establish a population model. The personalized model for each patient in the test set (15 patients) was acquired using model fine-tuning, which adopted the pair of the deformable-registered CT and the first daily MRI to fine-tune the population model. The pCT quality was quantitatively evaluated in the second and the last fractions with three metrics: intensity accuracy using mean absolute error (MAE); anatomical structure similarity using dice similarity coefficient (DSC); and dosimetric consistency using gamma-passing rate (GPR).

RESULTS

The image generation speed was 65 slices per second. For the last fractions, and for head-neck, thoracoabdominal, and pelvic cases, the average MAEs were 76.8 HU vs. 123.6 HU, 38.1 HU vs. 52.0 HU, and 29.5 HU vs. 39.7 HU, respectively. Furthermore, the average DSCs of bone were 0.92 vs. 0.80, 0.85 vs. 0.73, and 0.94 vs. 0.88; and the average GPRs (1%/1 mm) were 95.5% vs. 84.7%, 97.7% vs. 92.8%, and 95.5% vs. 88.7%, for personalized vs. population models, respectively. Results of the second fractions were similar.

CONCLUSIONS

The proposed personalized modeling framework remarkably improved pCT quality for multiple treatment sites and was well suited for the MRIgART clinical setting.

Unsupervised pseudo CT generation using heterogenous multicentric CT/MR images and CycleGAN: Dosimetric assessment for 3D conformal radiotherapy

PURPOSE

Absorbed dose calculation in magnetic resonance-guided radiation therapy (MRgRT) is commonly based on pseudo CT (pCT) images. This study investigated the feasibility of unsupervised pCT generation from MRI using a cycle generative adversarial network (CycleGAN) and a heterogenous multicentric dataset. A dosimetric analysis in three-dimensional conformal radiotherapy (3DCRT) planning was also performed.

MATERIAL AND METHODS

Overall, 87 T1-weighted and 102 T2-weighted MR images alongside with their corresponding computed tomography (CT) images of brain cancer patients from multiple centers were used. Initially, images underwent a number of preprocessing steps, including rigid registration, novel CT Masker, N4 bias field correction, resampling, resizing, and rescaling. To overcome the gradient vanishing problem, residual blocks and mean squared error (MSE) loss function were utilized in the generator and in both networks (generator and discriminator), respectively. The CycleGAN was trained and validated using 70 T1 and 80 T2 randomly selected patients in an unsupervised manner. The remaining patients were used as a holdout test set to report final evaluation metrics. The generated pCTs were validated in the context of 3DCRT.

RESULTS

The CycleGAN model using masked T2 images achieved better performance with a mean absolute error (MAE) of 61.87 ± 22.58 HU, peak signal to noise ratio (PSNR) of 27.05 ± 2.25 (dB), and structural similarity index metric (SSIM) of 0.84 ± 0.05 on the test dataset. T1-weighted MR images used for dosimetric assessment revealed a gamma index of 3%, 3 mm, 2%, 2 mm and 1%, 1 mm with acceptance criteria of 98.96% ± 1.1%, 95% ± 3.68%, 90.1% ± 6.05%, respectively. The DVH differences between CTs and pCTs were within 2%.

CONCLUSIONS

A promising pCT generation model capable of handling heterogenous multicenteric datasets was proposed. All MR sequences performed competitively with no significant difference in pCT generation. The proposed CT Masker proved promising in improving the model accuracy and robustness. There was no significant difference between using T1-weighted and T2-weighted MR images for pCT generation.

Deep learning for Dixon MRI-based attenuation correction in PET/MRI of head and neck cancer patients

Background

Quantitative whole-body PET/MRI relies on accurate patient-specific MRI-based attenuation correction (AC) of PET, which is a non-trivial challenge, especially for the anatomically complex head and neck region. We used a deep learning model developed for dose planning in radiation oncology to derive MRI-based attenuation maps of head and neck cancer patients and evaluated its performance on PET AC.

Methods

Eleven head and neck cancer patients, referred for radiotherapy, underwent CT followed by PET/MRI with acquisition of Dixon MRI. Both scans were performed in radiotherapy position. PET AC was performed with three different patient-specific attenuation maps derived from: (1) Dixon MRI using a deep learning network (PET_Deep). (2) Dixon MRI using the vendor-provided atlas-based method (PET_Atlas). (3) CT, serving as reference (PET_CT). We analyzed the effect of the MRI-based AC methods on PET quantification by assessing the average voxelwise error within the entire body, and the error as a function of distance to bone/air. The error in mean uptake within anatomical regions of interest and the tumor was also assessed.

Results

The average (± standard deviation) PET voxel error was 0.0 ± 11.4% for PET_Deep and −1.3 ± 21.8% for PET_Atlas. The error in mean PET uptake in bone/air was much lower for PET_Deep (−4%/12%) than for PET_Atlas (−15%/84%) and PET_Deep also demonstrated a more rapidly decreasing error with distance to bone/air affecting only the immediate surroundings (less than 1 cm). The regions with the largest error in mean uptake were those containing bone (mandible) and air (larynx) for both methods, and the error in tumor mean uptake was −0.6 ± 2.0% for PET_Deep and −3.5 ± 4.6% for PET_Atlas.

Conclusion

The deep learning network for deriving MRI-based attenuation maps of head and neck cancer patients demonstrated accurate AC and exceeded the performance of the vendor-provided atlas-based method both overall, on a lesion-level, and in vicinity of challenging regions such as bone and air.

Tomographic Ultrasound Imaging in the Diagnosis of Breast Tumors under the Guidance of Deep Learning Algorithms

This study was aimed to discuss the feasibility of distinguishing benign and malignant breast tumors under the tomographic ultrasound imaging (TUI) of deep learning algorithm. The deep learning algorithm was used to segment the images, and 120 patients with breast tumor were included in this study, all of whom underwent routine ultrasound examinations. Subsequently, TUI was used to assist in guiding the positioning, and the light scattering tomography system was used to further measure the lesions. A deep learning model was established to process the imaging results, and the pathological test results were undertaken as the gold standard for the efficiency of different imaging methods to diagnose the breast tumors. The results showed that, among 120 patients with breast tumor, 56 were benign lesions and 64 were malignant lesions. The average total amount of hemoglobin (HBT) of malignant lesions was significantly higher than that of benign lesions (P < 0.05). The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of TUI in the diagnosis of breast cancer were 90.4%, 75.6%, 81.4%, 84.7%, and 80.6%, respectively. The sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of ultrasound in the diagnosis of breast cancer were 81.7%, 64.9%, 70.5%, 75.9%, and 80.6%, respectively. In addition, for suspected breast malignant lesions, the combined application of ultrasound and tomography can increase the diagnostic specificity to 82.1% and the accuracy to 83.8%. Based on the above results, it was concluded that TUI combined with ultrasound had a significant effect on benign and malignant diagnosis of breast cancer and can significantly improve the specificity and accuracy of diagnosis. It also reflected that deep learning technology had a good auxiliary role in the examination of diseases and was worth the promotion of clinical application.

Synthetic Computed Tomography Generation from 0.35T Magnetic Resonance Images for Magnetic Resonance-Only Radiation Therapy Planning Using Perceptual Loss Models

PURPOSE

Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast, which makes it useful for delineating tumor and normal structures in radiation therapy planning, but MRI cannot readily provide electron density for dose calculation. Computed tomography (CT) is used but introduces registration uncertainty between MRI and CT. Previous studies have shown that synthetic CTs (sCTs) can be generated directly from MRI images with deep learning methods. However, mainly high-field MRI images have been validated. This study tested whether acceptable sCTs for MR-only radiation therapy planning can be synthesized using an integrated MR-guided linear accelerator at 0.35T, using MRI images and treatment plans in the liver region.

METHODS AND MATERIALS

Two models were investigated in this study: a convolutional neural network (Unet) with conventional mean square error (MSE) loss and a Unet using a secondary convolutional neural network for perceptual loss. A total of 37 cases were used in this study with 10-fold cross validation, and 37 treatment plans were generated and evaluated for target coverage and dose to organs at risk (OARs) in the MSE loss model, perceptual loss model, and original CT.

RESULTS

The sCTs predicted by the perceptual loss model had improved subjective visual quality compared with those predicted by the MSE loss model, but both were similar in mean absolute error (MAE), peak-signal-to-noise ratio (PSNR), and normalized cross-correlation (NCC). The MAE, PSNR, and NCC for the perceptual loss model were 35.64, 24.11, and 0.9539, respectively, and those for the MSE loss model were 35.67, 24.36, and 0.9566, respectively. No significant differences in target coverage and dose to OARs were found between the sCT predicted by the perceptual loss model or by the MSE model and the original CT image.

CONCLUSIONS

This study indicated that a Unet with both MSE loss and perceptual loss models can be used for generating sCT images from a 0.35T integrated MR linear accelerator.

Synthesis of pseudo-CT images from pelvic MRI images based on MD-CycleGAN model for radiotherapy

OBJECTIVE

A multi-discriminator-based cycle generative adversarial network (MD-CycleGAN) model was proposed to synthesize higher-quality pseudo-CT from MRI.

APPROACH

The MRI and CT images obtained at the simulation stage with cervical cancer were selected to train the model. The generator adopted the DenseNet as the main architecture. The local and global discriminators based on convolutional neural network jointly discriminated the authenticity of the input image data. In the testing phase, the model was verified by four-fold cross-validation method. In the prediction stage, the data were selected to evaluate the accuracy of the pseudo-CT in anatomy and dosimetry, and they were compared with the pseudo-CT synthesized by GAN with generator based on the architectures of ResNet, sU-Net, and FCN.

MAIN RESULTS

There are significant differences(P<0.05) in the four-fold-cross validation results on peak signal-to-noise ratio and structural similarity index metrics between the pseudo-CT obtained based on MD-CycleGAN and the ground truth CT (CTgt). The pseudo-CT synthesized by MD-CycleGAN had closer anatomical information to the CTgt with root mean square error of 47.83±2.92 HU and normalized mutual information value of 0.9014±0.0212 and mean absolute error value of 46.79±2.76 HU. The differences in dose distribution between the pseudo-CT obtained by MD-CycleGAN and the CTgt were minimal. The mean absolute dose errors of Dosemax, Dosemin and Dosemean based on the planning target volume were used to evaluate the dose uncertainty of the four pseudo-CT. The u-values of the Wilcoxon test were 55.407, 41.82 and 56.208, and the differences were statistically significant. The 2%/2 mm-based gamma pass rate (%) of the proposed method was 95.45±1.91, and the comparison methods (ResNet_GAN, sUnet_GAN and FCN_GAN) were 93.33±1.20, 89.64±1.63 and 87.31±1.94, respectively.

SIGNIFICANCE

The pseudo-CT obtained based on MD-CycleGAN have higher imaging quality and are closer to the CTgt in terms of anatomy and dosimetry than other GAN models.

Clinical Application of Artificial Intelligence in PET Imaging of Head and Neck Cancer

Applications of "artificial intelligence" (AI) have been exponentially expanding in health care. Readily accessible archives of enormous digital data in medical imaging have made radiology a leader in exploring and taking advantage of this technology. AI-assisted radiology has paved the way toward another level of precision in medicine. In this article, the authors aim to review current AI applications in PET imaging of head and neck cancers, beginning with radiomics and followed by deep learning in each section.

The Chinese Society of Clinical Oncology (CSCO) clinical guidelines for the diagnosis and treatment of nasopharyngeal carcinoma

Nasopharyngeal carcinoma (NPC) is a malignant epithelial tumor originating in the nasopharynx and has a high incidence in Southeast Asia and North Africa. To develop these comprehensive guidelines for the diagnosis and management of NPC, the Chinese Society of Clinical Oncology (CSCO) arranged a multi‐disciplinary team comprising of experts from all sub‐specialties of NPC to write, discuss, and revise the guidelines. Based on the findings of evidence‐based medicine in China and abroad, domestic experts have iteratively developed these guidelines to provide proper management of NPC. Overall, the guidelines describe the screening, clinical and pathological diagnosis, staging and risk assessment, therapies, and follow‐up of NPC, which aim to improve the management of NPC.

A practical methodology to improve the dosimetric accuracy of MR-based radiotherapy simulation for brain tumors

PURPOSE

To investigate the dosimetric accuracy of synthetic computed tomography (sCT) images generated by a clinically-ready voxel-based MRI simulation package, and to develop a simple and feasible method to improve the accuracy.

METHODS

20 patients with brain tumor were selected to undergo CT and MRI simulation. sCT images were generated by a clinical MRI simulation package. The discrepancy between planning CT and sCT in CT number and body contour were evaluated. To resolve the discrepancies, an sCT specific CT-relative electron density (RED) calibration curve was used, and a layer of pseudo-skin was created on the sCT. The dosimetric impact of these discrepancies, and the improvement brought about by the modifications, were evaluated by a planning study. Volumetric modulated arc therapy (VMAT) treatment plans for each patient were created and optimized on the planning CT, which were then transferred to the original sCT and the modified-sCT for dose re-calculation. Dosimetric comparisons and gamma analysis between the calculated doses in different images were performed.

RESULTS

The average gamma passing rate with 1%/1 mm criteria was only 70.8% for the comparison of dose distribution between planning CT and original sCT. The mean dose difference between the planning CT and the original sCT were -1.2% for PTV D₉₅ and -1.7% for PTV D_max, while the mean dose difference was within 0.7 Gy for all relevant OARs. After applying the modifications on the sCT, the average gamma passing rate was increased to 92.2%. Mean dose difference in PTV D₉₅ and D_max were reduced to -0.1% and -0.3% respectively. The mean dose difference was within 0.2 Gy for all OAR structures and no statistically significant difference were found.

CONCLUSIONS

The modified-sCT demonstrated improved dosimetric agreement with the planning CT. These results indicated the overall dosimetric accuracy and practicality of this improved MR-based treatment planning method.

Robustness and Generalizability of Deep Learning Synthetic Computed Tomography for Positron Emission Tomography/Magnetic Resonance Imaging-Based Radiation Therapy Planning of Patients With Head and Neck Cancer

Purpose

Radiotherapy planning based only on positron emission tomography/magnetic resonance imaging (PET/MRI) lacks computed tomography (CT) information required for dose calculations. In this study, a previously developed deep learning model for creating synthetic CT (sCT) from MRI in patients with head and neck cancer was evaluated in 2 scenarios: (1) using an independent external dataset, and (2) using a local dataset after an update of the model related to scanner software-induced changes to the input MRI.

Methods and Materials

Six patients from an external site and 17 patients from a local cohort were analyzed separately. Each patient underwent a CT and a PET/MRI with a Dixon MRI sequence over either one (external) or 2 (local) bed positions. For the external cohort, a previously developed deep learning model for deriving sCT from Dixon MRI was directly applied. For the local cohort, we adapted the model for an upgraded MRI acquisition using transfer learning and evaluated it in a leave-one-out process. The sCT mean absolute error for each patient was assessed. Radiotherapy dose plans based on sCT and CT were compared by assessing relevant absorbed dose differences in target volumes and organs at risk.

Results

The MAEs were 78 ± 13 HU and 76 ± 12 HU for the external and local cohort, respectively. For the external cohort, absorbed dose differences in target volumes were within ± 2.3% and within ± 1% in 95% of the cases. Differences in organs at risk were <2%. Similar results were obtained for the local cohort.

Conclusions

We have demonstrated a robust performance of a deep learning model for deriving sCT from MRI when applied to an independent external dataset. We updated the model to accommodate a larger axial field of view and software-induced changes to the input MRI. In both scenarios dose calculations based on sCT were similar to those of CT suggesting a robust and reliable method.

Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature review

PURPOSE

In radiotherapy, MRI is used for target volume and organs-at-risk delineation for its superior soft-tissue contrast as compared to CT imaging. However, MRI does not provide the electron density of tissue necessary for dose calculation. Several methods of synthetic-CT (sCT) generation from MRI data have been developed for radiotherapy dose calculation. This work reviewed deep learning (DL) sCT generation methods and their associated image and dose evaluation, in the context of MRI-based dose calculation.

METHODS

We searched the PubMed and ScienceDirect electronic databases from January 2010 to March 2021. For each paper, several items were screened and compiled in figures and tables.

RESULTS

This review included 57 studies. The DL methods were either generator-only based (45% of the reviewed studies), or generative adversarial network (GAN) architecture and its variants (55% of the reviewed studies). The brain and pelvis were the most commonly investigated anatomical localizations (39% and 28% of the reviewed studies, respectively), and more rarely, the head-and-neck (H&N) (15%), abdomen (10%), liver (5%) or breast (3%). All the studies performed an image evaluation of sCTs with a diversity of metrics, with only 36 studies performing dosimetric evaluations of sCT.

CONCLUSIONS

The median mean absolute errors were around 76 HU for the brain and H&N sCTs and 40 HU for the pelvis sCTs. For the brain, the mean dose difference between the sCT and the reference CT was <2%. For the H&N and pelvis, the mean dose difference was below 1% in most of the studies. Recent GAN architectures have advantages compared to generator-only, but no superiority was found in term of image or dose sCT uncertainties. Key challenges of DL-based sCT generation methods from MRI in radiotherapy is the management of movement for abdominal and thoracic localizations, the standardization of sCT evaluation, and the investigation of multicenter impacts.

Deep learning based synthetic-CT generation in radiotherapy and PET: A review

Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

Artificial Intelligence in magnetic Resonance guided Radiotherapy: Medical and physical considerations on state of art and future perspectives

Over the last years, technological innovation in Radiotherapy (RT) led to the introduction of Magnetic Resonance-guided RT (MRgRT) systems. Due to the higher soft tissue contrast compared to on-board CT-based systems, MRgRT is expected to significantly improve the treatment in many situations. MRgRT systems may extend the management of inter- and intra-fraction anatomical changes, offering the possibility of online adaptation of the dose distribution according to daily patient anatomy and to directly monitor tumor motion during treatment delivery by means of a continuous cine MR acquisition. Online adaptive treatments require a multidisciplinary and well-trained team, able to perform a series of operations in a safe, precise and fast manner while the patient is waiting on the treatment couch. Artificial Intelligence (AI) is expected to rapidly contribute to MRgRT, primarily by safely and efficiently automatising the various manual operations characterizing online adaptive treatments. Furthermore, AI is finding relevant applications in MRgRT in the fields of image segmentation, synthetic CT reconstruction, automatic (on-line) planning and the development of predictive models based on daily MRI. This review provides a comprehensive overview of the current AI integration in MRgRT from a medical physicist's perspective. Medical physicists are expected to be major actors in solving new tasks and in taking new responsibilities: their traditional role of guardians of the new technology implementation will change with increasing emphasis on the managing of AI tools, processes and advanced systems for imaging and data analysis, gradually replacing many repetitive manual tasks.

Cone beam computed tomography based image guidance and quality assessment of prostate cancer for magnetic resonance imaging-only radiotherapy in the pelvis

•

MRI-only IGRT accuracy is ≤2 mm as compared to CT but significant differences were observed.

•

MRI-only CBCT-based IGRT seems feasible but caution is advised.

•

The median absolute error (MeAE) for independent verification on the sCT quality is proposed.

•

A MeAE around 0.1 in mass density could call for sCT quality inspection.

Background and purpose

Radiotherapy (RT) based on magentic resonance imaging (MRI) only is currently used clinically in the pelvis. A synthetic computed tomography (sCT) is needed for dose planning. Here, we investigate the accuracy of cone beam CT (CBCT) based MRI-only image guided RT (IGRT) and sCT image quality.

Materials and methods

CT, MRI and CBCT scans of ten prostate cancer patients were included. The MRI was converted to a sCT using a multi-atlas approach. The sCT, CT and MR images were auto-matched with the CBCT on the bony anatomy. Paired sCT-CT and sCT-CBCT data were created. CT numbers were converted to relative electron (RED) and mass densities (DES) using a standard calibration curve for the CT and sCT. For the CBCT RED/DES conversion, a phantom and paired CT-CBCT population based calibration curve was used. For the latter, the CBCT numbers were averaged in 100 HU bins and the known RED/DES of the CT were assigned. The paired sCT-CT and sCT-CBCT data were averaged in bins of 10 HU or 0.01 RED/DES. The median absolute error (MeAE) between the sCT-CT and sCT-CBCT bins was calculated. Wilcoxon rank-sum tests were carried out for the IGRT and MeAE study.

Results

The mean sCT or MR IGRT difference from CT was ≤ 2 mm but significant differences were observed. A CBCT HU or phantom-based RED/DES MeAE did not estimate the sCT quality similar to a CT based MeAE but the CBCT population-based RED/DES MeAE did.

Conclusions

MRI-only CBCT-based IGRT seems feasible but caution is advised. A MeAE around 0.1 DES could call for sCT quality inspection.

Technical Challenges of Real-Time Adaptive MR-Guided Radiotherapy

In the past few years, radiotherapy (RT) has experienced a major technological innovation with the development of hybrid machines combining magnetic resonance (MR) imaging and linear accelerators. This new technology for MR-guided cancer treatment has the potential to revolutionize the field of adaptive RT due to the opportunity to provide high-resolution, real-time MR imaging before and during treatment application. However, from a technical point of view, several challenges remain which need to be tackled to ensure safe and robust real-time adaptive MR-guided RT delivery. In this manuscript, several technical challenges to MR-guided RT are discussed. Starting with magnetic field strength tradeoffs, the potential and limitations for purely MR-based RT workflows are discussed. Furthermore, the current status of real-time 3D MR imaging and its potential for real-time RT are summarized. Finally, the potential of quantitative MR imaging for future biological RT adaptation is highlighted.

Synthesizing CT images from MR images with deep learning: model generalization for different datasets through transfer learning

PURPOSE

Synthetic CT generation is the focus of many studies, however, only models on data with the same dataset were tested. Therefore, how well the trained model will work for data from different hospitals and MR protocols is still unknown. In this study, we addressed the model generalization problem for the MR-to-CT conversion task.

METHODS

Brain T2 MR and corresponding CT images were collected from one hospital and brain T1-FLAIR, T1-POST MR, and corresponding CT images were collected from another hospital. To investigate the model's generalizability ability, four potential solutions were proposed: source model, target model, combined model, and adapted model. All models were trained using the CycleGAN network. The source model was trained with a source domain dataset from scratch and tested with a target domain dataset. The target model was trained with a target domain dataset and tested with a target domain dataset. The combined model was trained with both source domain and target domain datasets, and tested with the target domain dataset. The adapted model used a transfer learning strategy to train a CycleGAN model with a source domain dataset and retrain the pre-trained model with a target domain dataset. MAE, RMSE, PSNR, and SSIM were used to quantitatively evaluate model performance on a target domain dataset.

RESULTS

The adapted model achieved best quantitative results of 74.56±8.61, 193.18±17.98, 28.30±0.83, and 0.84±0.01 for MAE, RMSE, PSNR, and SSIM using the T1-FLAIR dataset and 74.89±15.64, 195.73±31.29, 27.72±1.43, and 0.83±0.04 for MAE, RMSE, PSNR, and SSIM using the T1-POST dataset. The source model had the poorest performance.

CONCLUSIONS

This work indicates high generalization ability to generate synthetic CT images from small training datasets of MR images using pre-trained CycleGAN. The quantitative results of the test data, including different scanning protocols and different acquisition centers, indicated the proof of this concept.