1
|
Villegas F, Dal Bello R, Alvarez-Andres E, Dhont J, Janssen T, Milan L, Robert C, Salagean GAM, Tejedor N, Trnková P, Fusella M, Placidi L, Cusumano D. Challenges and opportunities in the development and clinical implementation of artificial intelligence based synthetic computed tomography for magnetic resonance only radiotherapy. Radiother Oncol 2024; 198:110387. [PMID: 38885905 DOI: 10.1016/j.radonc.2024.110387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 06/13/2024] [Accepted: 06/13/2024] [Indexed: 06/20/2024]
Abstract
Synthetic computed tomography (sCT) generated from magnetic resonance imaging (MRI) can serve as a substitute for planning CT in radiation therapy (RT), thereby removing registration uncertainties associated with multi-modality imaging pairing, reducing costs and patient radiation exposure. CE/FDA-approved sCT solutions are nowadays available for pelvis, brain, and head and neck, while more complex deep learning (DL) algorithms are under investigation for other anatomic sites. The main challenge in achieving a widespread clinical implementation of sCT lies in the absence of consensus on sCT commissioning and quality assurance (QA), resulting in variation of sCT approaches across different hospitals. To address this issue, a group of experts gathered at the ESTRO Physics Workshop 2022 to discuss the integration of sCT solutions into clinics and report the process and its outcomes. This position paper focuses on aspects of sCT development and commissioning, outlining key elements crucial for the safe implementation of an MRI-only RT workflow.
Collapse
Affiliation(s)
- Fernanda Villegas
- Department of Oncology-Pathology, Karolinska Institute, Solna, Sweden; Radiotherapy Physics and Engineering, Medical Radiation Physics and Nuclear Medicine, Karolinska University Hospital, Solna, Sweden
| | - Riccardo Dal Bello
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Emilie Alvarez-Andres
- OncoRay - National Center for Radiation Research in Oncology, Medical Faculty and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany; Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
| | - Jennifer Dhont
- Université libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (H.U.B), Institut Jules Bordet, Department of Medical Physics, Brussels, Belgium; Université Libre De Bruxelles (ULB), Radiophysics and MRI Physics Laboratory, Brussels, Belgium
| | - Tomas Janssen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Lisa Milan
- Medical Physics Unit, Imaging Institute of Southern Switzerland (IIMSI), Ente Ospedaliero Cantonale, Bellinzona, Switzerland
| | - Charlotte Robert
- UMR 1030 Molecular Radiotherapy and Therapeutic Innovations, ImmunoRadAI, Paris-Saclay University, Institut Gustave Roussy, Inserm, Villejuif, France; Department of Radiation Oncology, Gustave Roussy, Villejuif, France
| | - Ghizela-Ana-Maria Salagean
- Faculty of Physics, Babes-Bolyai University, Cluj-Napoca, Romania; Department of Radiation Oncology, TopMed Medical Centre, Targu Mures, Romania
| | - Natalia Tejedor
- Department of Medical Physics and Radiation Protection, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain
| | - Petra Trnková
- Department of Radiation Oncology, Medical University of Vienna, Vienna, Austria
| | - Marco Fusella
- Department of Radiation Oncology, Abano Terme Hospital, Italy
| | - Lorenzo Placidi
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Department of Diagnostic Imaging, Oncological Radiotherapy and Hematology, Rome, Italy.
| | - Davide Cusumano
- Mater Olbia Hospital, Strada Statale Orientale Sarda 125, Olbia, Sassari, Italy
| |
Collapse
|
2
|
Hu M, Qian J, Pan S, Li Y, Qiu RLJ, Yang X. Advancing medical imaging with language models: featuring a spotlight on ChatGPT. Phys Med Biol 2024; 69:10TR01. [PMID: 38537293 DOI: 10.1088/1361-6560/ad387d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 03/27/2024] [Indexed: 05/04/2024]
Abstract
This review paper aims to serve as a comprehensive guide and instructional resource for researchers seeking to effectively implement language models in medical imaging research. First, we presented the fundamental principles and evolution of language models, dedicating particular attention to large language models. We then reviewed the current literature on how language models are being used to improve medical imaging, emphasizing a range of applications such as image captioning, report generation, report classification, findings extraction, visual question response systems, interpretable diagnosis and so on. Notably, the capabilities of ChatGPT were spotlighted for researchers to explore its further applications. Furthermore, we covered the advantageous impacts of accurate and efficient language models in medical imaging analysis, such as the enhancement of clinical workflow efficiency, reduction of diagnostic errors, and assistance of clinicians in providing timely and accurate diagnoses. Overall, our goal is to have better integration of language models with medical imaging, thereby inspiring new ideas and innovations. It is our aspiration that this review can serve as a useful resource for researchers in this field, stimulating continued investigative and innovative pursuits of the application of language models in medical imaging.
Collapse
Affiliation(s)
- Mingzhe Hu
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
| | - Joshua Qian
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
| | - Shaoyan Pan
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
| | - Yuheng Li
- Department of Biomedical Engineering, Emory University, Atlanta, GA, United States of America
| | - Richard L J Qiu
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
| | - Xiaofeng Yang
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
- Department of Biomedical Engineering, Emory University, Atlanta, GA, United States of America
| |
Collapse
|
3
|
Sherwani MK, Gopalakrishnan S. A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy. FRONTIERS IN RADIOLOGY 2024; 4:1385742. [PMID: 38601888 PMCID: PMC11004271 DOI: 10.3389/fradi.2024.1385742] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Accepted: 03/11/2024] [Indexed: 04/12/2024]
Abstract
The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.
Collapse
Affiliation(s)
- Moiz Khan Sherwani
- Section for Evolutionary Hologenomics, Globe Institute, University of Copenhagen, Copenhagen, Denmark
| | | |
Collapse
|
4
|
Jahangiri L. Predicting Neuroblastoma Patient Risk Groups, Outcomes, and Treatment Response Using Machine Learning Methods: A Review. Med Sci (Basel) 2024; 12:5. [PMID: 38249081 PMCID: PMC10801560 DOI: 10.3390/medsci12010005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Revised: 12/28/2023] [Accepted: 01/03/2024] [Indexed: 01/23/2024] Open
Abstract
Neuroblastoma, a paediatric malignancy with high rates of cancer-related morbidity and mortality, is of significant interest to the field of paediatric cancers. High-risk NB tumours are usually metastatic and result in survival rates of less than 50%. Machine learning approaches have been applied to various neuroblastoma patient data to retrieve relevant clinical and biological information and develop predictive models. Given this background, this study will catalogue and summarise the literature that has used machine learning and statistical methods to analyse data such as multi-omics, histological sections, and medical images to make clinical predictions. Furthermore, the question will be turned on its head, and the use of machine learning to accurately stratify NB patients by risk groups and to predict outcomes, including survival and treatment response, will be summarised. Overall, this study aims to catalogue and summarise the important work conducted to date on the subject of expression-based predictor models and machine learning in neuroblastoma for risk stratification and patient outcomes including survival, and treatment response which may assist and direct future diagnostic and therapeutic efforts.
Collapse
Affiliation(s)
- Leila Jahangiri
- School of Science and Technology, Nottingham Trent University, Clifton Site, Nottingham NG11 8NS, UK;
- Division of Cellular and Molecular Pathology, Addenbrookes Hospital, University of Cambridge, Cambridge CB2 0QQ, UK
| |
Collapse
|
5
|
Wang H, Chen X, He L. A narrative review of radiomics and deep learning advances in neuroblastoma: updates and challenges. Pediatr Radiol 2023; 53:2742-2755. [PMID: 37945937 DOI: 10.1007/s00247-023-05792-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 10/09/2023] [Accepted: 10/10/2023] [Indexed: 11/12/2023]
Abstract
Neuroblastoma is an extremely heterogeneous tumor that commonly occurs in children. The diagnosis and treatment of this tumor pose considerable challenges due to its varied clinical presentations and intricate genetic aberrations. Presently, various imaging modalities, including computed tomography, magnetic resonance imaging, and positron emission tomography, are utilized to assess neuroblastoma. Nevertheless, these conventional imaging modalities have limitations in providing quantitative information for accurate diagnosis and prognosis. Radiomics, an emerging technique, can extract intricate medical imaging information that is imperceptible to the human eye and transform it into quantitative data. In conjunction with deep learning algorithms, radiomics holds great promise in complementing existing imaging modalities. The aim of this review is to showcase the potential of radiomics and deep learning advancements to enhance the diagnostic capabilities of current imaging modalities for neuroblastoma.
Collapse
Affiliation(s)
- Haoru Wang
- Department of Radiology, Children's Hospital of Chongqing Medical University, 136 Zhongshan Road 2, Yuzhong District, Chongqing, 400014, China
- National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing, China
- Chongqing Key Laboratory of Pediatrics, Chongqing, China
| | - Xin Chen
- Department of Radiology, Children's Hospital of Chongqing Medical University, 136 Zhongshan Road 2, Yuzhong District, Chongqing, 400014, China
- National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing, China
- Chongqing Key Laboratory of Pediatrics, Chongqing, China
| | - Ling He
- Department of Radiology, Children's Hospital of Chongqing Medical University, 136 Zhongshan Road 2, Yuzhong District, Chongqing, 400014, China.
- National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing, China.
- Chongqing Key Laboratory of Pediatrics, Chongqing, China.
| |
Collapse
|
6
|
Costa L, Schlosser TPC, Seevinck P, Kruyt MC, Castelein RM. The three-dimensional coupling mechanism in scoliosis and its consequences for correction. Spine Deform 2023; 11:1509-1516. [PMID: 37558820 PMCID: PMC10587017 DOI: 10.1007/s43390-023-00732-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 07/01/2023] [Indexed: 08/11/2023]
Abstract
INTRODUCTION In idiopathic scoliosis, the anterior spinal column has rotated away from the midline and has become longer through unloading and expansion of the intervertebral discs. Theoretically, extension of the spine in the sagittal plane should provide room for this longer anterior spinal column, allowing it to swing back towards the midline in the coronal and axial plane, thus reducing both the Cobb angle and the apical vertebral rotation. METHODS In this prospective experimental study, ten patients with primary thoracic adolescent idiopathic scoliosis (AIS) underwent MRI (BoneMRI and cVISTA sequences) in supine as well as in an extended position by placing a broad bolster, supporting both hemi-thoraces, under the scoliotic apex. Differences in T4-T12 kyphosis angle, coronal Cobb angle, vertebral rotation, as well as shape of the intervertebral disc and shape and position of the nucleus pulposus, were analysed and compared between the two positions. RESULTS Extension reduced T4-T12 thoracic kyphosis by 10° (p < 0.001), the coronal Cobb angle decreased by 9° (p < 0.001) and vertebral rotation by 4° (p = 0.036). The coronal wedge shape of the disc significantly normalized and the wedged and lateralized nucleus pulposus partially reduced to a more symmetrical position. CONCLUSION Simple extension of the scoliotic spine leads to a reduction of the deformity in the coronal and axial plane. The shape of the disc normalizes and the eccentric nucleus pulposus partially moves back to the midline.
Collapse
Affiliation(s)
- Lorenzo Costa
- Department of Orthopaedic Surgery, University Medical Centre Utrecht, Postbus 85500, G 05.228, 3508 GA Utrecht, The Netherlands
| | - Tom P. C. Schlosser
- Department of Orthopaedic Surgery, University Medical Centre Utrecht, Postbus 85500, G 05.228, 3508 GA Utrecht, The Netherlands
| | - Peter Seevinck
- Department of Imaging, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Moyo C. Kruyt
- Department of Orthopaedic Surgery, University Medical Centre Utrecht, Postbus 85500, G 05.228, 3508 GA Utrecht, The Netherlands
| | - René M. Castelein
- Department of Orthopaedic Surgery, University Medical Centre Utrecht, Postbus 85500, G 05.228, 3508 GA Utrecht, The Netherlands
| |
Collapse
|
7
|
Tian L, Lühr A. Proton range uncertainty caused by synthetic computed tomography generated with deep learning from pelvic magnetic resonance imaging. Acta Oncol 2023; 62:1461-1469. [PMID: 37703314 DOI: 10.1080/0284186x.2023.2256967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 09/04/2023] [Indexed: 09/15/2023]
Abstract
BACKGROUND In proton therapy, it is disputed whether synthetic computed tomography (sCT), derived from magnetic resonance imaging (MRI), permits accurate dose calculations. On the one hand, an MRI-only workflow could eliminate errors caused by, e.g., MRI-CT registration. On the other hand, the extra error would be induced due to an sCT generation model. This work investigated the systematic and random model error induced by sCT generation of a widely discussed deep learning model, pix2pix. MATERIAL AND METHODS An open-source image dataset of 19 patients with cancer in the pelvis was employed and split into 10, 5, and 4 for training, testing, and validation of the model, respectively. Proton pencil beams (200 MeV) were simulated on the real CT and generated sCT using the tool for particle simulation (TOPAS). Monte Carlo (MC) dropout was used for error estimation (50 random sCT samples). Systematic and random model errors were investigated for sCT generation and dose calculation on sCT. RESULTS For sCT generation, random model error near the edge of the body (∼200 HU) was higher than that within the body (∼100 HU near the bone edge and <10 HU in soft tissue). The mean absolute error (MAE) was 49 ± 5, 191 ± 23, and 503 ± 70 HU for the whole body, bone, and air in the patient, respectively. Random model errors of the proton range were small (<0.2 mm) for all spots and evenly distributed throughout the proton fields. Systematic errors of the proton range were -1.0(±2.2) mm and 0.4(±0.9)%, respectively, and were unevenly distributed within the proton fields. For 4.5% of the spots, large errors (>5 mm) were found, which may relate to MRI-CT mismatch due to, e.g., registration, MRI distortion anatomical changes, etc. CONCLUSION The sCT model was shown to be robust, i.e., had a low random model error. However, further investigation to reduce and even predict and manage systematic error is still needed for future MRI-only proton therapy.
Collapse
Affiliation(s)
- Liheng Tian
- Department of Physics, TU Dortmund University, Dortmund, Germany
| | - Armin Lühr
- Department of Physics, TU Dortmund University, Dortmund, Germany
| |
Collapse
|
8
|
Tsang B, Gupta A, Takahashi MS, Baffi H, Ola T, Doria AS. Applications of artificial intelligence in magnetic resonance imaging of primary pediatric cancers: a scoping review and CLAIM score assessment. Jpn J Radiol 2023; 41:1127-1147. [PMID: 37395982 DOI: 10.1007/s11604-023-01437-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 04/18/2023] [Indexed: 07/04/2023]
Abstract
PURPOSES To review the uses of AI for magnetic resonance (MR) imaging assessment of primary pediatric cancer and identify common literature topics and knowledge gaps. To assess the adherence of the existing literature to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines. MATERIALS AND METHODS A scoping literature search using MEDLINE, EMBASE and Cochrane databases was performed, including studies of > 10 subjects with a mean age of < 21 years. Relevant data were summarized into three categories based on AI application: detection, characterization, treatment and monitoring. Readers independently scored each study using CLAIM guidelines, and inter-rater reproducibility was assessed using intraclass correlation coefficients. RESULTS Twenty-one studies were included. The most common AI application for pediatric cancer MR imaging was pediatric tumor diagnosis and detection (13/21 [62%] studies). The most commonly studied tumor was posterior fossa tumors (14 [67%] studies). Knowledge gaps included a lack of research in AI-driven tumor staging (0/21 [0%] studies), imaging genomics (1/21 [5%] studies), and tumor segmentation (2/21 [10%] studies). Adherence to CLAIM guidelines was moderate in primary studies, with an average (range) of 55% (34%-73%) CLAIM items reported. Adherence has improved over time based on publication year. CONCLUSION The literature surrounding AI applications of MR imaging in pediatric cancers is limited. The existing literature shows moderate adherence to CLAIM guidelines, suggesting that better adherence is required for future studies.
Collapse
Affiliation(s)
- Brian Tsang
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Department of Diagnostic Imaging, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada
| | - Aaryan Gupta
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Department of Diagnostic Imaging, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada
| | - Marcelo Straus Takahashi
- Instituto de Radiologia do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InRad/HC-FMUSP), São Paulo, SP, Brazil
- Instituto da Criança do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (ICr/HC-FMUSP), São Paulo, SP, Brazil
- DasaInova, Diagnósticos da América SA (Dasa), São Paulo, SP, Brazil
| | | | - Tolulope Ola
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Department of Diagnostic Imaging, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada
| | - Andrea S Doria
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada.
- Department of Diagnostic Imaging, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada.
| |
Collapse
|
9
|
Dal Bello R, Lapaeva M, La Greca Saint-Esteven A, Wallimann P, Günther M, Konukoglu E, Andratschke N, Guckenberger M, Tanadini-Lang S. Patient-specific quality assurance strategies for synthetic computed tomography in magnetic resonance-only radiotherapy of the abdomen. Phys Imaging Radiat Oncol 2023; 27:100464. [PMID: 37497188 PMCID: PMC10366576 DOI: 10.1016/j.phro.2023.100464] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 06/21/2023] [Accepted: 06/22/2023] [Indexed: 07/28/2023] Open
Abstract
Background and purpose The superior tissue contrast of magnetic resonance (MR) compared to computed tomography (CT) led to an increasing interest towards MR-only radiotherapy. For the latter, the dose calculation should be performed on a synthetic CT (sCT). Patient-specific quality assurance (PSQA) methods have not been established yet and this study aimed to assess several software-based solutions. Materials and methods A retrospective study was performed on 20 patients treated at an MR-Linac, which were selected to evenly cover four subcategories: (i) standard, (ii) air pockets, (iii) lung and (iv) implant cases. The neural network (NN) CycleGAN was adopted to generate a reference sCT, which was then compared to four PSQA methods: (A) water override of body, (B) five tissue classes with bulk densities, (C) sCT generated by a separate NN (pix2pix) and (D) deformed CT. Results The evaluation of the dose endpoints demonstrated that while all methods A-D provided statistically equivalent results (p = 0.05) within the 2% level for the standard cases (i), only the methods C-D guaranteed the same result over the whole cohort. The bulk densities override was shown to be a valuable method in absence of lung tissue within the beam path. Conclusion The observations of this study suggested that the use of an additional sCT generated by a separate NN was an appropriate tool to perform PSQA of a sCT in an MR-only workflow at an MR-Linac. The time and dose endpoints requirements were respected, namely within 10 min and 2%.
Collapse
Affiliation(s)
- Riccardo Dal Bello
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Mariia Lapaeva
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
- Artificial Intelligence and Machine Learning Group, Department of Informatics, University of Zurich, Zurich, Switzerland
- Computer Vision Laboratory, ETH Zurich, Zurich, Switzerland
| | - Agustina La Greca Saint-Esteven
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
- Computer Vision Laboratory, ETH Zurich, Zurich, Switzerland
| | - Philipp Wallimann
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Manuel Günther
- Artificial Intelligence and Machine Learning Group, Department of Informatics, University of Zurich, Zurich, Switzerland
| | | | - Nicolaus Andratschke
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Matthias Guckenberger
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Stephanie Tanadini-Lang
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| |
Collapse
|
10
|
Dorsch S, Paul K, Beyer C, Karger CP, Jäkel O, Debus J, Klüter S. Quality assurance and temporal stability of a 1.5 T MRI scanner for MR-guided Photon and Particle Therapy. Z Med Phys 2023:S0939-3889(23)00046-6. [PMID: 37150727 DOI: 10.1016/j.zemedi.2023.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 03/12/2023] [Accepted: 04/04/2023] [Indexed: 05/09/2023]
Abstract
PURPOSE To describe performance measurements, adaptations and time stability over 20 months of a diagnostic MR scanner for integration into MR-guided photon and particle radiotherapy. MATERIAL AND METHODS For realization of MR-guided photon and particle therapy (MRgRT/MRgPT), a 1.5 T MR scanner was installed at the Heidelberg Ion Beam Therapy Center. To integrate MRI into the treatment process, a flat tabletop and dedicated coil holders for flex coils were used, which prevent deformation of the patient external contour and allow for the use of immobilization tools for reproducible positioning. The signal-to-noise ratio (SNR) was compared for the diagnostic and therapy-specific setup using the flat couch top and flexible coils for the a) head & neck and b) abdominal region as well as for different bandwidths and clinical pulse sequences. Additionally, a quality assurance (QA) protocol with monthly measurements of the ACR phantom and measurement of geometric distortions for a large field-of-view (FOV) was implemented to assess the imaging quality parameters of the device over the course of 20 months. RESULTS The SNR measurements showed a decreased SNR for the RT-specific as compared to the diagnostic setup of (a) 26% to 34% and (b) 11% to 33%. No significant bandwidth dependency for this ratio was found. The longitudinal assessment of the image quality parameters with the ACR and distortion phantom confirmed the long-term stability of the MRI device. CONCLUSION A diagnostic MRI was commissioned for use in MR-guided particle therapy. Using a radiotherapy specific setup, a high geometric accuracy and signal homogeneity was obtained after some adaptions and the measured parameters were shown to be stable over a period of 20 months.
Collapse
Affiliation(s)
- Stefan Dorsch
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), INF 280, 69120 Heidelberg, Germany; National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Department of Radiation Oncology, University Hospital Heidelberg, INF 400, 69120 Heidelberg, Germany.
| | - Katharina Paul
- National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Department of Radiation Oncology, University Hospital Heidelberg, INF 400, 69120 Heidelberg, Germany
| | - Cedric Beyer
- National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Department of Radiation Oncology, University Hospital Heidelberg, INF 400, 69120 Heidelberg, Germany
| | - Christian P Karger
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), INF 280, 69120 Heidelberg, Germany; National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany
| | - Oliver Jäkel
- Department of Medical Physics in Radiation Oncology, German Cancer Research Center (DKFZ), INF 280, 69120 Heidelberg, Germany; National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Heidelberg Ion-Beam Therapy Center (HIT), Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Jürgen Debus
- National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Department of Radiation Oncology, University Hospital Heidelberg, INF 400, 69120 Heidelberg, Germany; Heidelberg Ion-Beam Therapy Center (HIT), Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany; Clinical Cooperation Unit Radiation Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany; Core center Heidelberg, German Cancer Consortium (DKTK), Heidelberg, Germany
| | - Sebastian Klüter
- National Center for Radiation Research in Oncology (NCRO), Heidelberg Institute for Radiation Oncology (HIRO), Heidelberg, Germany; Department of Radiation Oncology, University Hospital Heidelberg, INF 400, 69120 Heidelberg, Germany.
| |
Collapse
|
11
|
Szmul A, Taylor S, Lim P, Cantwell J, Moreira I, Zhang Y, D’Souza D, Moinuddin S, Gaze MN, Gains J, Veiga C. Deep learning based synthetic CT from cone beam CT generation for abdominal paediatric radiotherapy. Phys Med Biol 2023; 68:105006. [PMID: 36996837 PMCID: PMC10160738 DOI: 10.1088/1361-6560/acc921] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 03/13/2023] [Accepted: 03/30/2023] [Indexed: 04/01/2023]
Abstract
Objective. Adaptive radiotherapy workflows require images with the quality of computed tomography (CT) for re-calculation and re-optimisation of radiation doses. In this work we aim to improve the quality of on-board cone beam CT (CBCT) images for dose calculation using deep learning.Approach. We propose a novel framework for CBCT-to-CT synthesis using cycle-consistent Generative Adversarial Networks (cycleGANs). The framework was tailored for paediatric abdominal patients, a challenging application due to the inter-fractional variability in bowel filling and small patient numbers. We introduced to the networks the concept of global residuals only learning and modified the cycleGAN loss function to explicitly promote structural consistency between source and synthetic images. Finally, to compensate for the anatomical variability and address the difficulties in collecting large datasets in the paediatric population, we applied a smart 2D slice selection based on the common field-of-view (abdomen) to our imaging dataset. This acted as a weakly paired data approach that allowed us to take advantage of scans from patients treated for a variety of malignancies (thoracic-abdominal-pelvic) for training purposes. We first optimised the proposed framework and benchmarked its performance on a development dataset. Later, a comprehensive quantitative evaluation was performed on an unseen dataset, which included calculating global image similarity metrics, segmentation-based measures and proton therapy-specific metrics.Main results. We found improved performance for our proposed method, compared to a baseline cycleGAN implementation, on image-similarity metrics such as Mean Absolute Error calculated for a matched virtual CT (55.0 ± 16.6 HU proposed versus 58.9 ± 16.8 HU baseline). There was also a higher level of structural agreement for gastrointestinal gas between source and synthetic images measured using the dice similarity coefficient (0.872 ± 0.053 proposed versus 0.846 ± 0.052 baseline). Differences found in water-equivalent thickness metrics were also smaller for our method (3.3 ± 2.4% proposed versus 3.7 ± 2.8% baseline).Significance. Our findings indicate that our innovations to the cycleGAN framework improved the quality and structure consistency of the synthetic CTs generated.
Collapse
Affiliation(s)
- Adam Szmul
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London, United Kingdom
| | - Sabrina Taylor
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Pei Lim
- Department of Oncology, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Jessica Cantwell
- Radiotherapy, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Isabel Moreira
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Ying Zhang
- Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Derek D’Souza
- Radiotherapy Physics Services, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Syed Moinuddin
- Radiotherapy, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Mark N. Gaze
- Department of Oncology, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Jennifer Gains
- Department of Oncology, University College London Hospitals NHS Foundation Trust, London, United Kingdom
| | - Catarina Veiga
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| |
Collapse
|
12
|
Nousiainen K, Santurio GV, Lundahl N, Cronholm R, Siversson C, Edmund JM. Evaluation of MRI-only based online adaptive radiotherapy of abdominal region on MR-linac. J Appl Clin Med Phys 2023; 24:e13838. [PMID: 36347050 PMCID: PMC10018672 DOI: 10.1002/acm2.13838] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2021] [Revised: 09/30/2022] [Accepted: 10/18/2022] [Indexed: 11/09/2022] Open
Abstract
PURPOSE A hybrid magnetic resonance linear accelerator (MRL) can perform magnetic resonance imaging (MRI) with high soft-tissue contrast to be used for online adaptive radiotherapy (oART). To obtain electron densities needed for the oART dose calculation, a computed tomography (CT) is often deformably registered to MRI. Our aim was to evaluate an MRI-only based synthetic CT (sCT) generation as an alternative to the deformed CT (dCT)-based oART in the abdominal region. METHODS The study data consisted of 57 patients who were treated on a 0.35 T MRL system mainly for abdominal tumors. Simulation MRI-CT pairs of 43 patients were used for training and validation of a prototype convolutional neural network sCT-generation algorithm, based on HighRes3DNet, for the abdominal region. For remaining test patients, sCT images were produced from simulation MRIs and daily MRIs. The dCT-based plans were re-calculated on sCT with identical calculation parameters. The sCT and dCT were compared in terms of geometric agreement and calculated dose. RESULTS The mean and one standard deviation of the geometric agreement metrics over dCT-sCT-pairs were: mean error of 8 ± 10 HU, mean absolute error of 49 ± 10 HU, and Dice similarity coefficient of 55 ± 12%, 60 ± 5%, and 82 ± 15% for bone, fat, and lung tissues, respectively. The dose differences between the sCT and dCT-based dose for planning target volumes were 0.5 ± 0.9%, 0.6 ± 0.8%, and 0.5 ± 0.8% at D2% , D50% , and D98% in physical dose and 0.8 ± 1.4%, 0.8 ± 1.2%, and 0.6 ± 1.1% in biologically effective dose (BED). For organs-at-risk, the dose differences of all evaluated dose-volume histogram points were within [-4.5%, 7.8%] and [-1.1 Gy, 3.5 Gy] in both physical dose and BED. CONCLUSIONS The geometric agreement metrics were within typically reported values and most average relative dose differences were within 1%. Thus, an MRI-only sCT-based approach is a promising alternative to the current clinical practice of the abdominal oART on MRL.
Collapse
Affiliation(s)
- Katri Nousiainen
- Department of Physics, University of Helsinki, Helsinki, Finland.,HUS Cancer Center, Helsinki University Hospital and University of Helsinki, Helsinki, Finland.,HUS Medical Imaging Center, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Grichar Valdes Santurio
- Department of Oncology, Radiotherapy Research Unit, Herlev and Gentofte Hospital, Copenhagen University, Herlev, Denmark
| | | | | | | | - Jens M Edmund
- Department of Oncology, Radiotherapy Research Unit, Herlev and Gentofte Hospital, Copenhagen University, Herlev, Denmark.,Nils Bohr Institute, Copenhagen University, Copenhagen, Denmark
| |
Collapse
|
13
|
Parrella G, Vai A, Nakas A, Garau N, Meschini G, Camagni F, Molinelli S, Barcellini A, Pella A, Ciocca M, Vitolo V, Orlandi E, Paganelli C, Baroni G. Synthetic CT in Carbon Ion Radiotherapy of the Abdominal Site. Bioengineering (Basel) 2023; 10:bioengineering10020250. [PMID: 36829745 PMCID: PMC9951997 DOI: 10.3390/bioengineering10020250] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 02/06/2023] [Accepted: 02/09/2023] [Indexed: 02/17/2023] Open
Abstract
The generation of synthetic CT for carbon ion radiotherapy (CIRT) applications is challenging, since high accuracy is required in treatment planning and delivery, especially in an anatomical site as complex as the abdomen. Thirty-nine abdominal MRI-CT volume pairs were collected and a three-channel cGAN (accounting for air, bones, soft tissues) was used to generate sCTs. The network was tested on five held-out MRI volumes for two scenarios: (i) a CT-based segmentation of the MRI channels, to assess the quality of sCTs and (ii) an MRI manual segmentation, to simulate an MRI-only treatment scenario. The sCTs were evaluated by means of similarity metrics (e.g., mean absolute error, MAE) and geometrical criteria (e.g., dice coefficient). Recalculated CIRT plans were evaluated through dose volume histogram, gamma analysis and range shift analysis. The CT-based test set presented optimal MAE on bones (86.03 ± 10.76 HU), soft tissues (55.39 ± 3.41 HU) and air (54.42 ± 11.48 HU). Higher values were obtained from the MRI-only test set (MAEBONE = 154.87 ± 22.90 HU). The global gamma pass rate reached 94.88 ± 4.9% with 3%/3 mm, while the range shift reached a median (IQR) of 0.98 (3.64) mm. The three-channel cGAN can generate acceptable abdominal sCTs and allow for CIRT dose recalculations comparable to the clinical plans.
Collapse
Affiliation(s)
- Giovanni Parrella
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
- Correspondence: ; Tel.: +39-02-2399-18-9022
| | - Alessandro Vai
- Medical Physics Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Anestis Nakas
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Noemi Garau
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Giorgia Meschini
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Francesca Camagni
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Silvia Molinelli
- Medical Physics Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Amelia Barcellini
- Radiotherapy Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
- Department of Internal Medicine and Medical Therapy, University of Pavia, 27100 Pavia, Italy
| | - Andrea Pella
- Bioengineering Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Mario Ciocca
- Medical Physics Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Viviana Vitolo
- Radiotherapy Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Ester Orlandi
- Clinical Unit, National Center of Oncological Hadrontherapy (CNAO), Strada Campeggi, 53, 27100 Pavia, Italy
| | - Chiara Paganelli
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| | - Guido Baroni
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milan, Italy
| |
Collapse
|
14
|
Zhong L, Chen Z, Shu H, Zheng Y, Zhang Y, Wu Y, Feng Q, Li Y, Yang W. QACL: Quartet attention aware closed-loop learning for abdominal MR-to-CT synthesis via simultaneous registration. Med Image Anal 2023; 83:102692. [PMID: 36442293 DOI: 10.1016/j.media.2022.102692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 10/27/2022] [Accepted: 11/09/2022] [Indexed: 11/18/2022]
Abstract
Synthesis of computed tomography (CT) images from magnetic resonance (MR) images is an important task to overcome the lack of electron density information in MR-only radiotherapy treatment planning (RTP). Some innovative methods have been proposed for abdominal MR-to-CT synthesis. However, it is still challenging due to the large misalignment between preprocessed abdominal MR and CT images and the insufficient feature information learned by models. Although several studies have used the MR-to-CT synthesis to alleviate the difficulty of multi-modal registration, this misalignment remains unsolved when training the MR-to-CT synthesis model. In this paper, we propose an end-to-end quartet attention aware closed-loop learning (QACL) framework for MR-to-CT synthesis via simultaneous registration. Specifically, the proposed quartet attention generator and mono-modal registration network form a closed-loop to improve the performance of MR-to-CT synthesis via simultaneous registration. In particular, a quartet-attention mechanism is developed to enlarge the receptive fields in networks to extract the long-range and cross-dimension spatial dependencies. Experimental results on two independent abdominal datasets demonstrate that our QACL achieves impressive results with MAE of 55.30±10.59 HU, PSNR of 22.85±1.43 dB, and SSIM of 0.83±0.04 for synthesis, and with Dice of 0.799±0.129 for registration. The proposed QACL outperforms the state-of-the-art MR-to-CT synthesis and multi-modal registration methods.
Collapse
Affiliation(s)
- Liming Zhong
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China
| | - Zeli Chen
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China
| | - Hai Shu
- Department of Biostatistics, School of Global Public Health, New York University, New York, NY, 10003, United States
| | - Yikai Zheng
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China
| | - Yiwen Zhang
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China
| | - Yuankui Wu
- Department of Medical Imaging Center, Nanfang Hospital, Southern Medical University, Guangzhou, 510515, China
| | - Qianjin Feng
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China
| | - Yin Li
- Department of Information, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510655, China.
| | - Wei Yang
- School of Biomedical Engineering, Southern Medical University, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, 510515, China.
| |
Collapse
|
15
|
Zhao B, Cheng T, Zhang X, Wang J, Zhu H, Zhao R, Li D, Zhang Z, Yu G. CT synthesis from MR in the pelvic area using Residual Transformer Conditional GAN. Comput Med Imaging Graph 2023; 103:102150. [PMID: 36493595 DOI: 10.1016/j.compmedimag.2022.102150] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 11/15/2022] [Accepted: 11/27/2022] [Indexed: 12/03/2022]
Abstract
Magnetic resonance (MR) image-guided radiation therapy is a hot topic in current radiation therapy research, which relies on MR to generate synthetic computed tomography (SCT) images for radiation therapy. Convolution-based generative adversarial networks (GAN) have achieved promising results in synthesizing CT from MR since the introduction of deep learning techniques. However, due to the local limitations of pure convolutional neural networks (CNN) structure and the local mismatch between paired MR and CT images, particularly in pelvic soft tissue, the performance of GAN in synthesizing CT from MR requires further improvement. In this paper, we propose a new GAN called Residual Transformer Conditional GAN (RTCGAN), which exploits the advantages of CNN in local texture details and Transformer in global correlation to extract multi-level features from MR and CT images. Furthermore, the feature reconstruction loss is used to further constrain the image potential features, reducing over-smoothing and local distortion of the SCT. The experiments show that RTCGAN is visually closer to the reference CT (RCT) image and achieves desirable results on local mismatch tissues. In the quantitative evaluation, the MAE, SSIM, and PSNR of RTCGAN are 45.05 HU, 0.9105, and 28.31 dB, respectively. All of them outperform other comparison methods, such as deep convolutional neural networks (DCNN), Pix2Pix, Attention-UNet, WPD-DAGAN, and HDL.
Collapse
Affiliation(s)
- Bo Zhao
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong 250358, China
| | - Tingting Cheng
- Department of General practice, Xiangya Hospital, Central South University, Changsha 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Changsha 410008, China
| | - Xueren Zhang
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong 250358, China
| | - Jingjing Wang
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong 250358, China
| | - Hong Zhu
- Department of Radiation Oncology, Xiangya Hospital, Central South University, Changsha 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Changsha 410008, China
| | - Rongchang Zhao
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Dengwang Li
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong 250358, China
| | - Zijian Zhang
- Department of Radiation Oncology, Xiangya Hospital, Central South University, Changsha 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Changsha 410008, China
| | - Gang Yu
- Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, Shandong 250358, China
| |
Collapse
|
16
|
Garcia Hernandez A, Fau P, Wojak J, Mailleux H, Benkreira M, Rapacchi S, Adel M. Synthetic computed tomography generation for abdominal adaptive radiotherapy using low-field magnetic resonance imaging. Phys Imaging Radiat Oncol 2023; 25:100425. [PMID: 36896334 PMCID: PMC9988674 DOI: 10.1016/j.phro.2023.100425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 02/12/2023] [Accepted: 02/17/2023] [Indexed: 02/24/2023] Open
Abstract
Background and Purpose Magnetic Resonance guided Radiotherapy (MRgRT) still needs the acquisition of Computed Tomography (CT) images and co-registration between CT and Magnetic Resonance Imaging (MRI). The generation of synthetic CT (sCT) images from the MR data can overcome this limitation. In this study we aim to propose a Deep Learning (DL) based approach for sCT image generation for abdominal Radiotherapy using low field MR images. Materials and methods CT and MR images were collected from 76 patients treated on abdominal sites. U-Net and conditional Generative Adversarial Network (cGAN) architectures were used to generate sCT images. Additionally, sCT images composed of only six bulk densities were generated with the aim of having a Simplified sCT.Radiotherapy plans calculated using the generated images were compared to the original plan in terms of gamma pass rate and Dose Volume Histogram (DVH) parameters. Results sCT images were generated in 2 s and 2.5 s with U-Net and cGAN architectures respectively.Gamma pass rates for 2%/2mm and 3%/3mm criteria were 91% and 95% respectively. Dose differences within 1% for DVH parameters on the target volume and organs at risk were obtained. Conclusion U-Net and cGAN architectures are able to generate abdominal sCT images fast and accurately from low field MRI.
Collapse
Affiliation(s)
- Armando Garcia Hernandez
- Aix Marseille Univ, CNRS, Centrale Marseille, Institut Fresnel, Marseille, France
- Corresponding author.
| | - Pierre Fau
- Institut Paoli-Calmettes, Bouches du Rhône, Marseille, France
| | - Julien Wojak
- Aix Marseille Univ, CNRS, Centrale Marseille, Institut Fresnel, Marseille, France
| | - Hugues Mailleux
- Institut Paoli-Calmettes, Bouches du Rhône, Marseille, France
| | | | | | - Mouloud Adel
- Aix Marseille Univ, CNRS, Centrale Marseille, Institut Fresnel, Marseille, France
| |
Collapse
|
17
|
Lapaeva M, La Greca Saint-Esteven A, Wallimann P, Günther M, Konukoglu E, Andratschke N, Guckenberger M, Tanadini-Lang S, Dal Bello R. Synthetic computed tomographies for low-field magnetic resonance-guided radiotherapy in the abdomen. Phys Imaging Radiat Oncol 2022; 24:173-179. [DOI: 10.1016/j.phro.2022.11.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 11/13/2022] [Accepted: 11/23/2022] [Indexed: 11/29/2022] Open
|
18
|
Chen S, Peng Y, Qin A, Liu Y, Zhao C, Deng X, Deraniyagala R, Stevens C, Ding X. MR-based synthetic CT image for intensity-modulated proton treatment planning of nasopharyngeal carcinoma patients. Acta Oncol 2022; 61:1417-1424. [DOI: 10.1080/0284186x.2022.2140017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Affiliation(s)
- Shupeng Chen
- Department of Radiation Oncology, William Beaumont Hospital, Royal Oak, MI, USA
| | - Yinglin Peng
- Department of Radiation Oncology, Sun Yat-Sen University, Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, PR China
- School of Biomedical Engineering, Sun Yat-Sen University, Guangzhou, PR China
| | - An Qin
- Department of Radiation Oncology, William Beaumont Hospital, Royal Oak, MI, USA
| | - Yimei Liu
- Department of Radiation Oncology, Sun Yat-Sen University, Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, PR China
| | - Chong Zhao
- Department of Radiation Oncology, Sun Yat-Sen University, Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, PR China
| | - Xiaowu Deng
- Department of Radiation Oncology, Sun Yat-Sen University, Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, PR China
| | - Rohan Deraniyagala
- Department of Radiation Oncology, William Beaumont Hospital, Royal Oak, MI, USA
| | - Craig Stevens
- Department of Radiation Oncology, William Beaumont Hospital, Royal Oak, MI, USA
| | - Xuanfeng Ding
- Department of Radiation Oncology, William Beaumont Hospital, Royal Oak, MI, USA
| |
Collapse
|
19
|
Gurney-Champion OJ, Landry G, Redalen KR, Thorwarth D. Potential of Deep Learning in Quantitative Magnetic Resonance Imaging for Personalized Radiotherapy. Semin Radiat Oncol 2022; 32:377-388. [DOI: 10.1016/j.semradonc.2022.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
20
|
Zhang X, He P, Li Y, Liu X, Ma Y, Shen G, Dai Z, Zhang H, Chen W, Li Q. DR-only Carbon-ion radiotherapy treatment planning via deep learning. Phys Med 2022; 100:120-128. [DOI: 10.1016/j.ejmp.2022.06.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 05/31/2022] [Accepted: 06/29/2022] [Indexed: 10/17/2022] Open
|
21
|
A Survey on Deep Learning for Precision Oncology. Diagnostics (Basel) 2022; 12:diagnostics12061489. [PMID: 35741298 PMCID: PMC9222056 DOI: 10.3390/diagnostics12061489] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 06/14/2022] [Accepted: 06/14/2022] [Indexed: 12/27/2022] Open
Abstract
Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.
Collapse
|
22
|
van der Kolk BBY, Slotman DJ, Nijholt IM, van Osch JA, Snoeijink TJ, Podlogar M, A.A.M. van Hasselt B, Boelhouwers HJ, van Stralen M, Seevinck PR, Schep NW, Maas M, Boomsma MF. Bone visualization of the cervical spine with deep learning-based synthetic CT compared to conventional CT: a single-center noninferiority study on image quality. Eur J Radiol 2022; 154:110414. [DOI: 10.1016/j.ejrad.2022.110414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2022] [Accepted: 06/13/2022] [Indexed: 11/03/2022]
|
23
|
Pediatric radiotherapy for thoracic and abdominal targets: organ motion, reported margin sizes, and delineation variations – a systematic review. Radiother Oncol 2022; 173:134-145. [DOI: 10.1016/j.radonc.2022.05.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 05/09/2022] [Accepted: 05/26/2022] [Indexed: 11/21/2022]
|
24
|
Sheikh K, Liu D, Li H, Acharya S, Ladra MM, Hrinivich WT. Dosimetric evaluation of cone-beam CT-based synthetic CTs in pediatric patients undergoing intensity-modulated proton therapy. J Appl Clin Med Phys 2022; 23:e13604. [PMID: 35413144 PMCID: PMC9194971 DOI: 10.1002/acm2.13604] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Revised: 03/10/2022] [Accepted: 03/21/2022] [Indexed: 11/07/2022] Open
Abstract
PURPOSE To evaluate dosimetric changes detected using synthetic computed tomography (sCT) derived from online cone-beam CTs (CBCT) in pediatric patients treated using intensity-modulated proton therapy (IMPT). METHODS Ten pediatric patients undergoing IMPT and aligned daily using proton gantry-mounted CBCT were identified for retrospective analysis with treated anatomical sites fully encompassed in the CBCT field of view. Dates were identified when the patient received both a CBCT and a quality assurance CT (qCT) for routine dosimetric evaluation. sCTs were generated based on a deformable registration between the initial plan CT (pCT) and CBCT. The clinical IMPT plans were re-computed on the same day qCT and sCT, and dosimetric changes due to tissue change or response from the initial plan were computed using each image. Linear regression analysis was performed to determine the correlation between dosimetric changes detected using the qCT and the sCT. Gamma analysis was also used to compare the dose distributions computed on the qCT and sCT. RESULTS The correlation coefficients (p-values) between qCTs and sCTs for changes detected in target coverage, overall maximum dose, and organ at risk dose were 0.97 (< .001), 0.84 (.002) and 0.91 (< .001), respectively. Mean ± SD gamma pass rates of the sCT-based dose compared to the qCT-based dose at 3%/3 mm, 3%/2 mm, and 2%/2 mm criteria were 96.5%±4.5%, 93.2%±6.3%, and 91.3%±7.8%, respectively. Pass rates tended to be lower for targets near lung. CONCLUSION While insufficient for re-planning, sCTs provide approximate dosimetry without administering additional imaging dose in pediatric patients undergoing IMPT. Dosimetric changes detected using sCTs are correlated with changes detected using clinically-standard qCTs; however, residual differences in dosimetry remain a limitation. Further improvements in sCT image quality may both improve online dosimetric evaluation and reduce imaging dose for pediatric patients by reducing the need for routine qCTs.
Collapse
Affiliation(s)
- Khadija Sheikh
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Dezhi Liu
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Heng Li
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Sahaja Acharya
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Matthew M Ladra
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - William T Hrinivich
- Department of Radiation Oncology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| |
Collapse
|
25
|
Wang C, Uh J, Patni T, Merchant T, Li Y, Hua CH, Acharya S. Toward MR-only proton therapy planning for pediatric brain tumors: synthesis of relative proton stopping power images with multiple sequence MRI and development of an online quality assurance tool. Med Phys 2022; 49:1559-1570. [PMID: 35075670 DOI: 10.1002/mp.15479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Revised: 12/23/2021] [Accepted: 01/11/2022] [Indexed: 11/11/2022] Open
Abstract
PURPOSE To generate synthetic relative proton-stopping-power (sRPSP) images from MRI sequence(s) and develop an online quality assurance (QA) tool for sRPSP to facilitate safe integration of MR-only proton planning into clinical practice. MATERIALS AND METHODS Planning CT and MR images of 195 pediatric brain tumor patients were utilized (training: 150, testing: 45). Seventeen consistent-cycle Generative Adversarial Network (ccGAN) models were trained separately using paired CT-converted RPSP and MRI datasets to transform a subject's MRI into sRPSP. T1-weighted (T1W), T2-weighted (T2W), and FLAIR MRI were permutated to form 17 combinations, with or without preprocessing, for determining the optimal training sequence(s). For evaluation, sRPSP images were converted to synthetic CT (sCT) and compared to the real CT in terms of mean absolute error (MAE) in HU. For QA, sCT was deformed and compared to a reference template built from training dataset to produce a flag map, highlighting pixels that deviate by >100 HU and fall outside the mean ± standard deviation reference intensity. The gamma intensity analysis (10%/3mm) of the deformed sCT against the QA template on the intensity difference was investigated as a surrogate of sCT accuracy. RESULTS The sRPSP images generated from a single T1W or T2W sequence outperformed that generated from multi-MRI sequences in terms of MAE (all P<0.05). Preprocessing with N4 bias and histogram matching reduced MAE of T2W MRI-based sCT (54±21 HU vs. 42±13 HU, P = .002). The gamma intensity analysis of sCT against the QA template was highly correlated with the MAE of sCT against the real CT in the testing cohort (r = -0.89 for T1W sCT; r = -0.93 for T2W sCT). CONCLUSION Accurate sRPSP images can be generated from T1W/T2W MRI for proton planning. A QA tool highlights regions of inaccuracy, flagging problematic cases unsuitable for clinical use. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Chuang Wang
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Jinsoo Uh
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Tushar Patni
- Department of Biostatistics, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Thomas Merchant
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Yimei Li
- Department of Biostatistics, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Chia-Ho Hua
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, TN, United States Of America
| | - Sahaja Acharya
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, TN, United States Of America.,Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins Medicine, Baltimore, MD, United States Of America
| |
Collapse
|
26
|
Florkow MC, Willemsen K, Mascarenhas VV, Oei EHG, van Stralen M, Seevinck PR. Magnetic Resonance Imaging Versus Computed Tomography for Three-Dimensional Bone Imaging of Musculoskeletal Pathologies: A Review. J Magn Reson Imaging 2022; 56:11-34. [PMID: 35044717 PMCID: PMC9305220 DOI: 10.1002/jmri.28067] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Revised: 12/31/2021] [Accepted: 01/05/2022] [Indexed: 12/18/2022] Open
Abstract
Magnetic resonance imaging (MRI) is increasingly utilized as a radiation‐free alternative to computed tomography (CT) for the diagnosis and treatment planning of musculoskeletal pathologies. MR imaging of hard tissues such as cortical bone remains challenging due to their low proton density and short transverse relaxation times, rendering bone tissues as nonspecific low signal structures on MR images obtained from most sequences. Developments in MR image acquisition and post‐processing have opened the path for enhanced MR‐based bone visualization aiming to provide a CT‐like contrast and, as such, ease clinical interpretation. The purpose of this review is to provide an overview of studies comparing MR and CT imaging for diagnostic and treatment planning purposes in orthopedic care, with a special focus on selective bone visualization, bone segmentation, and three‐dimensional (3D) modeling. This review discusses conventional gradient‐echo derived techniques as well as dedicated short echo time acquisition techniques and post‐processing techniques, including the generation of synthetic CT, in the context of 3D and specific bone visualization. Based on the reviewed literature, it may be concluded that the recent developments in MRI‐based bone visualization are promising. MRI alone provides valuable information on both bone and soft tissues for a broad range of applications including diagnostics, 3D modeling, and treatment planning in multiple anatomical regions, including the skull, spine, shoulder, pelvis, and long bones.
Collapse
Affiliation(s)
- Mateusz C Florkow
- Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Koen Willemsen
- Department of Orthopedics, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Vasco V Mascarenhas
- Musculoskeletal Imaging Unit, Imaging Center, Hospital da Luz, Lisbon, Portugal
| | - Edwin H G Oei
- Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
| | - Marijn van Stralen
- Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands.,MRIguidance BV, Utrecht, The Netherlands
| | - Peter R Seevinck
- Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands.,MRIguidance BV, Utrecht, The Netherlands
| |
Collapse
|
27
|
Feasibility of Synthetic Computed Tomography Images Generated from Magnetic Resonance Imaging Scans Using Various Deep Learning Methods in the Planning of Radiation Therapy for Prostate Cancer. Cancers (Basel) 2021; 14:cancers14010040. [PMID: 35008204 PMCID: PMC8750723 DOI: 10.3390/cancers14010040] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Revised: 12/17/2021] [Accepted: 12/20/2021] [Indexed: 11/17/2022] Open
Abstract
Simple Summary MRI-only simulation in radiation therapy (RT) planning has received attention because the CT scan can be omitted. For MRI-only simulation, synthetic CT (sCT) is necessary for the dose calculation. Various methodologies have been suggested for the generation of sCT and, recently, methods using the deep learning approaches are actively investigated. GAN and cycle-consistent GAN (CycGAN) have been mainly tested, however, very limited studies compared the qualities of sCTs generated from these methods or suggested other models for sCT generation. We have compared GAN, CycGAN, and, reference-guided GAN (RgGAN), a new model of deep learning method. We found that the performance in the HU conservation for soft tissue was poorest for GAN. All methods could generate sCTs feasible for VMAT planning with the trend that sCT generated from the RgGAN showed best performance in dosimetric conservation D98% and D95% than sCTs from other methodologies. Abstract We aimed to evaluate and compare the qualities of synthetic computed tomography (sCT) generated by various deep-learning methods in volumetric modulated arc therapy (VMAT) planning for prostate cancer. Simulation computed tomography (CT) and T2-weighted simulation magnetic resonance image from 113 patients were used in the sCT generation by three deep-learning approaches: generative adversarial network (GAN), cycle-consistent GAN (CycGAN), and reference-guided CycGAN (RgGAN), a new model which performed further adjustment of sCTs generated by CycGAN with available paired images. VMAT plans on the original simulation CT images were recalculated on the sCTs and the dosimetric differences were evaluated. For soft tissue, a significant difference in the mean Hounsfield unites (HUs) was observed between the original CT images and only sCTs from GAN (p = 0.03). The mean relative dose differences for planning target volumes or organs at risk were within 2% among the sCTs from the three deep-learning approaches. The differences in dosimetric parameters for D98% and D95% from original CT were lowest in sCT from RgGAN. In conclusion, HU conservation for soft tissue was poorest for GAN. There was the trend that sCT generated from the RgGAN showed best performance in dosimetric conservation D98% and D95% than sCTs from other methodologies.
Collapse
|
28
|
Chen Z, Lin L, Wu C, Li C, Xu R, Sun Y. Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine. Cancer Commun (Lond) 2021; 41:1100-1115. [PMID: 34613667 PMCID: PMC8626610 DOI: 10.1002/cac2.12215] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2021] [Revised: 07/10/2021] [Accepted: 09/01/2021] [Indexed: 12/12/2022] Open
Abstract
Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.
Collapse
Affiliation(s)
- Zi‐Hang Chen
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
- Zhongshan School of MedicineSun Yat‐sen UniversityGuangzhouGuangdong510080P. R. China
| | - Li Lin
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Chen‐Fei Wu
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Chao‐Feng Li
- Artificial Intelligence LaboratoryState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Rui‐Hua Xu
- Department of Medical OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| | - Ying Sun
- Department of Radiation OncologyState Key Laboratory of Oncology in South ChinaCollaborative Innovation Center for Cancer MedicineGuangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and TherapySun Yat‐sen University Cancer CenterGuangzhouGuangdong510060P. R. China
| |
Collapse
|
29
|
Boulanger M, Nunes JC, Chourak H, Largent A, Tahri S, Acosta O, De Crevoisier R, Lafond C, Barateau A. Deep learning methods to generate synthetic CT from MRI in radiotherapy: A literature review. Phys Med 2021; 89:265-281. [PMID: 34474325 DOI: 10.1016/j.ejmp.2021.07.027] [Citation(s) in RCA: 74] [Impact Index Per Article: 24.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/11/2021] [Revised: 07/15/2021] [Accepted: 07/19/2021] [Indexed: 01/04/2023] Open
Abstract
PURPOSE In radiotherapy, MRI is used for target volume and organs-at-risk delineation for its superior soft-tissue contrast as compared to CT imaging. However, MRI does not provide the electron density of tissue necessary for dose calculation. Several methods of synthetic-CT (sCT) generation from MRI data have been developed for radiotherapy dose calculation. This work reviewed deep learning (DL) sCT generation methods and their associated image and dose evaluation, in the context of MRI-based dose calculation. METHODS We searched the PubMed and ScienceDirect electronic databases from January 2010 to March 2021. For each paper, several items were screened and compiled in figures and tables. RESULTS This review included 57 studies. The DL methods were either generator-only based (45% of the reviewed studies), or generative adversarial network (GAN) architecture and its variants (55% of the reviewed studies). The brain and pelvis were the most commonly investigated anatomical localizations (39% and 28% of the reviewed studies, respectively), and more rarely, the head-and-neck (H&N) (15%), abdomen (10%), liver (5%) or breast (3%). All the studies performed an image evaluation of sCTs with a diversity of metrics, with only 36 studies performing dosimetric evaluations of sCT. CONCLUSIONS The median mean absolute errors were around 76 HU for the brain and H&N sCTs and 40 HU for the pelvis sCTs. For the brain, the mean dose difference between the sCT and the reference CT was <2%. For the H&N and pelvis, the mean dose difference was below 1% in most of the studies. Recent GAN architectures have advantages compared to generator-only, but no superiority was found in term of image or dose sCT uncertainties. Key challenges of DL-based sCT generation methods from MRI in radiotherapy is the management of movement for abdominal and thoracic localizations, the standardization of sCT evaluation, and the investigation of multicenter impacts.
Collapse
Affiliation(s)
- M Boulanger
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| | - Jean-Claude Nunes
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France.
| | - H Chourak
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France; CSIRO Australian e-Health Research Centre, Herston, Queensland, Australia
| | - A Largent
- Developing Brain Institute, Department of Diagnostic Imaging and Radiology, Children's National Hospital, Washington, DC, USA
| | - S Tahri
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| | - O Acosta
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| | - R De Crevoisier
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| | - C Lafond
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| | - A Barateau
- Univ. Rennes 1, CLCC Eugène Marquis, INSERM, LTSI - UMR 1099, F-35000 Rennes, France
| |
Collapse
|
30
|
Spadea MF, Maspero M, Zaffino P, Seco J. Deep learning based synthetic-CT generation in radiotherapy and PET: A review. Med Phys 2021; 48:6537-6566. [PMID: 34407209 DOI: 10.1002/mp.15150] [Citation(s) in RCA: 80] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 06/06/2021] [Accepted: 07/13/2021] [Indexed: 01/22/2023] Open
Abstract
Recently,deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: (i) to replace computed tomography in magnetic resonance (MR) based treatment planning, (ii) facilitate cone-beam computed tomography based image-guided adaptive radiotherapy, and (iii) derive attenuation maps for the correction of positron emission tomography. Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarizing the achievements. Lastly, the statistics of all the cited works from various aspects were analyzed, revealing the popularity and future trends and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.
Collapse
Affiliation(s)
- Maria Francesca Spadea
- Department Experimental and Clinical Medicine, University "Magna Graecia" of Catanzaro, Catanzaro, 88100, Italy
| | - Matteo Maspero
- Division of Imaging & Oncology, Department of Radiotherapy, University Medical Center Utrecht, Heidelberglaan, Utrecht, The Netherlands.,Computational Imaging Group for MR Diagnostics & Therapy, Center for Image Sciences, University Medical Center Utrecht, Heidelberglaan, Utrecht, The Netherlands
| | - Paolo Zaffino
- Department Experimental and Clinical Medicine, University "Magna Graecia" of Catanzaro, Catanzaro, 88100, Italy
| | - Joao Seco
- Division of Biomedical Physics in Radiation Oncology, DKFZ German Cancer Research Center, Heidelberg, Germany.,Department of Physics and Astronomy, Heidelberg University, Heidelberg, Germany
| |
Collapse
|
31
|
Guerreiro F, Svensson S, Seravalli E, Traneus E, Raaymakers BW. Intra-fractional per-beam adaptive workflow to mitigate the need for a rotating gantry during MRI-guided proton therapy. Phys Med Biol 2021; 66. [PMID: 34298523 DOI: 10.1088/1361-6560/ac176f] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Accepted: 07/23/2021] [Indexed: 11/12/2022]
Abstract
The integration of real-time magnetic resonance imaging (MRI)-guidance and proton therapy would potentially improve the proton dose steering capability by reducing daily uncertainties due to anatomical variations. The use of a fixed beamline coupled with an axial patient couch rotation would greatly simplify the proton delivery with MRI-guidance. Nonetheless, it is mandatory to assure that the plan quality is not deteriorated by the anatomical deformations due to patient rotation. In this work, an in-house tool allowing for intra-fractional per-beam adaptation of intensity-modulated proton plans (BeamAdapt) was implemented through features available in RayStation. A set of three MRIs was acquired for two healthy volunteers (V1, V2): (1) no rotation/static, (2) rotation to the right and (3) left. V1 was rotated by 15º, to simulate a clinical pediatric abdominal case and V2 by 45º, to simulate an extreme patient rotation case. For each volunteer, a total of four intensity-modulated pencil beam scanning plans were optimized on the static MRI using virtual abdominal targets and 2-3 posterior-oblique beams. Beam angles were defined according to the angulations on the rotated MRIs. With BeamAdapt, each original plan was first converted into separate plans with one beam per plan. In an iterative order, individual beam doses were non-rigidly deformed to the rotated anatomies and re-optimized accounting for the consequent deformations and the beam doses delivered so far. For evaluation, the final adapted dose distribution was propagated back to the static MRI. Planned and adapted dose distributions were compared by computing relative differences between dose-volume histogram (DVH) metrics. Absolute target dose differences were on average below 1% and mean dose organs-at-risk differences were below 3%. With BeamAdapt, not only intra-fractional per-beam proton plan adaptation coupled with axial patient rotation is possible but also the need for a rotating gantry during MRI-guidance might be mitigated.
Collapse
Affiliation(s)
- Filipa Guerreiro
- Department of Radiotherapy, University Medical Center Utrecht Imaging Division, Utrecht, NETHERLANDS
| | | | - Enrica Seravalli
- Department of Radiotherapy, University Medical Center Utrecht Imaging Division, Utrecht, NETHERLANDS
| | - Erik Traneus
- RaySearch Laboratories AB, Stockholm, Stockholm, SWEDEN
| | - Bas W Raaymakers
- Department of Radiotherapy, University Medical Center Utrecht Imaging Division, Utrecht, NETHERLANDS
| |
Collapse
|
32
|
Cusumano D, Boldrini L, Dhont J, Fiorino C, Green O, Güngör G, Jornet N, Klüter S, Landry G, Mattiucci GC, Placidi L, Reynaert N, Ruggieri R, Tanadini-Lang S, Thorwarth D, Yadav P, Yang Y, Valentini V, Verellen D, Indovina L. Artificial Intelligence in magnetic Resonance guided Radiotherapy: Medical and physical considerations on state of art and future perspectives. Phys Med 2021; 85:175-191. [PMID: 34022660 DOI: 10.1016/j.ejmp.2021.05.010] [Citation(s) in RCA: 49] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Revised: 04/15/2021] [Accepted: 05/04/2021] [Indexed: 12/14/2022] Open
Abstract
Over the last years, technological innovation in Radiotherapy (RT) led to the introduction of Magnetic Resonance-guided RT (MRgRT) systems. Due to the higher soft tissue contrast compared to on-board CT-based systems, MRgRT is expected to significantly improve the treatment in many situations. MRgRT systems may extend the management of inter- and intra-fraction anatomical changes, offering the possibility of online adaptation of the dose distribution according to daily patient anatomy and to directly monitor tumor motion during treatment delivery by means of a continuous cine MR acquisition. Online adaptive treatments require a multidisciplinary and well-trained team, able to perform a series of operations in a safe, precise and fast manner while the patient is waiting on the treatment couch. Artificial Intelligence (AI) is expected to rapidly contribute to MRgRT, primarily by safely and efficiently automatising the various manual operations characterizing online adaptive treatments. Furthermore, AI is finding relevant applications in MRgRT in the fields of image segmentation, synthetic CT reconstruction, automatic (on-line) planning and the development of predictive models based on daily MRI. This review provides a comprehensive overview of the current AI integration in MRgRT from a medical physicist's perspective. Medical physicists are expected to be major actors in solving new tasks and in taking new responsibilities: their traditional role of guardians of the new technology implementation will change with increasing emphasis on the managing of AI tools, processes and advanced systems for imaging and data analysis, gradually replacing many repetitive manual tasks.
Collapse
Affiliation(s)
- Davide Cusumano
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Rome, Italy
| | - Luca Boldrini
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Rome, Italy
| | | | - Claudio Fiorino
- Medical Physics, San Raffaele Scientific Institute, Milan, Italy
| | - Olga Green
- Department of Radiation Oncology, Washington University School of Medicine, St. Louis, MO, USA
| | - Görkem Güngör
- Acıbadem MAA University, School of Medicine, Department of Radiation Oncology, Maslak Istanbul, Turkey
| | - Núria Jornet
- Servei de Radiofísica i Radioprotecció, Hospital de la Santa Creu i Sant Pau, Spain
| | - Sebastian Klüter
- Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Guillaume Landry
- Department of Radiation Oncology, LMU Munich, Munich, Germany; German Cancer Consortium (DKTK), Munich, Germany
| | | | - Lorenzo Placidi
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Rome, Italy.
| | - Nick Reynaert
- Department of Medical Physics, Institut Jules Bordet, Belgium
| | - Ruggero Ruggieri
- Dipartimento di Radioterapia Oncologica Avanzata, IRCCS "Sacro cuore - don Calabria", Negrar di Valpolicella (VR), Italy
| | - Stephanie Tanadini-Lang
- Department of Radiation Oncology, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Daniela Thorwarth
- Section for Biomedical Physics, Department of Radiation Oncology, University Hospital Tüebingen, Tübingen, Germany
| | - Poonam Yadav
- Department of Human Oncology School of Medicine and Public Heath University of Wisconsin - Madison, USA
| | - Yingli Yang
- Department of Radiation Oncology, David Geffen School of Medicine, University of California Los Angeles, USA
| | - Vincenzo Valentini
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Rome, Italy
| | - Dirk Verellen
- Department of Medical Physics, Iridium Cancer Network, Belgium; Faculty of Medicine and Health Sciences, Antwerp University, Antwerp, Belgium
| | - Luca Indovina
- Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Rome, Italy
| |
Collapse
|