1
|
Veiga-Canuto D, Cerdà-Alberich L, Sangüesa Nebot C, Martínez de las Heras B, Pötschger U, Gabelloni M, Carot Sierra JM, Taschner-Mandl S, Düster V, Cañete A, Ladenstein R, Neri E, Martí-Bonmatí L. Comparative Multicentric Evaluation of Inter-Observer Variability in Manual and Automatic Segmentation of Neuroblastic Tumors in Magnetic Resonance Images. Cancers (Basel) 2022; 14:cancers14153648. [PMID: 35954314 PMCID: PMC9367307 DOI: 10.3390/cancers14153648] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 07/21/2022] [Accepted: 07/26/2022] [Indexed: 02/05/2023] Open
Abstract
Simple Summary Tumor segmentation is a key step in oncologic imaging processing and is a time-consuming process usually performed manually by radiologists. To facilitate it, there is growing interest in applying deep-learning segmentation algorithms. Thus, we explore the variability between two observers performing manual segmentation and use the state-of-the-art deep learning architecture nnU-Net to develop a model to detect and segment neuroblastic tumors on MR images. We were able to show that the variability between nnU-Net and manual segmentation is similar to the inter-observer variability in manual segmentation. Furthermore, we compared the time needed to manually segment the tumors from scratch with the time required for the automatic model to segment the same cases, with posterior human validation with manual adjustment when needed. Abstract Tumor segmentation is one of the key steps in imaging processing. The goals of this study were to assess the inter-observer variability in manual segmentation of neuroblastic tumors and to analyze whether the state-of-the-art deep learning architecture nnU-Net can provide a robust solution to detect and segment tumors on MR images. A retrospective multicenter study of 132 patients with neuroblastic tumors was performed. Dice Similarity Coefficient (DSC) and Area Under the Receiver Operating Characteristic Curve (AUC ROC) were used to compare segmentation sets. Two more metrics were elaborated to understand the direction of the errors: the modified version of False Positive (FPRm) and False Negative (FNR) rates. Two radiologists manually segmented 46 tumors and a comparative study was performed. nnU-Net was trained-tuned with 106 cases divided into five balanced folds to perform cross-validation. The five resulting models were used as an ensemble solution to measure training (n = 106) and validation (n = 26) performance, independently. The time needed by the model to automatically segment 20 cases was compared to the time required for manual segmentation. The median DSC for manual segmentation sets was 0.969 (±0.032 IQR). The median DSC for the automatic tool was 0.965 (±0.018 IQR). The automatic segmentation model achieved a better performance regarding the FPRm. MR images segmentation variability is similar between radiologists and nnU-Net. Time leverage when using the automatic model with posterior visual validation and manual adjustment corresponds to 92.8%.
Collapse
Affiliation(s)
- Diana Veiga-Canuto
- Grupo de Investigación Biomédica en Imagen, Instituto de Investigación Sanitaria La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain; (L.C.-A.); (L.M.-B.)
- Área Clínica de Imagen Médica, Hospital Universitario y Politécnico La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain;
- Correspondence:
| | - Leonor Cerdà-Alberich
- Grupo de Investigación Biomédica en Imagen, Instituto de Investigación Sanitaria La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain; (L.C.-A.); (L.M.-B.)
| | - Cinta Sangüesa Nebot
- Área Clínica de Imagen Médica, Hospital Universitario y Politécnico La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain;
| | - Blanca Martínez de las Heras
- Unidad de Oncohematología Pediátrica, Hospital Universitario y Politécnico La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain; (B.M.d.l.H.); (A.C.)
| | - Ulrike Pötschger
- St. Anna Children’s Cancer Research Institute, Zimmermannplatz 10, 1090 Vienna, Austria; (U.P.); (S.T.-M.); (V.D.); (R.L.)
| | - Michela Gabelloni
- Academic Radiology, Department of Translational Research, University of Pisa, Via Roma, 67, 56126 Pisa, Italy; (M.G.); (E.N.)
| | - José Miguel Carot Sierra
- Departamento de Estadística e Investigación Operativa Aplicadas y Calidad, Universitat Politècnica de València, Camí de Vera s/n, 46022 Valencia, Spain;
| | - Sabine Taschner-Mandl
- St. Anna Children’s Cancer Research Institute, Zimmermannplatz 10, 1090 Vienna, Austria; (U.P.); (S.T.-M.); (V.D.); (R.L.)
| | - Vanessa Düster
- St. Anna Children’s Cancer Research Institute, Zimmermannplatz 10, 1090 Vienna, Austria; (U.P.); (S.T.-M.); (V.D.); (R.L.)
| | - Adela Cañete
- Unidad de Oncohematología Pediátrica, Hospital Universitario y Politécnico La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain; (B.M.d.l.H.); (A.C.)
| | - Ruth Ladenstein
- St. Anna Children’s Cancer Research Institute, Zimmermannplatz 10, 1090 Vienna, Austria; (U.P.); (S.T.-M.); (V.D.); (R.L.)
| | - Emanuele Neri
- Academic Radiology, Department of Translational Research, University of Pisa, Via Roma, 67, 56126 Pisa, Italy; (M.G.); (E.N.)
| | - Luis Martí-Bonmatí
- Grupo de Investigación Biomédica en Imagen, Instituto de Investigación Sanitaria La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain; (L.C.-A.); (L.M.-B.)
- Área Clínica de Imagen Médica, Hospital Universitario y Politécnico La Fe, Avenida Fernando Abril Martorell, 106 Torre A 7planta, 46026 Valencia, Spain;
| |
Collapse
|
2
|
Hasman A, Ammenwerth E, Dickhaus H, Knaup P, Lovis C, Mantas J, Maojo V, Martin-Sanchez FJ, Musen M, Patel VL, Surjan G, Talmon JL, Sarkar IN. Biomedical informatics--a confluence of disciplines? Methods Inf Med 2012; 50:508-24. [PMID: 22146914 DOI: 10.3414/me11-06-0003] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
BACKGROUND Biomedical informatics is a broad discipline that borrows many methods and techniques from other disciplines. OBJECTIVE To reflect a) on the character of biomedical informatics and to determine whether it is multi-disciplinary or inter-disciplinary; b) on the question whether biomedical informatics is more than the sum of its supporting disciplines and c) on the position of biomedical informatics with respect to related disciplines. METHOD Inviting an international group of experts in biomedical informatics and related disciplines on the occasion of the 50th anniversary of Methods of Information in Medicine to present their viewpoints. RESULTS AND CONCLUSIONS This paper contains the reflections of a number of the invited experts on the character of biomedical informatics. Most of the authors agree that biomedical informatics is an interdisciplinary field of study where researchers with different scientific backgrounds alone or in combination carry out research. Biomedical informatics is a very broad scientific field and still expanding, yet comprised of a constructive aspect (designing and building systems). One author expressed that the essence of biomedical informatics, as opposed to related disciplines, lies in the modelling of the biomedical content. Interdisciplinarity also has consequences for education. Maintaining rigid disciplinary structures does not allow for sufficient adaptability to capitalize on important trends nor to leverage the influences these trends may have on biomedical informatics. It is therefore important for students to become aware of research findings in related disciplines. In this respect, it was also noted that the fact that many scientific fields use different languages and that the research findings are stored in separate bibliographic databases makes it possible that potentially connected findings will never be linked, despite the fact that these findings were published. Bridges between the sciences are needed for the success of biomedical informatics.
Collapse
Affiliation(s)
- A Hasman
- Department of Medical Informatics, University of Amsterdam, Academic Medical Center, Meibergdreef 15, 1105 AZ Amsterdam Z. O., The Netherlands.
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|