1
|
Contino S, Cruciata L, Gambino O, Pirrone R. IODeep: An IOD for the introduction of deep learning in the DICOM standard. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108113. [PMID: 38479148 DOI: 10.1016/j.cmpb.2024.108113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 02/22/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND AND OBJECTIVE In recent years, Artificial Intelligence (AI) and in particular Deep Neural Networks (DNN) became a relevant research topic in biomedical image segmentation due to the availability of more and more data sets along with the establishment of well known competitions. Despite the popularity of DNN based segmentation on the research side, these techniques are almost unused in the daily clinical practice even if they could support effectively the physician during the diagnostic process. Apart from the issues related to the explainability of the predictions of a neural model, such systems are not integrated in the diagnostic workflow, and a standardization of their use is needed to achieve this goal. METHODS This paper presents IODeep a new DICOM Information Object Definition (IOD) aimed at storing both the weights and the architecture of a DNN already trained on a particular image dataset that is labeled as regards the acquisition modality, the anatomical region, and the disease under investigation. RESULTS The IOD architecture is presented along with a DNN selection algorithm from the PACS server based on the labels outlined above, and a simple PACS viewer purposely designed for demonstrating the effectiveness of the DICOM integration, while no modifications are required on the PACS server side. Also a service based architecture in support of the entire workflow has been implemented. CONCLUSION IODeep ensures full integration of a trained AI model in a DICOM infrastructure, and it is also enables a scenario where a trained model can be either fine-tuned with hospital data or trained in a federated learning scheme shared by different hospitals. In this way AI models can be tailored to the real data produced by a Radiology ward thus improving the physician decision making process. Source code is freely available at https://github.com/CHILab1/IODeep.git.
Collapse
Affiliation(s)
- Salvatore Contino
- Department of Engineering, University of Palermo, Palermo, 90128, Sicily, Italy
| | - Luca Cruciata
- Department of Engineering, University of Palermo, Palermo, 90128, Sicily, Italy
| | - Orazio Gambino
- Department of Engineering, University of Palermo, Palermo, 90128, Sicily, Italy.
| | - Roberto Pirrone
- Department of Engineering, University of Palermo, Palermo, 90128, Sicily, Italy
| |
Collapse
|
2
|
Diot-Dejonghe T, Leporq B, Bouhamama A, Ratiney H, Pilleul F, Beuf O, Cervenansky F. Development of a Secure Web-Based Medical Imaging Analysis Platform: The AWESOMME Project. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01110-0. [PMID: 38689149 DOI: 10.1007/s10278-024-01110-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 03/12/2024] [Accepted: 04/02/2024] [Indexed: 05/02/2024]
Abstract
Precision medicine research benefits from machine learning in the creation of robust models adapted to the processing of patient data. This applies both to pathology identification in images, i.e., annotation or segmentation, and to computer-aided diagnostic for classification or prediction. It comes with the strong need to exploit and visualize large volumes of images and associated medical data. The work carried out in this paper follows on from a main case study piloted in a cancer center. It proposes an analysis pipeline for patients with osteosarcoma through segmentation, feature extraction and application of a deep learning model to predict response to treatment. The main aim of the AWESOMME project is to leverage this work and implement the pipeline on an easy-to-access, secure web platform. The proposed WEB application is based on a three-component architecture: a data server, a heavy computation and authentication server and a medical imaging web-framework with a user interface. These existing components have been enhanced to meet the needs of security and traceability for the continuous production of expert data. It innovates by covering all steps of medical imaging processing (visualization and segmentation, feature extraction and aided diagnostic) and enables the test and use of machine learning models. The infrastructure is operational, deployed in internal production and is currently being installed in the hospital environment. The extension of the case study and user feedback enabled us to fine-tune functionalities and proved that AWESOMME is a modular solution capable to analyze medical data and share research algorithms with in-house clinicians.
Collapse
Affiliation(s)
- Tiphaine Diot-Dejonghe
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
| | - Benjamin Leporq
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
| | - Amine Bouhamama
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
- Department of Radiology, Centre Léon Bérard, 28 Prom. Léa et Napoléon Bullukian, Lyon, 69008, France
| | - Helene Ratiney
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
| | - Frank Pilleul
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
- Department of Radiology, Centre Léon Bérard, 28 Prom. Léa et Napoléon Bullukian, Lyon, 69008, France
| | - Olivier Beuf
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France
| | - Frederic Cervenansky
- INSA-Lyon, Université Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, Lyon, F-69XXX, France.
| |
Collapse
|
3
|
Jesus R, Bastião Silva L, Sousa V, Carvalho L, Garcia Gonzalez D, Carias J, Costa C. Personalizable AI platform for universal access to research and diagnosis in digital pathology. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107787. [PMID: 37717524 DOI: 10.1016/j.cmpb.2023.107787] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 08/22/2023] [Accepted: 09/01/2023] [Indexed: 09/19/2023]
Abstract
BACKGROUND AND MOTIVATION Digital pathology has been evolving over the last years, proposing significant workflow advantages that have fostered its adoption in professional environments. Patient clinical and image data are readily available in remote data banks that can be consumed efficiently over standard communication technologies. The appearance of new imaging techniques and advanced artificial intelligence algorithms has significantly reduced the burden on medical professionals by speeding up the screening process. Despite these advancements, the usage of digital pathology in professional environments has been slowed down by poor interoperability between services resulting from a lack of standard interfaces and integrative solutions. This work addresses this issue by proposing a cloud-based digital pathology platform built on standard and open interfaces. METHODS The work proposes and describes a vendor-neutral platform that provides interfaces for managing digital slides, and medical reports, and integrating digital image analysis services compatible with existing standards. The solution integrates the open-source plugin-based Dicoogle PACS for interoperability and extensibility, which grants the proposed solution great feature customization. RESULTS The solution was developed in collaboration with iPATH research project partners, including the validation by medical pathologists. The result is a pure Web collaborative framework that supports both research and production environments. A total of 566 digital slides from different pathologies were successfully uploaded to the platform. Using the integration interfaces, a mitosis detection algorithm was successfully installed into the platform, and it was trained with 2400 annotations collected from breast carcinoma images. CONCLUSION Interoperability is a key factor when discussing digital pathology solutions, as it facilitates their integration into existing institutions' information systems. Moreover, it improves data sharing and integration of third-party services such as image analysis services, which have become relevant in today's digital pathology workflow. The proposed solution fully embraces the DICOM standard for digital pathology, presenting an interoperable cloud-based solution that provides great feature customization thanks to its extensible architecture.
Collapse
Affiliation(s)
- Rui Jesus
- University of A. Coruña, A Coruña, Spain; BMD Software, Aveiro, Portugal.
| | | | - Vítor Sousa
- Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | - Lina Carvalho
- Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | | | - João Carias
- Center for Computer Graphics, Braga, Portugal
| | - Carlos Costa
- IEETA/DETI, University of Aveiro, Aveiro, Portugal
| |
Collapse
|
4
|
Pielawski N, Andersson A, Avenel C, Behanova A, Chelebian E, Klemm A, Nysjö F, Solorzano L, Wählby C. TissUUmaps 3: Improvements in interactive visualization, exploration, and quality assessment of large-scale spatial omics data. Heliyon 2023; 9:e15306. [PMID: 37131430 PMCID: PMC10149187 DOI: 10.1016/j.heliyon.2023.e15306] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 03/30/2023] [Accepted: 04/03/2023] [Indexed: 05/04/2023] Open
Abstract
Background and objectives Spatially resolved techniques for exploring the molecular landscape of tissue samples, such as spatial transcriptomics, often result in millions of data points and images too large to view on a regular desktop computer, limiting the possibilities in visual interactive data exploration. TissUUmaps is a free, open-source browser-based tool for GPU-accelerated visualization and interactive exploration of 107+ data points overlaying tissue samples. Methods Herein we describe how TissUUmaps 3 provides instant multiresolution image viewing and can be customized, shared, and also integrated into Jupyter Notebooks. We introduce new modules where users can visualize markers and regions, explore spatial statistics, perform quantitative analyses of tissue morphology, and assess the quality of decoding in situ transcriptomics data. Results We show that thanks to targeted optimizations the time and cost associated with interactive data exploration were reduced, enabling TissUUmaps 3 to handle the scale of today's spatial transcriptomics methods. Conclusion TissUUmaps 3 provides significantly improved performance for large multiplex datasets as compared to previous versions. We envision TissUUmaps to contribute to broader dissemination and flexible sharing of largescale spatial omics data.
Collapse
Affiliation(s)
- Nicolas Pielawski
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Axel Andersson
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Christophe Avenel
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Andrea Behanova
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Eduard Chelebian
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Anna Klemm
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Fredrik Nysjö
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
| | - Leslie Solorzano
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Carolina Wählby
- Department of Information Technology and SciLifeLab BioImage Informatics Facility, Uppsala University, Uppsala, Sweden
- Corresponding author.
| |
Collapse
|
5
|
Gorman C, Punzo D, Octaviano I, Pieper S, Longabaugh WJR, Clunie DA, Kikinis R, Fedorov AY, Herrmann MD. Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology. Nat Commun 2023; 14:1572. [PMID: 36949078 PMCID: PMC10033920 DOI: 10.1038/s41467-023-37224-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 03/08/2023] [Indexed: 03/24/2023] Open
Abstract
The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.
Collapse
Affiliation(s)
- Chris Gorman
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | | | | | | | | | | | - Ron Kikinis
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA
| | - Andrey Y Fedorov
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA.
| | - Markus D Herrmann
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
6
|
Familiar AM, Mahtabfar A, Fathi Kazerooni A, Kiani M, Vossough A, Viaene A, Storm PB, Resnick AC, Nabavizadeh A. Radio-pathomic approaches in pediatric neuro-oncology: Opportunities and challenges. Neurooncol Adv 2023; 5:vdad119. [PMID: 37841693 PMCID: PMC10576517 DOI: 10.1093/noajnl/vdad119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2023] Open
Abstract
With medical software platforms moving to cloud environments with scalable storage and computing, the translation of predictive artificial intelligence (AI) models to aid in clinical decision-making and facilitate personalized medicine for cancer patients is becoming a reality. Medical imaging, namely radiologic and histologic images, has immense analytical potential in neuro-oncology, and models utilizing integrated radiomic and pathomic data may yield a synergistic effect and provide a new modality for precision medicine. At the same time, the ability to harness multi-modal data is met with challenges in aggregating data across medical departments and institutions, as well as significant complexity in modeling the phenotypic and genotypic heterogeneity of pediatric brain tumors. In this paper, we review recent pathomic and integrated pathomic, radiomic, and genomic studies with clinical applications. We discuss current challenges limiting translational research on pediatric brain tumors and outline technical and analytical solutions. Overall, we propose that to empower the potential residing in radio-pathomics, systemic changes in cross-discipline data management and end-to-end software platforms to handle multi-modal data sets are needed, in addition to embracing modern AI-powered approaches. These changes can improve the performance of predictive models, and ultimately the ability to advance brain cancer treatments and patient outcomes through the development of such models.
Collapse
Affiliation(s)
- Ariana M Familiar
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Aria Mahtabfar
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Thomas Jefferson University Hospital, Philadelphia, PA, USA
| | - Anahita Fathi Kazerooni
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Mahsa Kiani
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Arastoo Vossough
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Angela Viaene
- Department of Pathology and Laboratory Medicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Phillip B Storm
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Adam C Resnick
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Ali Nabavizadeh
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
7
|
Liu L, Wang L, Huang Q, Zhou L, Fu X, Liu L. An efficient architecture for medical high-resolution images transmission in mobile telemedicine systems. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 187:105088. [PMID: 31784039 DOI: 10.1016/j.cmpb.2019.105088] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2019] [Revised: 09/17/2019] [Accepted: 09/21/2019] [Indexed: 06/10/2023]
Abstract
BACKGROUND AND OBJECTIVE The medical high-resolution image is very important in image processing and computer vision applications, which plays a critical role in image-guided diagnosis, clinical trials, consultation, and case discussion. How to efficiently access medical high-resolution images in mobile telemedicine systems is becoming a big challenge. Therefore, this work proposes an efficient pyramid architecture for optimizing medical high-resolution images transmission and rendering. METHODS The proposed architecture consists of three core schemes: (1) unbalance pyramid scheme based on geometric relationship, (2) indexing scheme based on hash table and lattice partitioning and (3) query scheme based on similar matching. Then, we design the responsive service components: generating service, indexing service, and query service. Finally, these services are combined into a prototype system that enables efficient transmission and rendering of medical high-resolution images. RESULTS The result shows that the unbalance pyramid scheme can quickly generate the pyramid structure and the corresponding image files. The indexing scheme can create the index structure and the index file in real-time. The query scheme can not only match the best layer to which the image block belongs in real-time, but also can accurately capture the query image block. CONCLUSIONS The prototype system based on proposed architecture is fully compliant with the DICOM standard, which can be seamlessly integrated with other existing medical systems or mobile applications, and used in various scenarios such as diagnosis, research, and education.
Collapse
Affiliation(s)
- Lijun Liu
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming 650091, China; Computer Technology Application Key Laboratory of Yunnan Province (Faculty of Information Engineering and Automation, Kunming University of Science and Technology), Kunming, 650500, China; Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Lizhen Wang
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming 650091, China.
| | - Qingsong Huang
- Computer Technology Application Key Laboratory of Yunnan Province (Faculty of Information Engineering and Automation, Kunming University of Science and Technology), Kunming, 650500, China; Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Lihua Zhou
- Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming 650091, China
| | - Xiaodong Fu
- Computer Technology Application Key Laboratory of Yunnan Province (Faculty of Information Engineering and Automation, Kunming University of Science and Technology), Kunming, 650500, China; Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| | - Li Liu
- Computer Technology Application Key Laboratory of Yunnan Province (Faculty of Information Engineering and Automation, Kunming University of Science and Technology), Kunming, 650500, China; Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China
| |
Collapse
|