1
|
Machireddy A, Thibault G, Loftis KG, Stoltz K, Bueno CE, Smith HR, Riesterer JL, Gray JW, Song X. Segmentation of cellular ultrastructures on sparsely labeled 3D electron microscopy images using deep learning. Front Bioinform 2023; 3:1308708. [PMID: 38162124 PMCID: PMC10754953 DOI: 10.3389/fbinf.2023.1308708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 11/22/2023] [Indexed: 01/03/2024] Open
Abstract
Focused ion beam-scanning electron microscopy (FIB-SEM) images can provide a detailed view of the cellular ultrastructure of tumor cells. A deeper understanding of their organization and interactions can shed light on cancer mechanisms and progression. However, the bottleneck in the analysis is the delineation of the cellular structures to enable quantitative measurements and analysis. We mitigated this limitation using deep learning to segment cells and subcellular ultrastructure in 3D FIB-SEM images of tumor biopsies obtained from patients with metastatic breast and pancreatic cancers. The ultrastructures, such as nuclei, nucleoli, mitochondria, endosomes, and lysosomes, are relatively better defined than their surroundings and can be segmented with high accuracy using a neural network trained with sparse manual labels. Cell segmentation, on the other hand, is much more challenging due to the lack of clear boundaries separating cells in the tissue. We adopted a multi-pronged approach combining detection, boundary propagation, and tracking for cell segmentation. Specifically, a neural network was employed to detect the intracellular space; optical flow was used to propagate cell boundaries across the z-stack from the nearest ground truth image in order to facilitate the separation of individual cells; finally, the filopodium-like protrusions were tracked to the main cells by calculating the intersection over union measure for all regions detected in consecutive images along z-stack and connecting regions with maximum overlap. The proposed cell segmentation methodology resulted in an average Dice score of 0.93. For nuclei, nucleoli, and mitochondria, the segmentation achieved Dice scores of 0.99, 0.98, and 0.86, respectively. The segmentation of FIB-SEM images will enable interpretative rendering and provide quantitative image features to be associated with relevant clinical variables.
Collapse
Affiliation(s)
- Archana Machireddy
- Program of Computer Science and Electrical Engineering, Oregon Health and Science University, Portland, OR, United States
| | - Guillaume Thibault
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
- Knight Cancer Institute, Oregon Health and Science University, Portland, OR, United States
| | - Kevin G. Loftis
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
| | - Kevin Stoltz
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
| | - Cecilia E. Bueno
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
| | - Hannah R. Smith
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
| | - Jessica L. Riesterer
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
- Knight Cancer Institute, Oregon Health and Science University, Portland, OR, United States
| | - Joe W. Gray
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
- Knight Cancer Institute, Oregon Health and Science University, Portland, OR, United States
| | - Xubo Song
- Program of Computer Science and Electrical Engineering, Oregon Health and Science University, Portland, OR, United States
- Department of Biomedical Engineering, Oregon Health and Science University, Portland, OR, United States
- Knight Cancer Institute, Oregon Health and Science University, Portland, OR, United States
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, OR, United States
| |
Collapse
|