1
|
Reza MT, Alam MGR, Rahman R, Dipto SM. Domain affiliated distilled knowledge transfer for improved convergence of Ph-negative MPN identifier. PLoS One 2024; 19:e0303541. [PMID: 39331624 PMCID: PMC11433141 DOI: 10.1371/journal.pone.0303541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 04/26/2024] [Indexed: 09/29/2024] Open
Abstract
Ph-negative Myeloproliferative Neoplasm is a rare yet dangerous disease that can turn into more severe forms of disorders later on. Clinical diagnosis of the disease exists but often requires collecting multiple types of pathologies which can be tedious and time-consuming. Meanwhile, studies on deep learning-based research are rare and often need to rely on a small amount of pathological data due to the rarity of the disease. In addition, the existing research works do not address the data scarcity issue apart from using common techniques like data augmentation, which leaves room for performance improvement. To tackle the issue, the proposed research aims to utilize distilled knowledge learned from a larger dataset to boost the performance of a lightweight model trained on a small MPN dataset. Firstly, a 50-layer ResNet model is trained on a large lymph node image dataset of 3,27,680 images, followed by the trained knowledge being distilled to a small 4-layer CNN model. Afterward, the CNN model is initialized with the pre-trained weights to further train on a small MPN dataset of 300 images. Empirical analysis showcases that the CNN with distilled knowledge achieves 97% accuracy compared to 89.67% accuracy achieved by a clone CNN trained from scratch. The distilled knowledge transfer approach also proves to be more effective than more simple data scarcity handling approaches such as augmentation and manual feature extraction. Overall, the research affirms the effectiveness of transferring distilled knowledge to address the data scarcity issue and achieves better convergence when training on a Ph-Negative MPN image dataset with a lightweight model.
Collapse
|
2
|
Lin Y, Chen Q, Chen T. Recent advancements in machine learning for bone marrow cell morphology analysis. Front Med (Lausanne) 2024; 11:1402768. [PMID: 38947236 PMCID: PMC11211563 DOI: 10.3389/fmed.2024.1402768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Accepted: 05/31/2024] [Indexed: 07/02/2024] Open
Abstract
As machine learning progresses, techniques such as neural networks, decision trees, and support vector machines are being increasingly applied in the medical domain, especially for tasks involving large datasets, such as cell detection, recognition, classification, and visualization. Within the domain of bone marrow cell morphology analysis, deep learning offers substantial benefits due to its robustness, ability for automatic feature learning, and strong image characterization capabilities. Deep neural networks are a machine learning paradigm specifically tailored for image processing applications. Artificial intelligence serves as a potent tool in supporting the diagnostic process of clinical bone marrow cell morphology. Despite the potential of artificial intelligence to augment clinical diagnostics in this domain, manual analysis of bone marrow cell morphology remains the gold standard and an indispensable tool for identifying, diagnosing, and assessing the efficacy of hematologic disorders. However, the traditional manual approach is not without limitations and shortcomings, necessitating, the exploration of automated solutions for examining and analyzing bone marrow cytomorphology. This review provides a multidimensional account of six bone marrow cell morphology processes: automated bone marrow cell morphology detection, automated bone marrow cell morphology segmentation, automated bone marrow cell morphology identification, automated bone marrow cell morphology classification, automated bone marrow cell morphology enumeration, and automated bone marrow cell morphology diagnosis. Highlighting the attractiveness and potential of machine learning systems based on bone marrow cell morphology, the review synthesizes current research and recent advances in the application of machine learning in this field. The objective of this review is to offer recommendations to hematologists for selecting the most suitable machine learning algorithms to automate bone marrow cell morphology examinations, enabling swift and precise analysis of bone marrow cytopathic trends for early disease identification and diagnosis. Furthermore, the review endeavors to delineate potential future research avenues for machine learning-based applications in bone marrow cell morphology analysis.
Collapse
Affiliation(s)
- Yifei Lin
- The Second Affiliated Hospital of Fujian Medical University, Quanzhou, Fujian, China
- The School of Basic Medical Sciences, Fujian Medical University, Fuzhou, Fujian, China
| | - Qingquan Chen
- The Second Affiliated Hospital of Fujian Medical University, Quanzhou, Fujian, China
- The School of Public Health, Fujian Medical University, Fuzhou, Fujian, China
| | - Tebin Chen
- The Second Affiliated Hospital of Fujian Medical University, Quanzhou, Fujian, China
| |
Collapse
|
3
|
Ryou H, Lomas O, Theissen H, Thomas E, Rittscher J, Royston D. Quantitative interpretation of bone marrow biopsies in MPN-What's the point in a molecular age? Br J Haematol 2023; 203:523-535. [PMID: 37858962 PMCID: PMC10952168 DOI: 10.1111/bjh.19154] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 09/20/2023] [Accepted: 09/30/2023] [Indexed: 10/21/2023]
Abstract
The diagnosis of myeloproliferative neoplasms (MPN) requires the integration of clinical, morphological, genetic and immunophenotypic findings. Recently, there has been a transformation in our understanding of the cellular and molecular mechanisms underlying disease initiation and progression in MPN. This has been accompanied by the widespread application of high-resolution quantitative molecular techniques. By contrast, microscopic interpretation of bone marrow biopsies by haematologists/haematopathologists remains subjective and qualitative. However, advances in tissue image analysis and artificial intelligence (AI) promise to transform haematopathology. Pioneering studies in bone marrow image analysis offer to refine our understanding of the boundaries between reactive samples and MPN subtypes and better capture the morphological correlates of high-risk disease. They also demonstrate potential to improve the evaluation of current and novel therapeutics for MPN and other blood cancers. With increased therapeutic targeting of diverse molecular, cellular and extra-cellular components of the marrow, these approaches can address the unmet need for improved objective and quantitative measures of disease modification in the context of clinical trials. This review focuses on the state-of-the-art in image analysis/AI of bone marrow tissue, with an emphasis on its potential to complement and inform future clinical studies and research in MPN.
Collapse
Affiliation(s)
- Hosuk Ryou
- Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of MedicineUniversity of OxfordOxfordUK
| | - Oliver Lomas
- Department of HaematologyOxford University Hospitals NHS Foundation TrustOxfordUK
| | - Helen Theissen
- Department of Engineering Science, Institute of Biomedical Engineering (IBME)University of OxfordOxfordUK
| | - Emily Thomas
- Department of Engineering Science, Institute of Biomedical Engineering (IBME)University of OxfordOxfordUK
| | - Jens Rittscher
- Department of Engineering Science, Institute of Biomedical Engineering (IBME)University of OxfordOxfordUK
- Ground Truth LabsOxfordUK
- Oxford NIHR Biomedical Research CentreOxford University Hospitals NHS Foundation TrustOxfordUK
- Ludwig Institute for Cancer ResearchUniversity of OxfordOxfordUK
| | - Daniel Royston
- Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of MedicineUniversity of OxfordOxfordUK
- Department of PathologyOxford University Hospitals NHS Foundation TrustOxfordUK
| |
Collapse
|
4
|
Shiffman S, Rios Piedra EA, Adedeji AO, Ruff CF, Andrews RN, Katavolos P, Liu E, Forster A, Brumm J, Fuji RN, Sullivan R. Analysis of cellularity in H&E-stained rat bone marrow tissue via deep learning. J Pathol Inform 2023; 14:100333. [PMID: 37743975 PMCID: PMC10514468 DOI: 10.1016/j.jpi.2023.100333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/18/2023] [Accepted: 08/19/2023] [Indexed: 09/26/2023] Open
Abstract
Our objective was to develop an automated deep-learning-based method to evaluate cellularity in rat bone marrow hematoxylin and eosin whole slide images for preclinical safety assessment. We trained a shallow CNN for segmenting marrow, 2 Mask R-CNN models for segmenting megakaryocytes (MKCs), and small hematopoietic cells (SHCs), and a SegNet model for segmenting red blood cells. We incorporated the models into a pipeline that identifies and counts MKCs and SHCs in rat bone marrow. We compared cell segmentation and counts that our method generated to those that pathologists generated on 10 slides with a range of cell depletion levels from 10 studies. For SHCs, we compared cell counts that our method generated to counts generated by Cellpose and Stardist. The median Dice and object Dice scores for MKCs using our method vs pathologist consensus and the inter- and intra-pathologist variation were comparable, with overlapping first-third quartile ranges. For SHCs, the median scores were close, with first-third quartile ranges partially overlapping intra-pathologist variation. For SHCs, in comparison to Cellpose and Stardist, counts from our method were closer to pathologist counts, with a smaller 95% limits of agreement range. The performance of the bone marrow analysis pipeline supports its incorporation into routine use as an aid for hematotoxicity assessment by pathologists. The pipeline could help expedite hematotoxicity assessment in preclinical studies and consequently could expedite drug development. The method may enable meta-analysis of rat bone marrow characteristics from future and historical whole slide images and may generate new biological insights from cross-study comparisons.
Collapse
Affiliation(s)
- Smadar Shiffman
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Edgar A. Rios Piedra
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Adeyemi O. Adedeji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Catherine F. Ruff
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Rachel N. Andrews
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Paula Katavolos
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- Bristol Myers Squibb, New Brunswick, NJ 08901, USA
| | - Evan Liu
- Genentech Research and Early Development (gRED), Department of Development Sciences Informatics, Genentech Inc, South San Francisco, USA
| | - Ashley Forster
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
- University of Pennsylvania School of Veterinary Medicine, Philadelphia, PA 19104, USA
| | - Jochen Brumm
- Genentech Research and Early Development (gRED), Department of Nonclinical Biostatistics, Genentech Inc, South San Francisco, USA
| | - Reina N. Fuji
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| | - Ruth Sullivan
- Genentech Research and Early Development (gRED), Department of Safety Assessment, Genentech Inc., South San Francisco, USA
| |
Collapse
|
5
|
Zhou H, Sun C, Huang H, Fan M, Yang X, Zhou L. Feature-guided attention network for medical image segmentation. Med Phys 2023; 50:4871-4886. [PMID: 36746870 DOI: 10.1002/mp.16253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 01/03/2023] [Accepted: 01/06/2023] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND U-Net and its variations have achieved remarkable performances in medical image segmentation. However, they have two limitations. First, the shallow layer feature of the encoder always contains background noise. Second, semantic gaps exist between the features of the encoder and the decoder. Skip-connections directly connect the encoder to the decoder, which will lead to the fusion of semantically dissimilar feature maps. PURPOSE To overcome these two limitations, this paper proposes a novel medical image segmentation algorithm, called feature-guided attention network, which consists of U-Net, the cross-level attention filtering module (CAFM), and the attention-guided upsampling module (AUM). METHODS In the proposed method, the AUM and the CAFM were introduced into the U-Net, where the AUM learns to filter the background noise in the low-level feature map of the encoder and the CAFM tries to eliminate the semantic gap between the encoder and the decoder. Specifically, the AUM adopts a top-down pathway to use the high-level feature map so as to filter the background noise in the low-level feature map of the encoder. The AUM uses the encoder features to guide the upsampling of the corresponding decoder features, thus eliminating the semantic gap between them. Four medical image segmentation tasks, including coronary atherosclerotic plaque segmentation (Dataset A), retinal vessel segmentation (Dataset B), skin lesion segmentation (Dataset C), and multiclass retinal edema lesions segmentation (Dataset D), were used to validate the proposed method. RESULTS For Dataset A, the proposed method achieved higher Intersection over Union (IoU) (67.91 ± 3.82 % $67.91\pm 3.82\%$ ), dice (79.39 ± 3.37 % $79.39\pm 3.37\%$ ), accuracy (98.39 ± 0.34 % $98.39\pm 0.34\%$ ), and sensitivity (85.10 ± 3.74 % $85.10\pm 3.74\%$ ) than the previous best method: CA-Net. For Dataset B, the proposed method achieved higher sensitivity (83.50%) and accuracy (97.55%) than the previous best method: SCS-Net. For Dataset C, the proposed method had highest IoU (83.47 ± 0.41 % $83.47\pm 0.41\%$ ) and dice (90.81 ± 0.34 % $90.81\pm 0.34\%$ ) than those of all compared previous methods. For Dataset D, the proposed method had highest dice (average: 81.53%; retina edema area [REA]: 83.78%; pigment epithelial detachment [PED] 77.13%), sensitivity (REA: 89.01%; SRF: 85.50%), specificity (REA: 99.35%; PED: 100.00), and accuracy (98.73%) among all compared previous networks. In addition, the number of parameters of the proposed method was 2.43 M, which is less than CA-Net (3.21 M) and CPF-Net (3.07 M). CONCLUSIONS The proposed method demonstrated state-of-the-art performance, outperforming other top-notch medical image segmentation algorithms. The CAFM filtered the background noise in the low-level feature map of the encoder, while the AUM eliminated the semantic gap between the encoder and the decoder. Furthermore, the proposed method was of high computational efficiency.
Collapse
Affiliation(s)
- Hao Zhou
- National Key Laboratory of Science and Technology of Underwater Vehicle, Harbin Engineering University, Harbin, China
| | - Chaoyu Sun
- Fourth Affiliated Hospital, Harbin Medical University, Harbin, China
| | - Hai Huang
- National Key Laboratory of Science and Technology of Underwater Vehicle, Harbin Engineering University, Harbin, China
| | - Mingyu Fan
- College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, China
| | - Xu Yang
- State Key Laboratory of Management and Control for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Linxiao Zhou
- Fourth Affiliated Hospital, Harbin Medical University, Harbin, China
| |
Collapse
|
6
|
Basu A, Senapati P, Deb M, Rai R, Dhal KG. A survey on recent trends in deep learning for nucleus segmentation from histopathology images. EVOLVING SYSTEMS 2023; 15:1-46. [PMID: 38625364 PMCID: PMC9987406 DOI: 10.1007/s12530-023-09491-3] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Accepted: 02/13/2023] [Indexed: 03/08/2023]
Abstract
Nucleus segmentation is an imperative step in the qualitative study of imaging datasets, considered as an intricate task in histopathology image analysis. Segmenting a nucleus is an important part of diagnosing, staging, and grading cancer, but overlapping regions make it hard to separate and tell apart independent nuclei. Deep Learning is swiftly paving its way in the arena of nucleus segmentation, attracting quite a few researchers with its numerous published research articles indicating its efficacy in the field. This paper presents a systematic survey on nucleus segmentation using deep learning in the last five years (2017-2021), highlighting various segmentation models (U-Net, SCPP-Net, Sharp U-Net, and LiverNet) and exploring their similarities, strengths, datasets utilized, and unfolding research areas.
Collapse
Affiliation(s)
- Anusua Basu
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| | - Pradip Senapati
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| | - Mainak Deb
- Wipro Technologies, Pune, Maharashtra India
| | - Rebika Rai
- Department of Computer Applications, Sikkim University, Sikkim, India
| | - Krishna Gopal Dhal
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| |
Collapse
|
7
|
Karabağ C, Ortega-Ruíz MA, Reyes-Aldasoro CC. Impact of Training Data, Ground Truth and Shape Variability in the Deep Learning-Based Semantic Segmentation of HeLa Cells Observed with Electron Microscopy. J Imaging 2023; 9:59. [PMID: 36976110 PMCID: PMC10058680 DOI: 10.3390/jimaging9030059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 02/16/2023] [Accepted: 02/17/2023] [Indexed: 03/06/2023] Open
Abstract
This paper investigates the impact of the amount of training data and the shape variability on the segmentation provided by the deep learning architecture U-Net. Further, the correctness of ground truth (GT) was also evaluated. The input data consisted of a three-dimensional set of images of HeLa cells observed with an electron microscope with dimensions 8192×8192×517. From there, a smaller region of interest (ROI) of 2000×2000×300 was cropped and manually delineated to obtain the ground truth necessary for a quantitative evaluation. A qualitative evaluation was performed on the 8192×8192 slices due to the lack of ground truth. Pairs of patches of data and labels for the classes nucleus, nuclear envelope, cell and background were generated to train U-Net architectures from scratch. Several training strategies were followed, and the results were compared against a traditional image processing algorithm. The correctness of GT, that is, the inclusion of one or more nuclei within the region of interest was also evaluated. The impact of the extent of training data was evaluated by comparing results from 36,000 pairs of data and label patches extracted from the odd slices in the central region, to 135,000 patches obtained from every other slice in the set. Then, 135,000 patches from several cells from the 8192×8192 slices were generated automatically using the image processing algorithm. Finally, the two sets of 135,000 pairs were combined to train once more with 270,000 pairs. As would be expected, the accuracy and Jaccard similarity index improved as the number of pairs increased for the ROI. This was also observed qualitatively for the 8192×8192 slices. When the 8192×8192 slices were segmented with U-Nets trained with 135,000 pairs, the architecture trained with automatically generated pairs provided better results than the architecture trained with the pairs from the manually segmented ground truths. This suggests that the pairs that were extracted automatically from many cells provided a better representation of the four classes of the various cells in the 8192×8192 slice than those pairs that were manually segmented from a single cell. Finally, the two sets of 135,000 pairs were combined, and the U-Net trained with these provided the best results.
Collapse
Affiliation(s)
- Cefa Karabağ
- giCentre, Department of Computer Science, School of Science and Technology, City, University of London, London EC1V 0HB, UK
| | - Mauricio Alberto Ortega-Ruíz
- giCentre, Department of Computer Science, School of Science and Technology, City, University of London, London EC1V 0HB, UK
- Departamento de Ingeniería, Campus Coyoacán, Universidad del Valle de México, Ciudad de México C.P. 04910, Mexico
| | | |
Collapse
|
8
|
Wang X, Wang Y, Qi C, Qiao S, Yang S, Wang R, Jin H, Zhang J. The Application of Morphogo in the Detection of Megakaryocytes from Bone Marrow Digital Images with Convolutional Neural Networks. Technol Cancer Res Treat 2023; 22:15330338221150069. [PMID: 36700246 PMCID: PMC9896096 DOI: 10.1177/15330338221150069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023] Open
Abstract
The evaluation of megakaryocytes is an important part of the work up on bone marrow smear examination. It has significance in the differential diagnosis, therapeutic efficacy assessment, and predication of prognosis of many hematologic diseases. The process of manual identification of megakaryocytes are tedious and lack of reproducibility; therefore, a reliable method of automated megakaryocytic identification is urgently needed. Three hundred and thirty-three bone marrow aspirate smears were digitized by Morphogo system. Pathologists annotated megakaryocytes on the digital images of marrow smears are applied to construct a large dataset for testing the system's predictive performance. Subsequently, we obtained megakaryocyte count and classification for each sample by different methods (system-automated analysis, system-assisted analysis, and microscopic examination) to study the correlation between different counting and classification methods. Morphogo system localized cells likely to be megakaryocytes on digital smears, which were later annotated by pathologists and the system, respectively. The system showed outstanding performance in identifying megakaryocytes in bone marrow smears with high sensitivity (96.57%) and specificity (89.71%). The overall correlation between the different methods was confirmed the high consistency (r ≥ 0.7218, R2 ≥ 0.5211) with microscopic examination in classifying megakaryocytes. Morphogo system was proved as a reliable screen tool for analyzing megakaryocytes. The application of Morphogo system shows promises to advance the automation and standardization of bone marrow smear examination.
Collapse
Affiliation(s)
- Xiaofen Wang
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China
| | - Ying Wang
- Department of Medical Development, Hangzhou Zhiwei
Information&Technology Ltd., Hangzhou, China
| | - Chao Qi
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China
| | - Sai Qiao
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China
| | - Suwen Yang
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China
| | - Rongrong Wang
- Department of Clinical Pharmacy, the First Affiliated Hospital,
Zhejiang University, Hangzhou, Zhejiang, China
| | - Hong Jin
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China
| | - Jun Zhang
- Clinical Laboratory, Sir Run Run Shaw Hospital, School of Medicine,
Zhejiang University, Hangzhou, Zhejiang, China,Key Laboratory of Precision Medicine in Diagnosis and Monitoring
Research of Zhejiang Province, China,Jun Zhang, Clinical Laboratory, Sir Run Run
Shaw Hospital, School of Medicine, Zhejiang University, No.3, Qingchun East
Road, Shangcheng District, Hangzhou, Zhejiang 310016, China.
Hong Jin, Clinical Laboratory, Sir
Run Run Shaw Hospital, School of Medicine, Zhejiang University, No.3, Qingchun
East Road, Shangcheng District, Hangzhou, Zhejiang 310016, China.
| |
Collapse
|
9
|
Zhu H, Geng T, Wang J, Tang Q, Jiang W. Improved sub-category exploration and attention hybrid network for weakly supervised semantic segmentation. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08250-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
|
10
|
Alabdaly AA, El-Sayed WG, Hassan YF. RAMRU-CAM: Residual-Atrous MultiResUnet with Channel Attention Mechanism for cell segmentation. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-222631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
The task of cell segmentation in microscope images is difficult and popular. In recent years, deep learning-based techniques have made incredible progress in medical and microscopy image segmentation applications. In this paper, we propose a novel deep learning approach called Residual-Atrous MultiResUnet with Channel Attention Mechanism (RAMRU-CAM) for cell segmentation, which combines MultiResUnet architecture with Channel Attention Mechanism (CAM) and Residual-Atrous connections. The Residual-Atrous path mitigates the semantic gap between the encoder and decoder stages and manages the spatial dimension of feature maps. Furthermore, the Channel Attention Mechanism (CAM) blocks are used in the decoder stages to better maintain the spatial details before concatenating the feature maps from the encoder phases to the decoder phases. We evaluated our proposed model on the PhC-C2DH-U373 and Fluo-N2DH-GOWT1 datasets. The experimental results show that our proposed model outperforms recent variants of the U-Net model and the state-of-the-art approaches. We have demonstrated how our model can segment cells precisely while using fewer parameters and low computational complexity.
Collapse
Affiliation(s)
- Ammar A. Alabdaly
- Department of Mathematics and Computer Science, Alexandria University, Alexandria, Egypt
| | - Wagdy G. El-Sayed
- Department of Mathematics and Computer Science, Alexandria University, Alexandria, Egypt
| | - Yasser F. Hassan
- Faculty of Computer and Data Science, Alexandria University, Alexandria, Egypt
| |
Collapse
|
11
|
Wang Z, Saoud C, Wangsiricharoen S, James AW, Popel AS, Sulam J. Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3952-3968. [PMID: 36037454 PMCID: PMC9825360 DOI: 10.1109/tmi.2022.3202759] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. However, generating exhaustive and accurate annotations is labor-intensive, challenging, and costly. Drawing only coarse and approximate annotations is a much easier task, less costly, and it alleviates pathologists' workload. In this paper, we study the problem of refining these approximate annotations in digital pathology to obtain more accurate ones. Some previous works have explored obtaining machine learning models from these inaccurate annotations, but few of them tackle the refinement problem where the mislabeled regions should be explicitly identified and corrected, and all of them require a - often very large - number of training samples. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need for external training data. Patches cropped from a WSI with inaccurate labels are processed jointly within a multiple instance learning framework, mitigating their impact on the predictive model and refining the segmentation. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming state-of-the-art alternatives, even while learning from a single slide. Moreover, we demonstrate how real annotations drawn by pathologists can be efficiently refined and improved by the proposed approach. All these results demonstrate that LC-MIL is a promising, lightweight tool to provide fine-grained annotations from coarsely annotated pathology sets.
Collapse
|
12
|
Chanchal AK, Lal S, Kini J. Deep structured residual encoder-decoder network with a novel loss function for nuclei segmentation of kidney and breast histopathology images. MULTIMEDIA TOOLS AND APPLICATIONS 2022; 81:9201-9224. [PMID: 35125928 PMCID: PMC8809220 DOI: 10.1007/s11042-021-11873-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2020] [Revised: 11/10/2021] [Accepted: 12/23/2021] [Indexed: 06/14/2023]
Abstract
To improve the process of diagnosis and treatment of cancer disease, automatic segmentation of haematoxylin and eosin (H & E) stained cell nuclei from histopathology images is the first step in digital pathology. The proposed deep structured residual encoder-decoder network (DSREDN) focuses on two aspects: first, it effectively utilized residual connections throughout the network and provides a wide and deep encoder-decoder path, which results to capture relevant context and more localized features. Second, vanished boundary of detected nuclei is addressed by proposing an efficient loss function that better train our proposed model and reduces the false prediction which is undesirable especially in healthcare applications. The proposed architecture experimented on three different publicly available H&E stained histopathological datasets namely: (I) Kidney (RCC) (II) Triple Negative Breast Cancer (TNBC) (III) MoNuSeg-2018. We have considered F1-score, Aggregated Jaccard Index (AJI), the total number of parameters, and FLOPs (Floating point operations), which are mostly preferred performance measure metrics for comparison of nuclei segmentation. The evaluated score of nuclei segmentation indicated that the proposed architecture achieved a considerable margin over five state-of-the-art deep learning models on three different histopathology datasets. Visual segmentation results show that the proposed DSREDN model accurately segment the nuclear regions than those of the state-of-the-art methods.
Collapse
Affiliation(s)
- Amit Kumar Chanchal
- Department of Electronics and Communication Engineering, National Institute of Technology Karnataka, Surathkal, Mangaluru-575025 Karnataka India
| | - Shyam Lal
- Department of Electronics and Communication Engineering, National Institute of Technology Karnataka, Surathkal, Mangaluru-575025 Karnataka India
| | - Jyoti Kini
- Department of Pathology, Kasturba Medical College Mangalore, Manipal Academy of Higher Education, Manipal, India
| |
Collapse
|
13
|
Kaur A, Kaur L, Singh A. GA-UNet: UNet-based framework for segmentation of 2D and 3D medical images applicable on heterogeneous datasets. Neural Comput Appl 2021. [DOI: 10.1007/s00521-021-06134-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
14
|
Chanchal AK, Lal S, Kini J. High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images. Int J Comput Assist Radiol Surg 2021; 16:2159-2175. [PMID: 34622381 DOI: 10.1007/s11548-021-02497-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2021] [Accepted: 09/08/2021] [Indexed: 11/28/2022]
Abstract
PURPOSE Increasing cancer disease incidence worldwide has become a major public health issue. Manual histopathological analysis is a common diagnostic method for cancer detection. Due to the complex structure and wide variability in the texture of histopathology images, it has been challenging for pathologists to diagnose manually those images. Automatic segmentation of histopathology images to diagnose cancer disease is a continuous exploration field in recent times. Segmentation and analysis for diagnosis of histopathology images by using an efficient deep learning algorithm are the purpose of the proposed method. METHOD To improve the segmentation performance, we proposed a deep learning framework that consists of a high-resolution encoder path, an atrous spatial pyramid pooling bottleneck module, and a powerful decoder. Compared to the benchmark segmentation models having a deep and thin path, our network is wide and deep that effectively leverages the strength of residual learning as well as encoder-decoder architecture. RESULTS We performed careful experimentation and analysis on three publically available datasets namely kidney dataset, Triple Negative Breast Cancer (TNBC) dataset, and MoNuSeg histopathology image dataset. We have used the two most preferred performance metrics called F1 score and aggregated Jaccard index (AJI) to evaluate the performance of the proposed model. The measured values of F1 score and AJI score are (0.9684, 0.9394), (0.8419, 0.7282), and (0.8344, 0.7169) on the kidney dataset, TNBC histopathology dataset, and MoNuSeg dataset, respectively. CONCLUSION Our proposed method yields better results as compared to benchmark segmentation methods on three histopathology datasets. Visual segmentation results justify the high value of the F1 score and AJI scores which indicated that it is a very good prediction by our proposed model.
Collapse
Affiliation(s)
- Amit Kumar Chanchal
- Department of E&C Engineering, National Institute of Technology Karnataka, Surathkal, Mangaluru, Karnataka, 575025, India
| | - Shyam Lal
- Department of E&C Engineering, National Institute of Technology Karnataka, Surathkal, Mangaluru, Karnataka, 575025, India.
| | - Jyoti Kini
- Department of Pathology, Kasturba Medical College, Manipal Academy of Higher Education, Mangalore, Manipal, India
| |
Collapse
|
15
|
Sun Y, Huang X, Zhou H, Zhang Q. SRPN: similarity-based region proposal networks for nuclei and cells detection in histology images. Med Image Anal 2021; 72:102142. [PMID: 34198042 DOI: 10.1016/j.media.2021.102142] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Revised: 05/11/2021] [Accepted: 06/17/2021] [Indexed: 10/21/2022]
Abstract
The detection of nuclei and cells in histology images is of great value in both clinical practice and pathological studies. However, multiple reasons such as morphological variations of nuclei or cells make it a challenging task where conventional object detection methods cannot obtain satisfactory performance in many cases. A detection task consists of two sub-tasks, classification and localization. Under the condition of dense object detection, classification is a key to boost the detection performance. Considering this, we propose similarity based region proposal networks (SRPN) for nuclei and cells detection in histology images. In particular, a customised convolution layer termed as embedding layer is designed for network building. The embedding layer is added into the region proposal networks, enabling the networks to learn discriminative features based on similarity learning. Features obtained by similarity learning can significantly boost the classification performance compared to conventional methods. SRPN can be easily integrated into standard convolutional neural networks architectures such as the Faster R-CNN and RetinaNet. We test the proposed approach on tasks of multi-organ nuclei detection and signet ring cells detection in histological images. Experimental results show that networks applying similarity learning achieved superior performance on both tasks when compared to their counterparts. In particular, the proposed SRPN achieve state-of-the-art performance on the MoNuSeg benchmark for nuclei segmentation and detection while compared to previous methods, and on the signet ring cell detection benchmark when compared with baselines. The sourcecode is publicly available at: https://github.com/sigma10010/nuclei_cells_det.
Collapse
Affiliation(s)
- Yibao Sun
- School of Electronic Engineering and Computer Science, Queen Mary University of London, Mile End Road, London, E1 4NS, United Kingdom
| | - Xingru Huang
- School of Electronic Engineering and Computer Science, Queen Mary University of London, Mile End Road, London, E1 4NS, United Kingdom.
| | - Huiyu Zhou
- School of Informatics, University of Leicester, University Road, Leicester, LE1 7RH, United Kingdom
| | - Qianni Zhang
- School of Electronic Engineering and Computer Science, Queen Mary University of London, Mile End Road, London, E1 4NS, United Kingdom
| |
Collapse
|
16
|
Karabağ C, Jones ML, Reyes-Aldasoro CC. Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells. J Imaging 2021; 7:93. [PMID: 39080881 PMCID: PMC8321355 DOI: 10.3390/jimaging7060093] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 05/24/2021] [Accepted: 05/25/2021] [Indexed: 01/10/2023] Open
Abstract
In this work, an unsupervised volumetric semantic instance segmentation of the plasma membrane of HeLa cells as observed with serial block face scanning electron microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8192 × 8192 pixels each. The background was used to create a distance map, which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., the largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2000 × 2000 × 300 voxels that were treated as cell instances. Then, for each of these volumes, the nucleus was segmented, and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions of interest previously selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and the Jaccard similarity Index (JI): nucleus: JI =0.9665, AC =0.9975, cell including nucleus JI =0.8711, AC =0.9655, cell excluding nucleus JI =0.8094, AC =0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background. In samples with tightly packed cells, this may not be available. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data were released openly through GitHub, Zenodo and EMPIAR.
Collapse
Affiliation(s)
- Cefa Karabağ
- giCentre, Department of Computer Science, School of Mathematics, Computer Science and Engineering, City, University of London, London EC1V 0HB, UK;
| | - Martin L. Jones
- Electron Microscopy Science Technology Platform, The Francis Crick Institute, London NW1 1AT, UK;
| | - Constantino Carlos Reyes-Aldasoro
- giCentre, Department of Computer Science, School of Mathematics, Computer Science and Engineering, City, University of London, London EC1V 0HB, UK;
| |
Collapse
|
17
|
Hayakawa T, Prasath VBS, Kawanaka H, Aronow BJ, Tsuruoka S. Computational Nuclei Segmentation Methods in Digital Pathology: A Survey. ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING 2021; 28:1-13. [DOI: 10.1007/s11831-019-09366-4] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2019] [Accepted: 09/30/2019] [Indexed: 02/07/2023]
|
18
|
Xia H, Sun W, Song S, Mou X. Md-Net: Multi-scale Dilated Convolution Network for CT Images Segmentation. Neural Process Lett 2020. [DOI: 10.1007/s11063-020-10230-x] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
19
|
Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J. CE-Net: Context Encoder Network for 2D Medical Image Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:2281-2292. [PMID: 30843824 DOI: 10.1109/tmi.2019.2903562] [Citation(s) in RCA: 774] [Impact Index Per Article: 129.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Medical image segmentation is an important step in medical image analysis. With the rapid development of a convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, and so on. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations led to the loss of some spatial information. In this paper, we propose a context encoder network (CE-Net) to capture more high-level information and preserve spatial information for 2D medical image segmentation. CE-Net mainly contains three major components: a feature encoder module, a context extractor, and a feature decoder module. We use the pretrained ResNet block as the fixed feature extractor. The context extractor module is formed by a newly proposed dense atrous convolution block and a residual multi-kernel pooling block. We applied the proposed CE-Net to different 2D medical image segmentation tasks. Comprehensive results show that the proposed method outperforms the original U-Net method and other state-of-the-art methods for optic disc segmentation, vessel detection, lung segmentation, cell contour segmentation, and retinal optical coherence tomography layer segmentation.
Collapse
|
20
|
Deng X, Lan T, Zhang M, Chen Z, Tao Q, Lu Z. [A fast adaptive active contour model based on local gray difference for parotid duct]. NAN FANG YI KE DA XUE XUE BAO = JOURNAL OF SOUTHERN MEDICAL UNIVERSITY 2018; 38:1485-1491. [PMID: 30613018 DOI: 10.12122/j.issn.1673-4254.2018.12.14] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
OBJECTIVE To establish a fast adaptive active contour model based on local gray difference for parotid duct image segmentation. METHODS On the basis of the LBF model, we added the mean difference of the local gray scale inside and outside the contour as the energy term of the driving evolution curve, and the local gray-scale variance difference was used to replaceλ1 and λ2 as the control term of the energy parameter value. Two local similarity factors of different neighborhood sizes were introduced to correct the effects of image gray unevenness and boundary blur to improve the segmentation efficiency. RESULTS During image segmentation, this algorithm allowed for adaptive adjustment of the evolution direction, velocity and the energy weight of the internal and external regions according to the difference of gray mean and variance between the internal and external regions. This algorithm was also capable of detecting the actual boundary in a complex gradient boundary region, thus enabling the evolution curve to approach the target boundary quickly and accurately. CONCLUSIONS The proposed algorithm is superior to the existing segmentation algorithms and allows fast and accurate segmentation of the parotid duct with well-preserved image details.
Collapse
Affiliation(s)
- Xuan Deng
- Key Lab for Medical Imaging of Southern Medical University, Guangzhou 510515, China
| | - Tianjun Lan
- Department of Oral and Maxillofacial Surgery, Guanghua School of Stomatology, Hospital of Stomatology Affiliated to Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou 510055, China
| | - Minghui Zhang
- Key Lab for Medical Imaging of Southern Medical University, Guangzhou 510515, China
| | - Zhifeng Chen
- Department of Stomatology, Nanfang Hospital, Southern Medical University, Guangzhou 510515, China
| | - Qian Tao
- Department of Oral and Maxillofacial Surgery, Guanghua School of Stomatology, Hospital of Stomatology Affiliated to Sun Yat-sen University, Guangdong Provincial Key Laboratory of Stomatology, Guangzhou 510055, China
| | - Zhentai Lu
- Key Lab for Medical Imaging of Southern Medical University, Guangzhou 510515, China
| |
Collapse
|
21
|
Sanchez V, Hernandez-Cabronero M. Graph-Based Rate Control in Pathology Imaging With Lossless Region of Interest Coding. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2211-2223. [PMID: 29993629 DOI: 10.1109/tmi.2018.2824819] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The increasing availability of digital pathology images has motivated the design of tools to foster multidisciplinary collaboration among researchers, pathologists, and computer scientists. Telepathology plays an important role in the development of collaborative tools, as it facilitates the transmission and access to pathology images by multiple users. However, the huge file size associated with pathology images usually prevents full exploitation of the collaborative telepathology system potential. Within this context, rate control (RC) is an important tool that allows meeting storage and bandwidth requirements by controlling the bit rate of the coded image. In this paper, we propose a novel graph-based RC algorithm with lossless region of interest (RoI) coding for pathology images. The algorithm, which is designed for block-based predictive transform coding methods, compresses the non-RoI in a lossy manner according to a target bit rate and the RoI in a lossless manner. It employs a graph where each node represents a constituent block of the image to be coded. By incorporating information about the coding cost similarities of blocks into the graph, a graph kernel is used to distribute a target bit budget among the non-RoI blocks. In order to increase RC accuracy, the algorithm uses a rate-lambda (R- ) model to approximate the slope of the rate-distortion curve of the non-RoI, and a graph-based approach to guarantee that the target bit rate is accurately attained. The algorithm is implemented in the High-Efficiency Video Coding standard and tested over a wide range of pathology images with multiple RoIs. Evaluation results show that it outperforms the other state-of-the-art methods designed for single images by very accurately attaining the target bit rate of the non-RoI.
Collapse
|
22
|
Zhang Z, Lim YW, Zhao P, Kanchanawong P, Motegi F. ImaEdge: a platform for the quantitative analysis of cortical proteins spatiotemporal dynamics during cell polarization. J Cell Sci 2017; 130:4200-4212. [DOI: 10.1242/jcs.206870] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2017] [Accepted: 11/01/2017] [Indexed: 11/20/2022] Open
Abstract
Cell polarity involves the compartmentalization of the cell cortex. The establishment of cortical compartments arises from the spatial bias in the activity and concentration of cortical proteins. The mechanistic dissection of cell polarity requires the accurate detection of dynamic changes in cortical proteins, but the fluctuations of cell shape and the inhomogeneous distributions of cortical proteins greatly complicate the quantitative extraction of their global and local changes during cell polarization. To address these problems, we introduce an open-source software package, ImaEdge, which automates the segmentation of the cortex from time-lapse movies, and enables quantitative extraction of cortical protein intensities. We demonstrate that ImaEdge enables efficient and rigorous analysis of the dynamic evolution of cortical PAR proteins during C. elegans embryogenesis. It is also capable of accurate tracking of varying levels of transgene expression and discontinuous signals of the actomyosin cytoskeleton during multiple rounds of cell division. ImaEdge provides a unique resource for the quantitative studies of cortical polarization, with the potential for application to many types of polarized cells.
Collapse
Affiliation(s)
- Zhen Zhang
- Mechanobiology Institute, National University of Singapore, Singapore
| | - Yen Wei Lim
- Temasek Life-sciences Laboratory, Department of Biological Sciences, National University of Singapore, Singapore
| | - Peng Zhao
- Temasek Life-sciences Laboratory, Department of Biological Sciences, National University of Singapore, Singapore
| | - Pakorn Kanchanawong
- Mechanobiology Institute, National University of Singapore, Singapore
- Department of Biomedical engineering, National University of Singapore, Singapore
| | - Fumio Motegi
- Mechanobiology Institute, National University of Singapore, Singapore
- Temasek Life-sciences Laboratory, Department of Biological Sciences, National University of Singapore, Singapore
| |
Collapse
|