Deep learning for bone marrow cell detection and classification on whole-slide images

Digital morphology compared to the optical microscope: A validation study on reporting bone marrow aspirates

INTRODUCTION

This study aims to evaluate the effectiveness and reliability of the utilization for clinical reporting of the evaluation of digital images of bone marrow aspirates by morphologists and their comparability with the classic microscopic morphological evaluation.

METHODS

We scanned 180 consecutive bone marrow needle aspirates smears using the "Metafer4 VSlide" whole slide imaging (WSI) digital scanning system. We evaluated the statistical comparability and the risk of bias of the microscopic readings with those performed on the screen on the digitized medullary images.

RESULTS

The evaluation of cellularity on the screen was equivalent, with a higher frequency of "normal" than the analysis of digital preparations. The means and medians of the percentage values obtained on the different cell populations with the microscopic and digital reading were comparable as the main categories are concerned, with an average difference equal to 0 for the neutrophilic and eosinophilic granulocytic series, at -0.2% for the total myeloid cells, at 1.2% for the erythroid series, at -0.4% for the lymphocytes and at -0.4% for the blasts. Dysplastic features were consistently identified in 69/71 cell lineages.

CONCLUSION

Our study demonstrated that screen evaluation of digitized bone marrow needle aspirates provides quantitative and qualitative results comparable to traditional microscopic analysis of the corresponding slide smears. Digital images offer significant benefits in reducing the workload of experienced operators, reproducibility and sharing of observations, and image preservation. Even in routine diagnostic activities, their use does not alter the quality of the results obtained in evaluating bone marrow needle aspirates.

Deep learning to assess microsatellite instability directly from histopathological whole slide images in endometrial cancer

Molecular classification, particularly microsatellite instability-high (MSI-H), has gained attention for immunotherapy in endometrial cancer (EC). MSI-H is associated with DNA mismatch repair defects and is a crucial treatment predictor. The NCCN guidelines recommend pembrolizumab and nivolumab for advanced or recurrent MSI-H/mismatch repair deficient (dMMR) EC. However, evaluating MSI in all cases is impractical due to time and cost constraints. To overcome this challenge, we present an effective and efficient deep learning-based model designed to accurately and rapidly assess MSI status of EC using H&E-stained whole slide images. Our framework was evaluated on a comprehensive dataset of gigapixel histopathology images of 529 patients from the Cancer Genome Atlas (TCGA). The experimental results have shown that the proposed method achieved excellent performances in assessing MSI status, obtaining remarkably high results with 96%, 94%, 93% and 100% for endometrioid carcinoma G1G2, respectively, and 87%, 84%, 81% and 94% for endometrioid carcinoma G3, in terms of F-measure, accuracy, precision and sensitivity, respectively. Furthermore, the proposed deep learning framework outperforms four state-of-the-art benchmarked methods by a significant margin (p < 0.001) in terms of accuracy, precision, sensitivity and F-measure, respectively. Additionally, a run time analysis demonstrates that the proposed method achieves excellent quantitative results with high efficiency in AI inference time (1.03 seconds per slide), making the proposed framework viable for practical clinical usage. These results highlight the efficacy and efficiency of the proposed model to assess MSI status of EC directly from histopathological slides.

Classification and prediction of chemoradiotherapy response and survival from esophageal carcinoma histopathology images

Whole slide imaging (WSI) of Hematoxylin and Eosin-stained biopsy specimens has been used to predict chemoradiotherapy (CRT) response and overall survival (OS) of esophageal squamous cell carcinoma (ESCC) patients. This retrospective study collected 279 specimens in 89 non-surgical ESCC patients through endoscopic biopsy between January 2010 and January 2019. These patients were divided into a CRT response group (CR + PR group) and a CRT non-response group (SD + PD group). The WSIs have segmented approximately 1,206,000 non-overlapping patches. Two experienced pathologists manually delineated the eight types of tissues on 32 WSIs, including esophagus tumor cell (TUM), cancer-associated stroma (CAS), normal epithelium layer (NEL), smooth muscle (MUS), lymphocytes (LYM), Red cells (RED), debris (DEB), uneven areas (UNE). The chemoradiotherapy response prediction models were built using maximum relevance-minimum redundancy (MRMR) feature selection and least absolute shrinkage and selection operator (LASSO) regression. However, pathological features with p < 0.1 were selected and integrated to be further screened using a LASSO Cox regression model to build a multivariate Cox proportional hazards model for predicting the OS. The testing accuracy of the tissue classification model was 91.3 %. The pathological model created using two CAS in-depth features and eight TUM in-depth features performed best for the prediction of treatment response and achieved an AUC of 0.744. For the prediction of OS, the testing AUC of this model at one year and three years were 0.675 and 0.870, respectively. The TUM model showed the highest AUC at one year (0.712). With its high accuracy rate, the deep learning model has the potential to transform from bench to bedside in clinical practice, improve patient's quality of life, and prolong the OS rate.

Advances in the Application of Artificial Intelligence in Fetal Echocardiography

Congenital heart disease is a severe health risk for newborns. Early detection of abnormalities in fetal cardiac structure and function during pregnancy can help patients seek timely diagnostic and therapeutic advice, and early intervention planning can significantly improve fetal survival rates. Echocardiography is one of the most accessible and widely used diagnostic tools in the diagnosis of fetal congenital heart disease. However, traditional fetal echocardiography has limitations due to fetal, maternal, and ultrasound equipment factors and is highly dependent on the skill level of the operator. Artificial intelligence (AI) technology, with its rapid development utilizing advanced computer algorithms, has great potential to empower sonographers in time-saving and accurate diagnosis and to bridge the skill gap in different regions. In recent years, AI-assisted fetal echocardiography has been successfully applied to a wide range of ultrasound diagnoses. This review systematically reviews the applications of AI in the field of fetal echocardiography over the years in terms of image processing, biometrics, and disease diagnosis and provides an outlook for future research.

Development of a Deep Learning Model for the Analysis of Dorsal Root Ganglion Chromatolysis in Rat Spinal Stenosis

Objective

To create a deep learning (DL) model that can accurately detect and classify three distinct types of rat dorsal root ganglion neurons: normal, segmental chromatolysis, and central chromatolysis. The DL model has the potential to improve the efficiency and precision of neuron classification in research related to spinal injuries and diseases.

Methods

H&E slide images were divided into an internal training set (80%) and a test set (20%). The training dataset was labeled by two pathologists using pre-defined grades. Using this dataset, a two-component DL model was developed with the first component being a convolutional neural network (CNN) that was trained to detect the region of interest (ROI) and the second component being another CNN used for classification.

Results

A total of 240 lumbar dorsal root ganglion (DRG) pathology slide images from rats were analyzed. The internal testing results showed an accuracy of 93.13%, and the external dataset testing demonstrated an accuracy of 93.44%.

Conclusion

The DL model demonstrated a level of agreement comparable to that of pathologists in detecting and classifying normal and segmental chromatolysis neurons, although its agreement was slightly lower for central chromatolysis neurons. Significance: DL in improving the accuracy and efficiency of pathological analysis suggests that it may have a role in enhancing medical decision-making.

GravityNet for end-to-end small lesion detection

This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. To address this, our approach introduces a new type of pixel-based anchor that dynamically moves towards the targeted lesion for detection. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions. We conducted experiments on two well-established medical problems involving small lesions to evaluate the performance of the proposed approach: microcalcifications detection in digital mammograms and microaneurysms detection in digital fundus images. Our method demonstrates promising results in effectively detecting small lesions in these medical imaging tasks.

The nexus of nuclear envelope dynamics, circular economy and cancer cell pathophysiology

The nuclear envelope (NE) is a critical component in maintaining the function and structure of the eukaryotic nucleus. The NE and lamina are disassembled during each cell cycle to enable an open mitosis. Nuclear architecture construction and deconstruction is a prime example of a circular economy, as it fulfills a highly efficient recycling program bound to continuous assessment of the quality and functionality of the building blocks. Alterations in the nuclear dynamics and lamina structure have emerged as important contributors to both oncogenic transformation and cancer progression. However, the knowledge of the NE breakdown and reassembly is still limited to a fraction of participating proteins and complexes. As cancer cells contain highly diverse nuclei in terms of DNA content, but also in terms of nuclear number, size, and shape, it is of great interest to understand the intricate relationship between these nuclear features in cancer cell pathophysiology. In this review, we provide insights into how those NE dynamics are regulated, and how lamina destabilization processes may alter the NE circular economy. Moreover, we expand the knowledge of the lamina-associated domain region by using strategic algorithms, including Artificial Intelligence, to infer protein associations, assess their function and location, and predict cancer-type specificity with implications for the future of cancer diagnosis, prognosis and treatment. Using this approach we identified NUP98 and MECP2 as potential proteins that exhibit upregulation in Acute Myeloid Leukemia (LAML) patients with implications for early diagnosis.

WBC YOLO-ViT: 2 Way - 2 stage white blood cell detection and classification with a combination of YOLOv5 and vision transformer

Accurate detection and classification of white blood cells, otherwise known as leukocytes, play a critical role in diagnosing and monitoring various illnesses. However, conventional methods, such as manual classification by trained professionals, must be revised in terms of accuracy, efficiency, and potential bias. Moreover, applying deep learning techniques to detect and classify white blood cells using microscopic images is challenging owing to limited data, resolution noise, irregular shapes, and varying colors from different sources. This study presents a novel approach integrating object detection and classification for numerous type-white blood cell. We designed a 2-way approach to use two types of images: WBC and nucleus. YOLO (fast object detection) and ViT (powerful image representation capabilities) are effectively integrated into 16 classes. The proposed model demonstrates an exceptional 96.449% accuracy rate in classification.

Viable tumor cell density after neoadjuvant chemotherapy assessed using deep learning model reflects the prognosis of osteosarcoma

Prognosis after neoadjuvant chemotherapy (NAC) for osteosarcoma is generally predicted using manual necrosis-rate assessments; however, necrosis rates obtained in these assessments are not reproducible and do not adequately reflect individual cell responses. We aimed to investigate whether viable tumor cell density assessed using a deep-learning model (DLM) reflects the prognosis of osteosarcoma. Seventy-one patients were included in this study. Initially, the DLM was trained to detect viable tumor cells, following which it calculated their density. Patients were stratified into high and low-viable tumor cell density groups based on DLM measurements, and survival analysis was performed to evaluate disease-specific survival and metastasis-free survival (DSS and MFS). The high viable tumor cell density group exhibited worse DSS (p = 0.023) and MFS (p = 0.033). DLM-evaluated viable density showed correct stratification of prognosis groups. Therefore, this evaluation method may enable precise stratification of the prognosis in osteosarcoma patients treated with NAC.

Automated bone marrow cell classification through dual attention gates dense neural networks

PURPOSE

The morphology of bone marrow cells is essential in identifying malignant hematological disorders. The automatic classification model of bone marrow cell morphology based on convolutional neural networks shows considerable promise in terms of diagnostic efficiency and accuracy. However, due to the lack of acceptable accuracy in bone marrow cell classification algorithms, automatic classification of bone marrow cells is now infrequently used in clinical facilities. To address the issue of precision, in this paper, we propose a Dual Attention Gates DenseNet (DAGDNet) to construct a novel efficient, and high-precision bone marrow cell classification model for enhancing the classification model's performance even further.

METHODS

DAGDNet is constructed by embedding a novel dual attention gates (DAGs) mechanism in the architecture of DenseNet. DAGs are used to filter and highlight the position-related features in DenseNet to improve the precision and recall of neural network-based cell classifiers. We have constructed a dataset of bone marrow cell morphology from the First Affiliated Hospital of Chongqing Medical University, which mainly consists of leukemia samples, to train and test our proposed DAGDNet together with the bone marrow cell classification dataset.

RESULTS

When evaluated on a multi-center dataset, experimental results show that our proposed DAGDNet outperforms image classification models such as DenseNet and ResNeXt in bone marrow cell classification performance. The mean precision of DAGDNet on the Munich Leukemia Laboratory dataset is 88.1%, achieving state-of-the-art performance while still maintaining high efficiency.

CONCLUSION

Our data demonstrate that the DAGDNet can improve the efficacy of automatic bone marrow cell classification and can be exploited as an assisting diagnosis tool in clinical applications. Moreover, the DAGDNet is also an efficient model that can swiftly inspect a large number of bone marrow cells and offers the benefit of reducing the probability of an incorrect diagnosis.

Building Automation Pipeline for Diagnostic Classification of Sporadic Odontogenic Keratocysts and Non-Keratocysts Using Whole-Slide Images

The microscopic diagnostic differentiation of odontogenic cysts from other cysts is intricate and may cause perplexity for both clinicians and pathologists. Of particular interest is the odontogenic keratocyst (OKC), a developmental cyst with unique histopathological and clinical characteristics. Nevertheless, what distinguishes this cyst is its aggressive nature and high tendency for recurrence. Clinicians encounter challenges in dealing with this frequently encountered jaw lesion, as there is no consensus on surgical treatment. Therefore, the accurate and early diagnosis of such cysts will benefit clinicians in terms of treatment management and spare subjects from the mental agony of suffering from aggressive OKCs, which impact their quality of life. The objective of this research is to develop an automated OKC diagnostic system that can function as a decision support tool for pathologists, whether they are working locally or remotely. This system will provide them with additional data and insights to enhance their decision-making abilities. This research aims to provide an automation pipeline to classify whole-slide images of OKCs and non-keratocysts (non-KCs: dentigerous and radicular cysts). OKC diagnosis and prognosis using the histopathological analysis of tissues using whole-slide images (WSIs) with a deep-learning approach is an emerging research area. WSIs have the unique advantage of magnifying tissues with high resolution without losing information. The contribution of this research is a novel, deep-learning-based, and efficient algorithm that reduces the trainable parameters and, in turn, the memory footprint. This is achieved using principal component analysis (PCA) and the ReliefF feature selection algorithm (ReliefF) in a convolutional neural network (CNN) named P-C-ReliefF. The proposed model reduces the trainable parameters compared to standard CNN, achieving 97% classification accuracy.

Quantitative interpretation of bone marrow biopsies in MPN-What's the point in a molecular age?

The diagnosis of myeloproliferative neoplasms (MPN) requires the integration of clinical, morphological, genetic and immunophenotypic findings. Recently, there has been a transformation in our understanding of the cellular and molecular mechanisms underlying disease initiation and progression in MPN. This has been accompanied by the widespread application of high-resolution quantitative molecular techniques. By contrast, microscopic interpretation of bone marrow biopsies by haematologists/haematopathologists remains subjective and qualitative. However, advances in tissue image analysis and artificial intelligence (AI) promise to transform haematopathology. Pioneering studies in bone marrow image analysis offer to refine our understanding of the boundaries between reactive samples and MPN subtypes and better capture the morphological correlates of high-risk disease. They also demonstrate potential to improve the evaluation of current and novel therapeutics for MPN and other blood cancers. With increased therapeutic targeting of diverse molecular, cellular and extra-cellular components of the marrow, these approaches can address the unmet need for improved objective and quantitative measures of disease modification in the context of clinical trials. This review focuses on the state-of-the-art in image analysis/AI of bone marrow tissue, with an emphasis on its potential to complement and inform future clinical studies and research in MPN.

An Intelligent Attention-Based Transfer Learning Model for Accurate Differentiation of Bone Marrow Stains to Diagnose Hematological Disorder

Bone marrow (BM) is an essential part of the hematopoietic system, which generates all of the body's blood cells and maintains the body's overall health and immune system. The classification of bone marrow cells is pivotal in both clinical and research settings because many hematological diseases, such as leukemia, myelodysplastic syndromes, and anemias, are diagnosed based on specific abnormalities in the number, type, or morphology of bone marrow cells. There is a requirement for developing a robust deep-learning algorithm to diagnose bone marrow cells to keep a close check on them. This study proposes a framework for categorizing bone marrow cells into seven classes. In the proposed framework, five transfer learning models-DenseNet121, EfficientNetB5, ResNet50, Xception, and MobileNetV2-are implemented into the bone marrow dataset to classify them into seven classes. The best-performing DenseNet121 model was fine-tuned by adding one batch-normalization layer, one dropout layer, and two dense layers. The proposed fine-tuned DenseNet121 model was optimized using several optimizers, such as AdaGrad, AdaDelta, Adamax, RMSprop, and SGD, along with different batch sizes of 16, 32, 64, and 128. The fine-tuned DenseNet121 model was integrated with an attention mechanism to improve its performance by allowing the model to focus on the most relevant features or regions of the image, which can be particularly beneficial in medical imaging, where certain regions might have critical diagnostic information. The proposed fine-tuned and integrated DenseNet121 achieved the highest accuracy, with a training success rate of 99.97% and a testing success rate of 97.01%. The key hyperparameters, such as batch size, number of epochs, and different optimizers, were all considered for optimizing these pre-trained models to select the best model. This study will help in medical research to effectively classify the BM cells to prevent diseases like leukemia.

From WSI-level to patch-level: Structure prior-guided binuclear cell fine-grained detection

Accurate and quick binuclear cell (BC) detection plays a significant role in predicting the risk of leukemia and other malignant tumors. However, manual counting of BCs using microscope images is time consuming and subjective. Moreover, traditional image processing approaches perform poorly due to the limitations in staining quality and the diversity of morphological features in binuclear cell (BC) microscopy whole-slide images (WSIs). To overcome this challenge, we propose a multi-task method inspired by the structure prior of BCs based on deep learning, which cascades to implement BC coarse detection at the WSI level and fine-grained classification at the patch level. The coarse detection network is a multitask detection framework based on circular bounding boxes for cell detection and central key points for nucleus detection. Circle representation reduces the degrees of freedom, mitigates the effect of surrounding impurities compared to usual rectangular boxes and can be rotation invariant in WSIs. Detecting key points in the nucleus can assist in network perception and be used for unsupervised color layer segmentation in later fine-grained classification. The fine classification network consists of a background region suppression module based on color layer mask supervision and a key region selection module based on a transformer due to its global modeling capability. Additionally, an unsupervised and unpaired cytoplasm generator network is first proposed to expand the long-tailed distribution dataset. Finally, experiments are performed on BC multicenter datasets. The proposed BC fine detection method outperforms other benchmarks in almost all evaluation criteria, providing clarification and support for tasks such as cancer screenings.

Cell projection plots: A novel visualization of bone marrow aspirate cytology

Deep models for cell detection have demonstrated utility in bone marrow cytology, showing impressive results in terms of accuracy and computational efficiency. However, these models have yet to be implemented in the clinical diagnostic workflow. Additionally, the metrics used to evaluate cell detection models are not necessarily aligned with clinical goals and targets. In order to address these issues, we introduce novel, automatically generated visual summaries of bone marrow aspirate specimens called cell projection plots (CPPs). Encompassing relevant biological patterns such as neutrophil maturation, CPPs provide a compact summary of bone marrow aspirate cytology. To gauge clinical relevance, CPPs were inspected by 3 hematopathologists, who decided whether corresponding diagnostic synopses matched with generated CPPs. Pathologists were able to match CPPs to the correct synopsis with a matching degree of 85%. Our finding suggests CPPs can represent clinically relevant information from bone marrow aspirate specimens and may be used to efficiently summarize bone marrow cytology to pathologists. CPPs could be a step toward human-centered implementation of artificial intelligence (AI) in hematopathology, and a basis for a diagnostic-support tool for digital pathology workflows.

Analysis of cellularity in H&E-stained rat bone marrow tissue via deep learning

Our objective was to develop an automated deep-learning-based method to evaluate cellularity in rat bone marrow hematoxylin and eosin whole slide images for preclinical safety assessment. We trained a shallow CNN for segmenting marrow, 2 Mask R-CNN models for segmenting megakaryocytes (MKCs), and small hematopoietic cells (SHCs), and a SegNet model for segmenting red blood cells. We incorporated the models into a pipeline that identifies and counts MKCs and SHCs in rat bone marrow. We compared cell segmentation and counts that our method generated to those that pathologists generated on 10 slides with a range of cell depletion levels from 10 studies. For SHCs, we compared cell counts that our method generated to counts generated by Cellpose and Stardist. The median Dice and object Dice scores for MKCs using our method vs pathologist consensus and the inter- and intra-pathologist variation were comparable, with overlapping first-third quartile ranges. For SHCs, the median scores were close, with first-third quartile ranges partially overlapping intra-pathologist variation. For SHCs, in comparison to Cellpose and Stardist, counts from our method were closer to pathologist counts, with a smaller 95% limits of agreement range. The performance of the bone marrow analysis pipeline supports its incorporation into routine use as an aid for hematotoxicity assessment by pathologists. The pipeline could help expedite hematotoxicity assessment in preclinical studies and consequently could expedite drug development. The method may enable meta-analysis of rat bone marrow characteristics from future and historical whole slide images and may generate new biological insights from cross-study comparisons.

Detection of ERBB2 and CEN17 signals in fluorescent in situ hybridization and dual in situ hybridization for guiding breast cancer HER2 target therapy

The overexpression of the human epidermal growth factor receptor 2 (HER2) is a predictive biomarker in therapeutic effects for metastatic breast cancer. Accurate HER2 testing is critical for determining the most suitable treatment for patients. Fluorescent in situ hybridization (FISH) and dual in situ hybridization (DISH) have been recognized as FDA-approved methods to determine HER2 overexpression. However, analysis of HER2 overexpression is challenging. Firstly, the boundaries of cells are often unclear and blurry, with large variations in cell shapes and signals, making it challenging to identify the precise areas of HER2-related cells. Secondly, the use of sparsely labeled data, where some unlabeled HER2-related cells are classified as background, can significantly confuse fully supervised AI learning and result in unsatisfactory model outcomes. In this study, we present a weakly supervised Cascade R-CNN (W-CRCNN) model to automatically detect HER2 overexpression in HER2 DISH and FISH images acquired from clinical breast cancer samples. The experimental results demonstrate that the proposed W-CRCNN achieves excellent results in identification of HER2 amplification in three datasets, including two DISH datasets and a FISH dataset. For the FISH dataset, the proposed W-CRCNN achieves an accuracy of 0.970±0.022, precision of 0.974±0.028, recall of 0.917±0.065, F1-score of 0.943±0.042 and Jaccard Index of 0.899±0.073. For DISH datasets, the proposed W-CRCNN achieves an accuracy of 0.971±0.024, precision of 0.969±0.015, recall of 0.925±0.020, F1-score of 0.947±0.036 and Jaccard Index of 0.884±0.103 for dataset 1, and an accuracy of 0.978±0.011, precision of 0.975±0.011, recall of 0.918±0.038, F1-score of 0.946±0.030 and Jaccard Index of 0.884±0.052 for dataset 2, respectively. In comparison with the benchmark methods, the proposed W-CRCNN significantly outperforms all the benchmark approaches in identification of HER2 overexpression in FISH and DISH datasets (p<0.05). With the high degree of accuracy, precision and recall , the results show that the proposed method in DISH analysis for assessment of HER2 overexpression in breast cancer patients has significant potential to assist precision medicine.

A state-of-the-art survey of artificial neural networks for Whole-slide Image analysis: From popular Convolutional Neural Networks to potential visual transformers

In recent years, with the advancement of computer-aided diagnosis (CAD) technology and whole slide image (WSI), histopathological WSI has gradually played a crucial aspect in the diagnosis and analysis of diseases. To increase the objectivity and accuracy of pathologists' work, artificial neural network (ANN) methods have been generally needed in the segmentation, classification, and detection of histopathological WSI. However, the existing review papers only focus on equipment hardware, development status and trends, and do not summarize the art neural network used for full-slide image analysis in detail. In this paper, WSI analysis methods based on ANN are reviewed. Firstly, the development status of WSI and ANN methods is introduced. Secondly, we summarize the common ANN methods. Next, we discuss publicly available WSI datasets and evaluation metrics. These ANN architectures for WSI processing are divided into classical neural networks and deep neural networks (DNNs) and then analyzed. Finally, the application prospect of the analytical method in this field is discussed. The important potential method is Visual Transformers.

Digital assessment of peripheral blood and bone marrow aspirate smears

The diagnosis of benign and neoplastic hematologic disorders relies on analysis of peripheral blood and bone marrow aspirate smears. As demonstrated by the widespread laboratory adoption of hematology analyzers for automated assessment of peripheral blood, digital analysis of these samples provides many significant benefits compared to relying solely on manual review. Nonetheless, analogous instruments for digital bone marrow aspirate smear assessment have yet to be clinically implemented. In this review, we first provide a historical overview detailing the implementation of hematology analyzers for digital peripheral blood assessment in the clinical laboratory, including the improvements in accuracy, scope, and throughput of current instruments over prior generations. We also describe recent research in digital peripheral blood assessment, particularly in the development of advanced machine learning models that may soon be incorporated into commercial instruments. Next, we provide an overview of recent research in digital assessment of bone marrow aspirate smears and how these approaches could soon lead to development and clinical adoption of instrumentation for automated bone marrow aspirate smear analysis. Finally, we describe the relative advantages and provide our vision for the future of digital assessment of peripheral blood and bone marrow aspirate smears, including what improvements we can soon expect in the hematology laboratory.

Interpretable attention-based deep learning ensemble for personalized ovarian cancer treatment without manual annotations

Inhibition of pathological angiogenesis has become one of the first FDA approved targeted therapies widely tested in anti-cancer treatment, i.e. VEGF-targeting monoclonal antibody bevacizumab, in combination with chemotherapy for frontline and maintenance therapy for women with newly diagnosed ovarian cancer. Identification of the best predictive biomarkers of bevacizumab response is necessary in order to select patients most likely to benefit from this therapy. Hence, this study investigates the protein expression patterns on immunohistochemical whole slide images of three angiogenesis related proteins, including Vascular endothelial growth factor, Angiopoietin 2 and Pyruvate kinase isoform M2, and develops an interpretable and annotation-free attention based deep learning ensemble framework to predict the bevacizumab therapeutic effect on patients with epithelial ovarian cancer or peritoneal serous papillary carcinoma using tissue microarrays (TMAs). In evaluation with five-fold cross validation, the proposed ensemble model using the protein expressions of both Pyruvate kinase isoform M2 and Angiopoietin 2 achieves a notably high F-score (0.99±0.02), accuracy (0.99±0.03), precision (0.99±0.02), recall (0.99±0.02) and AUC (1.00±0). Kaplan-Meier progression free survival analysis confirms that the proposed ensemble is able to identify patients in the predictive therapeutic sensitive group with low cancer recurrence (p<0.001), and the Cox proportional hazards model analysis further confirms the above statement (p=0.012). In conclusion, the experimental results demonstrate that the proposed ensemble model using the protein expressions of both Pyruvate kinase isoform M2 and Angiopoietin 2 can assist treatment planning of bevacizumab targeted therapy for patients with ovarian cancer.

Mass Spectrometry Imaging for Single-Cell or Subcellular Lipidomics: A Review of Recent Advancements and Future Development

Mass Spectrometry Imaging (MSI) has emerged as a powerful imaging technique for the analysis of biological samples, providing valuable insights into the spatial distribution and structural characterization of lipids. The advancements in high-resolution MSI have made it an indispensable tool for single-cell or subcellular lipidomics. By preserving both intracellular and intercellular information, MSI enables a comprehensive analysis of lipidomics in individual cells and organelles. This enables researchers to delve deeper into the diversity of lipids within cells and to understand the role of lipids in shaping cell behavior. In this review, we aim to provide a comprehensive overview of recent advancements and future prospects of MSI for cellular/subcellular lipidomics. By keeping abreast of the cutting-edge studies in this field, we will continue to push the boundaries of the understanding of lipid metabolism and the impact of lipids on cellular behavior.

An Automated Pipeline for Differential Cell Counts on Whole-Slide Bone Marrow Aspirate Smears

The pathologic diagnosis of bone marrow disorders relies in part on the microscopic analysis of bone marrow aspirate (BMA) smears and the manual counting of marrow nucleated cells to obtain a differential cell count (DCC). This manual process has significant limitations, including the analysis of only a small subset of optimal slide areas and nucleated cells, as well as interobserver variability due to differences in cell selection and classification. To address these shortcomings, we developed an automated machine learning-based pipeline for obtaining 11-component DCCs on whole-slide BMAs. This pipeline uses a sequential process of identifying optimal BMA regions with high proportions of marrow nucleated cells, detecting individual cells within these optimal areas, and classifying these cells into 1 of 11 DCC components. Convolutional neural network models were trained on 396,048 BMA region, 28,914 cell boundary, and 1,510,976 cell class images from manual annotations. The resulting automated pipeline produced 11-component DCCs that demonstrated a high statistical and diagnostic concordance with manual DCCs among a heterogeneous group of testing BMA slides with varying pathologies and cellularities. Additionally, we demonstrated that an automated analysis can reduce the intraslide variance in DCCs by analyzing the whole slide and marrow nucleated cells within all optimal regions. Finally, the pipeline outputs of region classification, cell detection, and cell classification can be visualized using whole-slide image analysis software. This study demonstrates the feasibility of a fully automated pipeline for generating DCCs on scanned whole-slide BMA images, with the potential for improving the current standard of practice for utilizing BMA smears in the laboratory analysis of hematologic disorders.

An artificial intelligence model for the pathological diagnosis of invasion depth and histologic grade in bladder cancer

BACKGROUND

Accurate pathological diagnosis of invasion depth and histologic grade is key for clinical management in patients with bladder cancer (BCa), but it is labour-intensive, experience-dependent and subject to interobserver variability. Here, we aimed to develop a pathological artificial intelligence diagnostic model (PAIDM) for BCa diagnosis.

METHODS

A total of 854 whole slide images (WSIs) from 692 patients were included and divided into training and validation sets. The PAIDM was developed using the training set based on the deep learning algorithm ScanNet, and the performance was verified at the patch level in validation set 1 and at the WSI level in validation set 2. An independent validation cohort (validation set 3) was employed to compare the PAIDM and pathologists. Model performance was evaluated using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value and negative predictive value.

RESULTS

The AUCs of the PAIDM were 0.878 (95% CI 0.875-0.881) at the patch level in validation set 1 and 0.870 (95% CI 0.805-0.923) at the WSI level in validation set 2. In comparing the PAIDM and pathologists, the PAIDM achieved an AUC of 0.847 (95% CI 0.779-0.905), which was non-inferior to the average diagnostic level of pathologists. There was high consistency between the model-predicted and manually annotated areas, improving the PAIDM's interpretability.

CONCLUSIONS

We reported an artificial intelligence-based diagnostic model for BCa that performed well in identifying invasion depth and histologic grade. Importantly, the PAIDM performed admirably in patch-level recognition, with a promising application for transurethral resection specimens.

Automated Bone Marrow Cell Classification for Haematological Disease Diagnosis Using Siamese Neural Network

The critical structure and nature of different bone marrow cells which form a base in the diagnosis of haematological ailments requires a high-grade classification which is a very prolonged approach and accounts for human error if performed manually, even by field experts. Therefore, the aim of this research is to automate the process to study and accurately classify the structure of bone marrow cells which will help in the diagnosis of haematological ailments at a much faster and better rate. Various machine learning algorithms and models, such as CNN + SVM, CNN + XGB Boost and Siamese network, were trained and tested across a dataset of 170,000 expert-annotated cell images from 945 patients' bone marrow smears with haematological disorders. The metrics used for evaluation of this research are accuracy of model, precision and recall of all the different classes of cells. Based on these performance metrics the CNN + SVM, CNN + XGB, resulted in 32%, 28% accuracy, respectively, and therefore these models were discarded. Siamese neural resulted in 91% accuracy and 84% validation accuracy. Moreover, the weighted average recall values of the Siamese neural network were 92% for training and 91% for validation. Hence, the final results are based on Siamese neural network model as it was outperforming all the other algorithms used in this research.

Deep learning application of the discrimination of bone marrow aspiration cells in patients with myelodysplastic syndromes

Myelodysplastic syndromes (MDS) are a group of hematologic neoplasms accompanied by dysplasia of the bone marrow hematopoietic cells with cytopenia. Detecting dysplasia is important in the diagnosis of MDS, but it takes considerable time and effort. Also, since the assessment of dysplasia is subjective and difficult to quantify, a more efficient tool is needed for quality control and standardization of bone marrow aspiration smear interpretation. In this study, we developed and evaluated an algorithm to automatically discriminate hematopoietic cell lineages and detect dysplastic cells in bone marrow aspiration smears using deep learning technology. Bone marrow aspiration images were acquired from 34 patients diagnosed with MDS and from 24 normal bone marrow slides. In total, 8065 cells were classified into eight categories: normal erythrocytes, normal granulocytes, normal megakaryocytes, dysplastic erythrocytes, dysplastic granulocytes, dysplastic megakaryocytes, blasts, and others. The algorithm demonstrated acceptable performance in classifying dysplastic cells, with an AUC of 0.945-0.996 and accuracy of 0.912-0.993. The algorithm developed in this study could be used as an auxiliary tool for diagnosing patients with MDS and is expected to contribute to shortening the time required for MDS bone marrow aspiration diagnosis and standardizing visual reading.

A Soft Label Deep Learning to Assist Breast Cancer Target Therapy and Thyroid Cancer Diagnosis

According to the World Health Organization Report 2022, cancer is the most common cause of death contributing to nearly one out of six deaths worldwide. Early cancer diagnosis and prognosis have become essential in reducing the mortality rate. On the other hand, cancer detection is a challenging task in cancer pathology. Trained pathologists can detect cancer, but their decisions are subjective to high intra- and inter-observer variability, which can lead to poor patient care owing to false-positive and false-negative results. In this study, we present a soft label fully convolutional network (SL-FCN) to assist in breast cancer target therapy and thyroid cancer diagnosis, using four datasets. To aid in breast cancer target therapy, the proposed method automatically segments human epidermal growth factor receptor 2 (HER2) amplification in fluorescence in situ hybridization (FISH) and dual in situ hybridization (DISH) images. To help in thyroid cancer diagnosis, the proposed method automatically segments papillary thyroid carcinoma (PTC) on Papanicolaou-stained fine needle aspiration and thin prep whole slide images (WSIs). In the evaluation of segmentation of HER2 amplification in FISH and DISH images, we compare the proposed method with thirteen deep learning approaches, including U-Net, U-Net with InceptionV5, Ensemble of U-Net with Inception-v4, Inception-Resnet-v2 encoder, and ResNet-34 encoder, SegNet, FCN, modified FCN, YOLOv5, CPN, SOLOv2, BCNet, and DeepLabv3+ with three different backbones, including MobileNet, ResNet, and Xception, on three clinical datasets, including two DISH datasets on two different magnification levels and a FISH dataset. The result on DISH breast dataset 1 shows that the proposed method achieves high accuracy of 87.77 ± 14.97%, recall of 91.20 ± 7.72%, and F1-score of 81.67 ± 17.76%, while, on DISH breast dataset 2, the proposed method achieves high accuracy of 94.64 ± 2.23%, recall of 83.78 ± 6.42%, and F1-score of 85.14 ± 6.61% and, on the FISH breast dataset, the proposed method achieves high accuracy of 93.54 ± 5.24%, recall of 83.52 ± 13.15%, and F1-score of 86.98 ± 9.85%, respectively. Furthermore, the proposed method outperforms most of the benchmark approaches by a significant margin (p <0.001). In evaluation of segmentation of PTC on Papanicolaou-stained WSIs, the proposed method is compared with three deep learning methods, including Modified FCN, U-Net, and SegNet. The experimental result demonstrates that the proposed method achieves high accuracy of 99.99 ± 0.01%, precision of 92.02 ± 16.6%, recall of 90.90 ± 14.25%, and F1-score of 89.82 ± 14.92% and significantly outperforms the baseline methods, including U-Net and FCN (p <0.001). With the high degree of accuracy, precision, and recall, the results show that the proposed method could be used in assisting breast cancer target therapy and thyroid cancer diagnosis with faster evaluation and minimizing human judgment errors.

Deep Learning Using Endobronchial-Ultrasound-Guided Transbronchial Needle Aspiration Image to Improve the Overall Diagnostic Yield of Sampling Mediastinal Lymphadenopathy

Lung cancer is the biggest cause of cancer-related death worldwide. An accurate nodal staging is critical for the determination of treatment strategy for lung cancer patients. Endobronchial-ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) has revolutionized the field of pulmonology and is considered to be extremely sensitive, specific, and secure for lung cancer staging through rapid on-site evaluation (ROSE), but manual visual inspection on the entire slide of EBUS smears is challenging, time consuming, and worse, subjective, on a large interobserver scale. To satisfy ROSE’s needs, a rapid, automated, and accurate diagnosis system using EBUS-TBNA whole-slide images (WSIs) is highly desired to improve diagnosis accuracy and speed, minimize workload and labor costs, and ensure reproducibility. We present a fast, efficient, and fully automatic deep-convolutional-neural-network-based system for advanced lung cancer staging on gigapixel EBUS-TBNA cytological WSIs. Each WSI was converted into a patch-based hierarchical structure and examined by the proposed deep convolutional neural network, generating the segmentation of metastatic lesions in EBUS-TBNA WSIs. To the best of the authors’ knowledge, this is the first research on fully automated enlarged mediastinal lymph node analysis using EBUS-TBNA cytological WSIs. We evaluated the robustness of the proposed framework on a dataset of 122 WSIs, and the proposed method achieved a high precision of 93.4%, sensitivity of 89.8%, DSC of 82.2%, and IoU of 83.2% for the first experiment (37.7% training and 62.3% testing) and a high precision of 91.8 ± 1.2, sensitivity of 96.3 ± 0.8, DSC of 94.0 ± 1.0, and IoU of 88.7 ± 1.8 for the second experiment using a three-fold cross-validation, respectively. Furthermore, the proposed method significantly outperformed the three state-of-the-art baseline models, including U-Net, SegNet, and FCN, in terms of precision, sensitivity, DSC, and Jaccard index, based on Fisher’s least significant difference (LSD) test (p<0.001). For a computational time comparison on a WSI, the proposed method was 2.5 times faster than U-Net, 2.3 times faster than SegNet, and 3.4 times faster than FCN, using a single GeForce GTX 1080 Ti, respectively. With its high precision and sensitivity, the proposed method demonstrated that it manifested the potential to reduce the workload of pathologists in their routine clinical practice.

Digital morphology in hematology diagnosis and education: The experience of the European LeukemiaNet WP10

Hematological diagnostics is based on increasingly precise techniques of cellular and molecular analysis. The correct interpretation of the blood and bone marrow smears observed under an optical microscope still represents a cornerstone. Precise quantitative and qualitative cytomorphological criteria have recently been codified by up-to-date guidelines for diagnosing hematopoietic neoplasms. Morphological analysis has found formidable support in digital reproduction techniques, which have simplified the circulation of images for educational or consultation purposes. From 2007 to 2019, the Working Group WP10 of European LeukemiaNet (ELN) used, in annual exercises, digital images to support training in cytomorphology and verify harmonization and comparability in the interpretation of blood and bone marrow smears. We describe the design, development, and results of this program, which had 741 participants in-person or remotely, to which 2055 questions were submitted regarding the interpretation of cytomorphological images. We initially used circulation and presentation of digital microphotographs and then introduced a virtual microscopy (VM). Virtual slides were obtained using a whole slide imaging technique, similar to the one largely used in histopathology, to produce digitized scans of consecutive microscopic fields and reassembles them to obtain a complete virtual smear by stitching. Participants were required to identify cells in labeled fields of view of the virtual slides to obtain a morphological diagnosis. This work has demonstrated substantial improvements in diagnostic accuracy and harmonization with the VM technique. Between-observer concordance increased from 62.5% to 83.0%. The integrity of the digitalized film image, which provides a general context for cell abnormalities, was the main factor for this outcome.

Efficient and Highly Accurate Diagnosis of Malignant Hematological Diseases Based on Whole-Slide Images Using Deep Learning

Hematopoietic disorders are serious diseases that threaten human health, and the diagnosis of these diseases is essential for treatment. However, traditional diagnosis methods rely on manual operation, which is time consuming and laborious, and examining entire slide is challenging. In this study, we developed a weakly supervised deep learning method for diagnosing malignant hematological diseases requiring only slide-level labels. The method improves efficiency by converting whole-slide image (WSI) patches into low-dimensional feature representations. Then the patch-level features of each WSI are aggregated into slide-level representations by an attention-based network. The model provides final diagnostic predictions based on these slide-level representations. By applying the proposed model to our collection of bone marrow WSIs at different magnifications, we found that an area under the receiver operating characteristic curve of 0.966 on an independent test set can be obtained at 10× magnification. Moreover, the performance on microscopy images can achieve an average accuracy of 94.2% on two publicly available datasets. In conclusion, we have developed a novel method that can achieve fast and accurate diagnosis in different scenarios of hematological disorders.

A Survey on Deep Learning for Precision Oncology

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Synthesis of Microscopic Cell Images Obtained from Bone Marrow Aspirate Smears through Generative Adversarial Networks

Simple Summary

This paper proposes a hybrid generative adversarial networks model—WGAN-GP-AC—to generate synthetic microscopic cell images. We generate the synthetic data for the cell types containing fewer data to obtain a balanced dataset. A balanced dataset would help enhance the classification accuracy of each cell type and help with an easy and quick diagnosis that is critical for leukemia patients. In this work, we combine images from three datasets to form a single concrete dataset with variations of multiple microscopic cell images. We provide experimental results that prove the correlation between the original and our synthetically generated data. We also deliver classification results to showcase that the generated synthetic data can be used for real-life experiments and the advancement of the medical domain.

Abstract

Every year approximately 1.24 million people are diagnosed with blood cancer. While the rate increases each year, the availability of data for each kind of blood cancer remains scarce. It is essential to produce enough data for each blood cell type obtained from bone marrow aspirate smears to diagnose rare types of cancer. Generating data would help easy and quick diagnosis, which are the most critical factors in cancer. Generative adversarial networks (GAN) are the latest emerging framework for generating synthetic images and time-series data. This paper takes microscopic cell images, preprocesses them, and uses a hybrid GAN architecture to generate synthetic images of the cell types containing fewer data. We prepared a single dataset with expert intervention by combining images from three different sources. The final dataset consists of 12 cell types and has 33,177 microscopic cell images. We use the discriminator architecture of auxiliary classifier GAN (AC-GAN) and combine it with the Wasserstein GAN with gradient penalty model (WGAN-GP). We name our model as WGAN-GP-AC. The discriminator in our proposed model works to identify real and generated images and classify every image with a cell type. We provide experimental results demonstrating that our proposed model performs better than existing individual and hybrid GAN models in generating microscopic cell images. We use the generated synthetic data with classification models, and the results prove that the classification rate increases significantly. Classification models achieved 0.95 precision and 0.96 recall value for synthetic data, which is higher than the original, augmented, or combined datasets.

A deep learning method and device for bone marrow imaging cell detection

BACKGROUND

Morphological analysis of bone marrow cells is considered as the gold standard for the diagnosis of leukemia. However, due to the diverse morphology of bone marrow cells, extensive experience and patience are needed for morphological examination. automatic diagnosis system through the comprehensive application of image analysis and pattern recognition technology is urgently needed to reduce work intensity, error probability and improves work efficiency.

METHODS

In this article, we establish a new morphological diagnosis system for bone marrow cell detection based on the deep learning object detection framework. The model is based on the Faster Region-Convolutional Neural Network (R-CNN), a classical object detection model. The system automatically detects bone marrow cells and determines their types. As specimens have severe long-tail distribution, i.e., the frequency of different types of cells varies dramatically, we proposed a general score ranking loss to solve such a problem. The general score ranking loss considers the ranking relationship between positive and negative samples and optimizes the positive sample with a higher classification probability value.

RESULTS

We verified this system with 70 bone marrow specimens of leukemia patients, which proved that it can realize intelligent recognition with high efficiency. The software is finally integrated into the microscope system to build an augmented reality system.

CONCLUSIONS

Clinical tests show that the response speed of the newly developed diagnostic system is faster than that of trained diagnostic experts.