Omni-Seg: A Scale-Aware Dynamic Network for Renal Pathological Image Segmentation

Advances in kidney biopsy lesion assessment through dense instance segmentation

Renal biopsies are the gold standard for the diagnosis of kidney diseases. Lesion scores made by renal pathologists are semi-quantitative and exhibit high inter-observer variability. Automating lesion classification within segmented anatomical structures can provide decision support in quantification analysis, thereby reducing inter-observer variability. Nevertheless, classifying lesions in regions-of-interest (ROIs) is clinically challenging due to (a) a large amount of densely packed anatomical objects, (b) class imbalance across different compartments (at least 3), (c) significant variation in size and shape of anatomical objects and (d) the presence of multi-label lesions per anatomical structure. Existing models cannot address these complexities in an efficient and generic manner. This paper presents an analysis for a generalized solution to datasets from various sources (pathology departments) with different types of lesions. Our approach utilizes two sub-networks: dense instance segmentation and lesion classification. We introduce DiffRegFormer, an end-to-end dense instance segmentation sub-network designed for multi-class, multi-scale objects within ROIs. Combining diffusion models, transformers, and RCNNs, DiffRegFormer is a computational-friendly framework that can efficiently recognize over 500 objects across three anatomical classes, i.e., glomeruli, tubuli, and arteries, within ROIs. In a dataset of 303 ROIs from 148 Jones' silver-stained renal Whole Slide Images (WSIs), our approach outperforms previous methods, achieving an Average Precision of 52.1% (detection) and 46.8% (segmentation). Moreover, our lesion classification sub-network achieves 89.2% precision and 64.6% recall on 21889 object patches out of the 303 ROIs. Lastly, our model demonstrates direct domain transfer to PAS-stained renal WSIs without fine-tuning.

Explainable attention-enhanced heuristic paradigm for multi-view prognostic risk score development in hepatocellular carcinoma

PURPOSE

Existing prognostic staging systems depend on expensive manual extraction by pathologists, potentially overlooking latent patterns critical for prognosis, or use black-box deep learning models, limiting clinical acceptance. This study introduces a novel deep learning-assisted paradigm that complements existing approaches by generating interpretable, multi-view risk scores to stratify prognostic risk in hepatocellular carcinoma (HCC) patients.

METHODS

510 HCC patients were enrolled in an internal dataset (SYSUCC) as training and validation cohorts to develop the Hybrid Deep Score (HDS). The Attention Activator (ATAT) was designed to heuristically identify tissues with high prognostic risk, and a multi-view risk-scoring system based on ATAT established HDS from microscopic to macroscopic levels. HDS was also validated on an external testing cohort (TCGA-LIHC) with 341 HCC patients. We assessed prognostic significance using Cox regression and the concordance index (c-index).

RESULTS

The ATAT first heuristically identified regions where necrosis, lymphocytes, and tumor tissues converge, particularly focusing on their junctions in high-risk patients. From this, this study developed three independent risk factors: microscopic morphological, co-localization, and deep global indicators, which were concatenated and then input into a neural network to generate the final HDS for each patient. The HDS demonstrated competitive results with hazard ratios (HR) (HR 3.24, 95% confidence interval (CI) 1.91-5.43 in SYSUCC; HR 2.34, 95% CI 1.58-3.47 in TCGA-LIHC) and c-index values (0.751 in SYSUCC; 0.729 in TCGA-LIHC) for Disease-Free Survival (DFS). Furthermore, integrating HDS into existing clinical staging systems allows for more refined stratification, which enables the identification of potential high-risk patients within low-risk groups.

CONCLUSION

This novel paradigm, from identifying high-risk tissues to constructing prognostic risk scores, offers fresh insights into HCC research. Additionally, the integration of HDS complements the existing clinical staging system by facilitating more detailed stratification in DFS and Overall Survival (OS).

Glo-net: A dual task branch based neural network for multi-class glomeruli segmentation

Accurate segmentation and classification of glomeruli are fundamental to histopathology slide analysis in renal pathology, which helps to characterize individual kidney disease. Accurate segmentation of glomeruli of different types faces two main challenges compared to traditional primitives segmentation in computational image analysis. Limited by small kernel size, traditional convolutional neural networks could hardly understand the complete context information of different glomeruli. Moreover, typical semantic segmentation networks lack adequate attention to difficult glomerular samples during the training process due to serious class imbalance between different glomeruli types. We propose a new deep learning approach, Glo-Net, which accurately segments and classifies glomeruli based on digitized pathology slides. Specifically, Glo-Net divides the traditional semantic segmentation network into two branches, i.e., segmentation and classification. While the segmentation branch specifically aims at localizing and delineating the boundary of individual glomerulus, the classification branch could focus on differentiating the glomerular types based on segmented pixels. In addition, an innovative loss function is added to the classification task to compensate for the class imbalance and minor types of glomeruli. The proposed network's average accuracy and F-score in classification tasks on the multi-institution datasets (including an external validation set) are 0.858 and 0.704, respectively. The average intersection over union (IoU) in segmentation tasks is 0.866. The Glo-Net demonstrates a 5 % improvement in classification accuracy, with up to 14 % increases for minor classes and an average 6 % IoU increase for segmentation tasks. Quantitative results show that our network achieves overall higher accuracy for segmentation and classification among nine subtypes of glomeruli compared to previous work with improved robustness and generalizability.

Learnable Context in Multiple Instance Learning for Whole Slide Image Classification and Segmentation

Multiple instance learning (MIL) has become a cornerstone in whole slide image (WSI) analysis. In this paradigm, a WSI is conceptualized as a bag of instances. Instance features are extracted by a feature extractor, and then a feature aggregator fuses these instance features into a bag representation. In this paper, we advocate that both feature extraction and aggregation can be enhanced by considering the context or correlation between instances. We learn contextual features between instances, and then fuse contextual features with instance features to enhance instance representations. For feature aggregation, we observe performance instability particularly when disease-positive instances are only a minor fraction of the WSI. We introduce a self-attention mechanism to discover correlation among instances and foster more effective bag representations. Through comprehensive testing, we have demonstrated that the proposed method outperforms existing WSI classification methods by 1 to 4% classification accuracy, based on the Camelyon16 and the TCGA-NSCLC datasets. The proposed method also outperforms the most recent weakly supervised WSI segmentation method by 0.6 in terms of the Dice coefficient, based on the Camelyon16 dataset.

HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis

Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel Hierarchical Adaptive Taxonomy Segmentation (HATs) method, which is designed to thoroughly segment panoramic views of kidney structures by leveraging detailed anatomical insights. Our approach entails (1) the innovative HATs technique which translates spatial relationships among 15 distinct object classes into a versatile "plug-and-play" loss function that spans across regions, functional units, and cells, (2) the incorporation of anatomical hierarchies and scale considerations into a unified simple matrix representation for all panoramic entities, (3) the adoption of the latest AI foundation model (EfficientSAM) as a feature extraction tool to boost the model's adaptability, yet eliminating the need for manual prompt generation in conventional segment anything model (SAM). Experimental findings demonstrate that the HATs method offers an efficient and effective strategy for integrating clinical insights and imaging precedents into a unified segmentation model across more than 15 categories. The official implementation is publicly available at https://github.com/hrlblab/HATs.

NMGrad: Advancing Histopathological Bladder Cancer Grading with Weakly Supervised Deep Learning

The most prevalent form of bladder cancer is urothelial carcinoma, characterized by a high recurrence rate and substantial lifetime treatment costs for patients. Grading is a prime factor for patient risk stratification, although it suffers from inconsistencies and variations among pathologists. Moreover, absence of annotations in medical imaging renders it difficult to train deep learning models. To address these challenges, we introduce a pipeline designed for bladder cancer grading using histological slides. First, it extracts urothelium tissue tiles at different magnification levels, employing a convolutional neural network for processing for feature extraction. Then, it engages in the slide-level prediction process. It employs a nested multiple-instance learning approach with attention to predict the grade. To distinguish different levels of malignancy within specific regions of the slide, we include the origins of the tiles in our analysis. The attention scores at region level are shown to correlate with verified high-grade regions, giving some explainability to the model. Clinical evaluations demonstrate that our model consistently outperforms previous state-of-the-art methods, achieving an F1 score of 0.85.

PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation

Understanding the anatomy of renal pathology is crucial for advancing disease diagnostics, treatment evaluation, and clinical research. The complex kidney system comprises various components across multiple levels, including regions (cortex, medulla), functional units (glomeruli, tubules), and cells (podocytes, mesangial cells in glomerulus). Prior studies have predominantly overlooked the intricate spatial interrelations among objects from clinical knowledge. In this research, we introduce a novel universal proposition learning approach, called panoramic renal pathology segmentation (PrPSeg), designed to segment comprehensively panoramic structures within kidney by integrating extensive knowledge of kidney anatomy. In this paper, we propose (1) the design of a comprehensive universal proposition matrix for renal pathology, facilitating the incorporation of classification and spatial relationships into the segmentation process; (2) a token-based dynamic head single network architecture, with the improvement of the partial label image segmentation and capability for future data enlargement; and (3) an anatomy loss function, quantifying the inter-object relationships across the kidney.

COSST: Multi-Organ Segmentation With Partially Labeled Datasets Using Comprehensive Supervisions and Self-Training

Deep learning models have demonstrated remarkable success in multi-organ segmentation but typically require large-scale datasets with all organs of interest annotated. However, medical image datasets are often low in sample size and only partially labeled, i.e., only a subset of organs are annotated. Therefore, it is crucial to investigate how to learn a unified model on the available partially labeled datasets to leverage their synergistic potential. In this paper, we systematically investigate the partial-label segmentation problem with theoretical and empirical analyses on the prior techniques. We revisit the problem from a perspective of partial label supervision signals and identify two signals derived from ground truth and one from pseudo labels. We propose a novel two-stage framework termed COSST, which effectively and efficiently integrates comprehensive supervision signals with self-training. Concretely, we first train an initial unified model using two ground truth-based signals and then iteratively incorporate the pseudo label signal to the initial model using self-training. To mitigate performance degradation caused by unreliable pseudo labels, we assess the reliability of pseudo labels via outlier detection in latent space and exclude the most unreliable pseudo labels from each self-training iteration. Extensive experiments are conducted on one public and three private partial-label segmentation tasks over 12 CT datasets. Experimental results show that our proposed COSST achieves significant improvement over the baseline method, i.e., individual networks trained on each partially labeled dataset. Compared to the state-of-the-art partial-label segmentation methods, COSST demonstrates consistent superior performance on various segmentation tasks and with different training data sizes.

A Comprehensive Look at In Vitro Angiogenesis Image Analysis Software

One of the complex challenges faced presently by tissue engineering (TE) is the development of vascularized constructs that accurately mimic the extracellular matrix (ECM) of native tissue in which they are inserted to promote vessel growth and, consequently, wound healing and tissue regeneration. TE technique is characterized by several stages, starting from the choice of cell culture and the more appropriate scaffold material that can adequately support and supply them with the necessary biological cues for microvessel development. The next step is to analyze the attained microvasculature, which is reliant on the available labeling and microscopy techniques to visualize the network, as well as metrics employed to characterize it. These are usually attained with the use of software, which has been cited in several works, although no clear standard procedure has been observed to promote the reproduction of the cell response analysis. The present review analyzes not only the various steps previously described in terms of the current standards for evaluation, but also surveys some of the available metrics and software used to quantify networks, along with the detection of analysis limitations and future improvements that could lead to considerable progress for angiogenesis evaluation and application in TE research.

Artificial Intelligence Assists in the Detection of Blood Vessels in Whole Slide Images: Practical Benefits for Oncological Pathology

The analysis of the microvasculature and the assessment of angiogenesis have significant prognostic value in various diseases, including cancer. The search for invasion into the blood and lymphatic vessels and the assessment of angiogenesis are important aspects of oncological diagnosis. These features determine the prognosis and aggressiveness of the tumor. Traditional manual evaluation methods are time consuming and subject to inter-observer variability. Blood vessel detection is a perfect task for artificial intelligence, which is capable of rapid analyzing thousands of tissue structures in whole slide images. The development of computer vision solutions requires the segmentation of tissue regions, the extraction of features and the training of machine learning models. In this review, we focus on the methodologies employed by researchers to identify blood vessels and vascular invasion across a range of tumor localizations, including breast, lung, colon, brain, renal, pancreatic, gastric and oral cavity cancers. Contemporary models herald a new era of computational pathology in morphological diagnostics.