Automated analysis of immunohistochemistry images identifies candidate location biomarkers for cancers

Differentially localized protein identification for breast cancer based on deep learning in immunohistochemical images

The mislocalization of proteins leads to breast cancer, one of the world's most prevalent cancers, which can be identified from immunohistochemical images. Here, based on the deep learning framework, location prediction models were constructed using the features of breast immunohistochemical images. Ultimately, six differentially localized proteins that with stable differentially predictive localization, maximum localization differences, and whose predicted results are not affected by removing a single image are obtained (CCNT1, NSUN5, PRPF4, RECQL4, UTP6, ZNF500). Further verification reveals that these proteins are not differentially expressed, but are closely associated with breast cancer and have great classification performance. Potential mechanism analysis shows that their co-expressed or co-located proteins and RNAs may affect their localization, leading to changes in interactions and functions that further causes breast cancer. They have the potential to help shed light on the molecular mechanisms of breast cancer and provide assistance for its early diagnosis and treatment.

Vislocas: Vision transformers for identifying protein subcellular mis-localization signatures of different cancer subtypes from immunohistochemistry images

Proteins must be sorted to specific subcellular compartments to perform their functions. Abnormal protein subcellular localizations are related to many diseases. Although many efforts have been made in predicting protein subcellular localization from various static information, including sequences, structures and interactions, such static information cannot predict protein mis-localization events in diseases. On the contrary, the IHC (immunohistochemistry) images, which have been widely applied in clinical diagnosis, contains information that can be used to find protein mis-localization events in disease states. In this study, we create the Vislocas method, which is capable of finding mis-localized proteins from IHC images as markers of cancer subtypes. By combining CNNs and vision transformer encoders, Vislocas can automatically extract image features at both global and local level. Vislocas can be trained with full-sized IHC images from scratch. It is the first attempt to create an end-to-end IHC image-based protein subcellular location predictor. Vislocas achieved comparable or better performances than state-of-the-art methods. We applied Vislocas to find significant protein mis-localization events in different subtypes of glioma, melanoma and skin cancer. The mis-localized proteins, which were found purely from IHC images by Vislocas, are in consistency with clinical or experimental results in literatures. All codes of Vislocas have been deposited in a Github repository (https://github.com/JingwenWen99/Vislocas). All datasets of Vislocas have been deposited in Zenodo (https://zenodo.org/records/10632698).

HAR_Locator: a novel protein subcellular location prediction model of immunohistochemistry images based on hybrid attention modules and residual units

Introduction: Proteins located in subcellular compartments have played an indispensable role in the physiological function of eukaryotic organisms. The pattern of protein subcellular localization is conducive to understanding the mechanism and function of proteins, contributing to investigating pathological changes of cells, and providing technical support for targeted drug research on human diseases. Automated systems based on featurization or representation learning and classifier design have attracted interest in predicting the subcellular location of proteins due to a considerable rise in proteins. However, large-scale, fine-grained protein microscopic images are prone to trapping and losing feature information in the general deep learning models, and the shallow features derived from statistical methods have weak supervision abilities. Methods: In this work, a novel model called HAR_Locator was developed to predict the subcellular location of proteins by concatenating multi-view abstract features and shallow features, whose advanced advantages are summarized in the following three protocols. Firstly, to get discriminative abstract feature information on protein subcellular location, an abstract feature extractor called HARnet based on Hybrid Attention modules and Residual units was proposed to relieve gradient dispersion and focus on protein-target regions. Secondly, it not only improves the supervision ability of image information but also enhances the generalization ability of the HAR_Locator through concatenating abstract features and shallow features. Finally, a multi-category multi-classifier decision system based on an Artificial Neural Network (ANN) was introduced to obtain the final output results of samples by fitting the most representative result from five subset predictors. Results: To evaluate the model, a collection of 6,778 immunohistochemistry (IHC) images from the Human Protein Atlas (HPA) database was used to present experimental results, and the accuracy, precision, and recall evaluation indicators were significantly increased to 84.73%, 84.77%, and 84.70%, respectively, compared with baseline predictors.

A review from biological mapping to computation-based subcellular localization

Subcellular localization is crucial to the study of virus and diseases. Specifically, research on protein subcellular localization can help identify clues between virus and host cells that can aid in the design of targeted drugs. Research on RNA subcellular localization is significant for human diseases (such as Alzheimer's disease, colon cancer, etc.). To date, only reviews addressing subcellular localization of proteins have been published, which are outdated for reference, and reviews of RNA subcellular localization are not comprehensive. Therefore, we collated (the most up-to-date) literature on protein and RNA subcellular localization to help researchers understand changes in the field of protein and RNA subcellular localization. Extensive and complete methods for constructing subcellular localization models have also been summarized, which can help readers understand the changes in application of biotechnology and computer science in subcellular localization research and explore how to use biological data to construct improved subcellular localization models. This paper is the first review to cover both protein subcellular localization and RNA subcellular localization. We urge researchers from biology and computational biology to jointly pay attention to transformation patterns, interrelationships, differences, and causality of protein subcellular localization and RNA subcellular localization.

PLA-GNN: Computational inference of protein subcellular location alterations under drug treatments with deep graph neural networks

The aberrant protein sorting has been observed in many conditions, including complex diseases, drug treatments, and environmental stresses. It is important to systematically identify protein mis-localization events in a given condition. Experimental methods for finding mis-localized proteins are always costly and time consuming. Predicting protein subcellular localizations has been studied for many years. However, only a handful of existing works considered protein subcellular location alterations. We proposed a computational method for identifying alterations of protein subcellular locations under drug treatments. We took three drugs, including TSA (trichostain A), bortezomib and tacrolimus, as instances for this study. By introducing dynamic protein-protein interaction networks, graph neural network algorithms were applied to aggregate topological information under different conditions. We systematically reported potential protein mis-localization events under drug treatments. As far as we know, this is the first attempt to find protein mis-localization events computationally in drug treatment conditions. Literatures validated that a number of proteins, which are highly related to pharmacological mechanisms of these drugs, may undergo protein localization alterations. We name our method as PLA-GNN (Protein Localization Alteration by Graph Neural Networks). It can be extended to other drugs and other conditions. All datasets and codes of this study has been deposited in a GitHub repository (https://github.com/quinlanW/PLA-GNN).

Tensor Decomposition of Largest Convolutional Eigenvalues Reveals Pathologic Predictive Power of RhoB in Rectal Cancer Biopsy

RhoB protein belongs to the Rho GTPase family, which plays an important role in governing cell signaling and tissue morphology. RhoB expression is known to have implications in pathologic processes of diseases. Investigation in the regulation and communication of this protein, detected by immunohistochemical staining on the microscope, is worth exploring to gain insightful information that may lead to identifying optimal disease treatment options. In particular, the role of RhoB in rectal cancer is not well discovered. Here, we report that methods of deep learning-based image analysis and the decomposition of multiway arrays discover the predictive factor of RhoB in two cohorts of patients with rectal cancer having survival rates of <5 and >5 years. The analysis results show distinctions between the tensor decomposition factors of the two cohorts.

Automated classification of protein expression levels in immunohistochemistry images to improve the detection of cancer biomarkers

Background

The expression changes of some proteins are associated with cancer progression, and can be used as biomarkers in cancer diagnosis. Automated systems have been frequently applied in the large-scale detection of protein biomarkers and have provided a valuable complement for wet-laboratory experiments. For example, our previous work used an immunohistochemical image-based machine learning classifier of protein subcellular locations to screen biomarker proteins that change locations in colon cancer tissues. The tool could recognize the location of biomarkers but did not consider the effect of protein expression level changes on the screening process.

Results

In this study, we built an automated classification model that recognizes protein expression levels in immunohistochemical images, and used the protein expression levels in combination with subcellular locations to screen cancer biomarkers. To minimize the effect of non-informative sections on the immunohistochemical images, we employed the representative image patches as input and applied a Wasserstein distance method to determine the number of patches. For the patches and the whole images, we compared the ability of color features, characteristic curve features, and deep convolutional neural network features to distinguish different levels of protein expression and employed deep learning and conventional classification models. Experimental results showed that the best classifier can achieve an accuracy of 73.72% and an F1-score of 0.6343. In the screening of protein biomarkers, the detection accuracy improved from 63.64 to 95.45% upon the incorporation of the protein expression changes.

Conclusions

Machine learning can distinguish different protein expression levels and speed up their annotation in the future. Combining information on the expression patterns and subcellular locations of protein can improve the accuracy of automatic cancer biomarker screening. This work could be useful in discovering new cancer biomarkers for clinical diagnosis and research.

GraphLoc: a graph neural network model for predicting protein subcellular localization from immunohistochemistry images

Motivation

Recognition of protein subcellular distribution patterns and identification of location biomarker proteins in cancer tissues are important for understanding protein functions and related diseases. Immunohistochemical (IHC) images enable visualizing the distribution of proteins at the tissue level, providing an important resource for the protein localization studies. In the past decades, several image-based protein subcellular location prediction methods have been developed, but the prediction accuracies still have much space to improve due to the complexity of protein patterns resulting from multi-label proteins and variation of location patterns across cell types or states.

Results

Here, we propose a multi-label multi-instance model based on deep graph convolutional neural networks, GraphLoc, to recognize protein subcellular location patterns. GraphLoc builds a graph of multiple IHC images for one protein, learns protein-level representations by graph convolutions, and predicts multi-label information by a dynamic threshold method. Our results show that GraphLoc is a promising model for image-based protein subcellular location prediction with model interpretability. Furthermore, we apply GraphLoc to the identification of candidate location biomarkers and potential members for protein networks. A large portion of the predicted results have supporting evidence from the existing literatures and the new candidates also provide guidance for further experimental screening.

Availability

The dataset and code are available at: www.csbio.sjtu.edu.cn/bioinf/GraphLoc.

Supplementary information

Supplementary data are available at Bioinformatics online.

Tropolone derivative hinokitiol ameliorates cerulein-induced acute pancreatitis in mice

Hinokitiol is a natural bio-active tropolone derivative with promising antioxidant and anti-inflammatory properties. This study was conducted to evaluate the ameliorative effects of hinokitiol against acute pancreatitis induced by cerulein. Mice were pre-treated with hinokitiol intraperitoneally for 7 days (50 and 100 mg/kg), and on the final day of study, cerulein (6 × 50 μg/kg) was injected every hour for six times. Six hours after the last dose of cerulein, blood was collected from the mice through retro-orbital plexus for biochemical analysis. After blood collection, mice were euthanized and the pancreas was harvested for studying effects on oxidative stress, pro-inflammatory cytokines, immunohistochemistry and histopathology of tissue sections. Hinokitiol treatment significantly reduced edema of the pancreas and reduced the plasma levels of lipase and amylase in mice with cerulein-induced acute pancreatitis. It also attenuated the oxidative and nitrosative stress related damage as evident from the reduced malondialdehyde (MDA) and nitrite levels, which were significantly increased in the mice with acute pancreatitis. Furthermore, hinokitiol administration significantly reduced the pancreatitis-evoked decrease in the activity of catalase, glutathione (GSH) and superoxide dismutase (SOD) in the pancreatic tissue. Pre-treatment with hinokitiol significantly reduced the elevated levels of pro-inflammatory cytokines like interleukin-6 (IL-6), interleukin-1β (IL-1β), tumor necrosis factor-alpha (TNF-α) as well as increased the levels of anti-inflammatory cytokine interleukin-10 (IL-10) in the pancreatic tissue of mice with acute pancreatitis. The immunohistochemical expression of nuclear factor kappa light chain enhancer of activated B cells (NF-κB), cyclooxygenase (COX-2) and TNF-α were significantly decreased by hinokitiol in mice with cerulein-induced acute pancreatitis. In conclusion, the results of the present study demonstrate that hinokitiol has significant potential to prevent cerulein-induced acute pancreatitis.

The Studies of Prognostic Factors and the Genetic Polymorphism of Methylenetetrahydrofolate Reductase C667T in Thymic Epithelial Tumors

Objective

To describe the clinical features of a cohort of patients with thymic epithelial tumors (TETs) and to analyze their prognostic factors. In particular, we investigated the correlation between the genetic polymorphism of methylenetetrahydrofolate reductase (MTHFR) C667T and the incidence of TETs.

Methods

Pathological records were reviewed from the database of the Second Affiliated Hospital of Jiaxing University, from January 2010 to December 2020, and 84 patients with TETs were recruited for this study. Univariate and multivariate analyses were performed to determine the prognostic factors. The genetic polymorphism of MTHFR C667T was examined in the patients with TETs and in a group of healthy individuals. The correlation between MTHFR transcriptional levels and methylation was analyzed using The Cancer Genome Atlas (TCGA) thymoma dataset from the cBioPortal platform.

Results

Kaplan–Meier univariate survival analysis showed that sex, age, the maximum tumor diameter, surgery, chemotherapy, radiotherapy, WHO histological classification, Masaoka–Koga stage, and 8th UICC/AJCC TNM staging, were statistically significantly correlated with the prognosis of patients with TETs. The Masaoka–Koga stage and 8th UICC/AJCC TNM staging were strongly correlated with each other in this study (r=0.925, P<0.001). Cox multivariate survival analysis showed that the maximum tumor diameter, Masaoka–Koga stage, and 8th UICC/AJCC TNM staging were independent prognostic factors affecting the overall survival (OS) of patients with TETs (P<0.05). The MTHFR C667T genotype (χ^{2 =} 7.987, P=0.018) and allele distribution (χ^{2 =} 5.750, P=0.016) were significantly different between the patients and healthy controls. CT heterozygous and TT homozygous genotypes at this MTHFR polymorphism significantly increased the risk of TETs (odds ratio [OR] =4.721, P=0.008). Kaplan–Meier univariate survival analysis showed that there was no correlation between different genotypes and the prognosis of TETs (CC versus CT + TT, χ^{2 =}0.003, P=0.959). Finally, a negative correlation between the transcriptional and methylation levels of MTHFR was observed in the TCGA thymoma dataset (r=-0.24, P=0.010).

Conclusions

The Masaoka–Koga stage, 8th UICC/AJCC TNM staging, and maximum tumor diameter were independent prognostic factors for TETs. Reduced methylation levels of MTHFR and particular polymorphic variants may contribute to the susceptibility to developing TETs.

Parallel analysis of transcription, integration, and sequence of single HIV-1 proviruses

HIV-1-infected cells that persist despite antiretroviral therapy (ART) are frequently considered "transcriptionally silent," but active viral gene expression may occur in some cells, challenging the concept of viral latency. Applying an assay for profiling the transcriptional activity and the chromosomal locations of individual proviruses, we describe a global genomic and epigenetic map of transcriptionally active and silent proviral species and evaluate their longitudinal evolution in persons receiving suppressive ART. Using genome-wide epigenetic reference data, we show that proviral transcriptional activity is associated with activating epigenetic chromatin features in linear proximity of integration sites and in their inter- and intrachromosomal contact regions. Transcriptionally active proviruses were actively selected against during prolonged ART; however, this pattern was violated by large clones of virally infected cells that may outcompete negative selection forces through elevated intrinsic proliferative activity. Our results suggest that transcriptionally active proviruses are dynamically evolving under selection pressure by host factors.

Learning protein subcellular localization multi-view patterns from heterogeneous data of imaging, sequence and networks

Location proteomics seeks to provide automated high-resolution descriptions of protein location patterns within cells. Many efforts have been undertaken in location proteomics over the past decades, thereby producing plenty of automated predictors for protein subcellular localization. However, most of these predictors are trained solely from high-throughput microscopic images or protein amino acid sequences alone. Unifying heterogeneous protein data sources has yet to be exploited. In this paper, we present a pipeline called sequence, image, network-based protein subcellular locator (SIN-Locator) that constructs a multi-view description of proteins by integrating multiple data types including images of protein expression in cells or tissues, amino acid sequences and protein-protein interaction networks, to classify the patterns of protein subcellular locations. Proteins were encoded by both handcrafted features and deep learning features, and multiple combining methods were implemented. Our experimental results indicated that optimal integrations can considerately enhance the classification accuracy, and the utility of SIN-Locator has been demonstrated through applying to new released proteins in the human protein atlas. Furthermore, we also investigate the contribution of different data sources and influence of partial absence of data. This work is anticipated to provide clues for reconciliation and combination of multi-source data for protein location analysis.

Incorporating label correlations into deep neural networks to classify protein subcellular location patterns in immunohistochemistry images

Analysis of protein subcellular localization is a critical part of proteomics. In recent years, as both the number and quality of microscopic images are increasing rapidly, many automated methods, especially convolutional neural networks (CNN), have been developed to predict protein subcellular location(s) based on bioimages, but their performance always suffers from some inherent properties of the problem. First, many microscopic images have non-informative or noisy sections, like unstained stroma and unspecific background, which affect the extraction of protein expression information. Second, the patterns of protein subcellular localization are very complex, as a lot of proteins locate in more than one compartment. In this study, we propose a new label-correlation enhanced deep neural network, laceDNN, to classify the subcellular locations of multi-label proteins from immunohistochemistry images. The model uses small representative patches as input to alleviate the image noise issue, and its backbone is a hybrid architecture of CNN and recurrent neural network, where the former network extracts representative image features and the latter learns the organelle dependency relationships. Our experimental results indicate that the proposed model can improve the performance of multi-label protein subcellular classification.

DeepHistoClass: A Novel Strategy for Confident Classification of Immunohistochemistry Images Using Deep Learning

A multitude of efforts worldwide aim to create a single-cell reference map of the human body, for fundamental understanding of human health, molecular medicine, and targeted treatment. Antibody-based proteomics using immunohistochemistry (IHC) has proven to be an excellent technology for integration with large-scale single-cell transcriptomics datasets. The golden standard for evaluation of IHC staining patterns is manual annotation, which is expensive and may lead to subjective errors. Artificial intelligence holds much promise for efficient and accurate pattern recognition, but confidence in prediction needs to be addressed. Here, the aim was to present a reliable and comprehensive framework for automated annotation of IHC images. We developed a multilabel classification of 7848 complex IHC images of human testis corresponding to 2794 unique proteins, generated as part of the Human Protein Atlas (HPA) project. Manual annotation data for eight different cell types was generated as a basis for training and testing a proposed Hybrid Bayesian Neural Network. By combining the deep learning model with a novel uncertainty metric, DeepHistoClass (DHC) Confidence Score, the average diagnostic performance improved from 86.9% to 96.3%. This metric not only reveals which images are reliably classified by the model, but can also be utilized for identification of manual annotation errors. The proposed streamlined workflow can be developed further for other tissue types in health and disease and has important implications for digital pathology initiatives or large-scale protein mapping efforts such as the HPA project.

•

A novel method for automated annotation of immunohistochemistry images.

•

Introduction of an uncertainty metric, the DeepHistoClass (DHC) confidence score.

•

Increased accuracy of automated image predictions.

•

Identification of manual annotation errors.

Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics

Purpose of Review

Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images.

Recent Findings

Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond.

Summary

Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.

Bird Eye View of Protein Subcellular Localization Prediction

Proteins are made up of long chain of amino acids that perform a variety of functions in different organisms. The activity of the proteins is determined by the nucleotide sequence of their genes and by its 3D structure. In addition, it is essential for proteins to be destined to their specific locations or compartments to perform their structure and functions. The challenge of computational prediction of subcellular localization of proteins is addressed in various in silico methods. In this review, we reviewed the progress in this field and offered a bird eye view consisting of a comprehensive listing of tools, types of input features explored, machine learning approaches employed, and evaluation matrices applied. We hope the review will be useful for the researchers working in the field of protein localization predictions.

Consistency and variation of protein subcellular location annotations

A major challenge for protein databases is reconciling information from diverse sources. This is especially difficult when some information consists of secondary, human-interpreted rather than primary data. For example, the Swiss-Prot database contains curated annotations of subcellular location that are based on predictions from protein sequence, statements in scientific articles, and published experimental evidence. The Human Protein Atlas (HPA) consists of millions of high-resolution microscopic images that show protein spatial distribution on a cellular and subcellular level. These images are manually annotated with protein subcellular locations by trained experts. The image annotations in HPA can capture the variation of subcellular location across different cell lines, tissues, or tissue states. Systematic investigation of the consistency between HPA and Swiss-Prot assignments of subcellular location, which is important for understanding and utilizing protein location data from the two databases, has not been described previously. In this paper, we quantitatively evaluate the consistency of subcellular location annotations between HPA and Swiss-Prot at multiple levels, as well as variation of protein locations across cell lines and tissues. Our results show that annotations of these two databases differ significantly in many cases, leading to proposed procedures for deriving and integrating the protein subcellular location data. We also find that proteins having highly variable locations are more likely to be biomarkers of diseases, providing support for incorporating analysis of subcellular location in protein biomarker identification and screening.

ImPLoc: a multi-instance deep learning model for the prediction of protein subcellular localization based on immunohistochemistry images

MOTIVATION

The tissue atlas of the human protein atlas (HPA) houses immunohistochemistry (IHC) images visualizing the protein distribution from the tissue level down to the cell level, which provide an important resource to study human spatial proteome. Especially, the protein subcellular localization patterns revealed by these images are helpful for understanding protein functions, and the differential localization analysis across normal and cancer tissues lead to new cancer biomarkers. However, computational tools for processing images in this database are highly underdeveloped. The recognition of the localization patterns suffers from the variation in image quality and the difficulty in detecting microscopic targets.

RESULTS

We propose a deep multi-instance multi-label model, ImPLoc, to predict the subcellular locations from IHC images. In this model, we employ a deep convolutional neural network-based feature extractor to represent image features, and design a multi-head self-attention encoder to aggregate multiple feature vectors for subsequent prediction. We construct a benchmark dataset of 1186 proteins including 7855 images from HPA and 6 subcellular locations. The experimental results show that ImPLoc achieves significant enhancement on the prediction accuracy compared with the current computational methods. We further apply ImPLoc to a test set of 889 proteins with images from both normal and cancer tissues, and obtain 8 differentially localized proteins with a significance level of 0.05.

AVAILABILITY AND IMPLEMENTATION

https://github.com/yl2019lw/ImPloc.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Learning complex subcellular distribution patterns of proteins via analysis of immunohistochemistry images

MOTIVATION

Systematic and comprehensive analysis of protein subcellular location as a critical part of proteomics ('location proteomics') has been studied for many years, but annotating protein subcellular locations and understanding variation of the location patterns across various cell types and states is still challenging.

RESULTS

In this work, we used immunohistochemistry images from the Human Protein Atlas as the source of subcellular location information, and built classification models for the complex protein spatial distribution in normal and cancerous tissues. The models can automatically estimate the fractions of protein in different subcellular locations, and can help to quantify the changes of protein distribution from normal to cancer tissues. In addition, we examined the extent to which different annotated protein pathways and complexes showed similarity in the locations of their member proteins, and then predicted new potential proteins for these networks.

AVAILABILITY AND IMPLEMENTATION

The dataset and code are available at: www.csbio.sjtu.edu.cn/bioinf/complexsubcellularpatterns.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Automated classification of protein subcellular localization in immunohistochemistry images to reveal biomarkers in colon cancer

BACKGROUND

Protein biomarkers play important roles in cancer diagnosis. Many efforts have been made on measuring abnormal expression intensity in biological samples to identity cancer types and stages. However, the change of subcellular location of proteins, which is also critical for understanding and detecting diseases, has been rarely studied.

RESULTS

In this work, we developed a machine learning model to classify protein subcellular locations based on immunohistochemistry images of human colon tissues, and validated the ability of the model to detect subcellular location changes of biomarker proteins related to colon cancer. The model uses representative image patches as inputs, and integrates feature engineering and deep learning methods. It achieves 92.69% accuracy in classification of new proteins. Two validation datasets of colon cancer biomarkers derived from published literatures and the human protein atlas database respectively are employed. It turns out that 81.82 and 65.66% of the biomarker proteins can be identified to change locations.

CONCLUSIONS

Our results demonstrate that using image patches and combining predefined and deep features can improve the performance of protein subcellular localization, and our model can effectively detect biomarkers based on protein subcellular translocations. This study is anticipated to be useful in annotating unknown subcellular localization for proteins and discovering new potential location biomarkers.

Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

BACKGROUND

Multiplex immunohistochemistry (mIHC) permits the labeling of six or more distinct cell types within a single histologic tissue section. The classification of each cell type requires detection of the unique colored chromogens localized to cells expressing biomarkers of interest. The most comprehensive and reproducible method to evaluate such slides is to employ digital pathology and image analysis pipelines to whole-slide images (WSIs). Our suite of deep learning tools quantitatively evaluates the expression of six biomarkers in mIHC WSIs. These methods address the current lack of readily available methods to evaluate more than four biomarkers and circumvent the need for specialized instrumentation to spectrally separate different colors. The use case application for our methods is a study that investigates tumor immune interactions in pancreatic ductal adenocarcinoma (PDAC) with a customized mIHC panel.

METHODS

Six different colored chromogens were utilized to label T-cells (CD3, CD4, CD8), B-cells (CD20), macrophages (CD16), and tumor cells (K17) in formalin-fixed paraffin-embedded (FFPE) PDAC tissue sections. We leveraged pathologist annotations to develop complementary deep learning-based methods: (1) ColorAE is a deep autoencoder which segments stained objects based on color; (2) U-Net is a convolutional neural network (CNN) trained to segment cells based on color, texture and shape; and ensemble methods that employ both ColorAE and U-Net, collectively referred to as (3) ColorAE:U-Net. We assessed the performance of our methods using: structural similarity and DICE score to evaluate segmentation results of ColorAE against traditional color deconvolution; F1 score, sensitivity, positive predictive value, and DICE score to evaluate the predictions from ColorAE, U-Net, and ColorAE:U-Net ensemble methods against pathologist-generated ground truth. We then used prediction results for spatial analysis (nearest neighbor).

RESULTS

We observed that (1) the performance of ColorAE is comparable to traditional color deconvolution for single-stain IHC images (note: traditional color deconvolution cannot be used for mIHC); (2) ColorAE and U-Net are complementary methods that detect 6 different classes of cells with comparable performance; (3) combinations of ColorAE and U-Net into ensemble methods outperform using either ColorAE and U-Net alone; and (4) ColorAE:U-Net ensemble methods can be employed for detailed analysis of the tumor microenvironment (TME). We developed a suite of scalable deep learning methods to analyze 6 distinctly labeled cell populations in mIHC WSIs. We evaluated our methods and found that they reliably detected and classified cells in the PDAC tumor microenvironment. We also present a use case, wherein we apply the ColorAE:U-Net ensemble method across 3 mIHC WSIs and use the predictions to quantify all stained cell populations and perform nearest neighbor spatial analysis. Thus, we provide proof of concept that these methods can be employed to quantitatively describe the spatial distribution immune cells within the tumor microenvironment. These complementary deep learning methods are readily deployable for use in clinical research studies.

MIC_Locator: a novel image-based protein subcellular location multi-label prediction model based on multi-scale monogenic signal representation and intensity encoding strategy

Background

Protein subcellular localization plays a crucial role in understanding cell function. Proteins need to be in the right place at the right time, and combine with the corresponding molecules to fulfill their functions. Furthermore, prediction of protein subcellular location not only should be a guiding role in drug design and development due to potential molecular targets but also be an essential role in genome annotation. Taking the current status of image-based protein subcellular localization as an example, there are three common drawbacks, i.e., obsolete datasets without updating label information, stereotypical feature descriptor on spatial domain or grey level, and single-function prediction algorithm’s limited capacity of handling single-label database.

Results

In this paper, a novel human protein subcellular localization prediction model MIC_Locator is proposed. Firstly, the latest datasets are collected and collated as our benchmark dataset instead of obsolete data while training prediction model. Secondly, Fourier transformation, Riesz transformation, Log-Gabor filter and intensity coding strategy are employed to obtain frequency feature based on three components of monogenic signal with different frequency scales. Thirdly, a chained prediction model is proposed to handle multi-label instead of single-label datasets. The experiment results showed that the MIC_Locator can achieve 60.56% subset accuracy and outperform the existing majority of prediction models, and the frequency feature and intensity coding strategy can be conducive to improving the classification accuracy.

Conclusions

Our results demonstrate that the frequency feature is more beneficial for improving the performance of model compared to features extracted from spatial domain, and the MIC_Locator proposed in this paper can speed up validation of protein annotation, knowledge of protein function and proteomics research.

Translational AI and Deep Learning in Diagnostic Pathology

There has been an exponential growth in the application of AI in health and in pathology. This is resulting in the innovation of deep learning technologies that are specifically aimed at cellular imaging and practical applications that could transform diagnostic pathology. This paper reviews the different approaches to deep learning in pathology, the public grand challenges that have driven this innovation and a range of emerging applications in pathology. The translation of AI into clinical practice will require applications to be embedded seamlessly within digital pathology workflows, driving an integrated approach to diagnostics and providing pathologists with new tools that accelerate workflow and improve diagnostic consistency and reduce errors. The clearance of digital pathology for primary diagnosis in the US by some manufacturers provides the platform on which to deliver practical AI. AI and computational pathology will continue to mature as researchers, clinicians, industry, regulatory organizations and patient advocacy groups work together to innovate and deliver new technologies to health care providers: technologies which are better, faster, cheaper, more precise, and safe.

A method for semi-automated image analysis of HLA class I tumour epithelium expression in rectal cancer

Biomarkers may hold the key towards development and improvement of personalized cancer treatment. For instance, tumour expression of immune system-related proteins may reveal the tumour immune status and, accordingly, determine choice for type of immunotherapy. Therefore, objective evaluation of tumour biomarker expression is needed but often challenging. For instance, human leukocyte antigen (HLA) class I tumour epithelium expression is cumbersome to quantify by eye due to its presence on both tumour epithelial cells and tumour stromal cells, as well as tumourinfiltrating immune cells. In this study, we solved this problem by setting up an immunohistochemical (IHC) double staining using a tissue microarray (TMA) of rectal tumours wherein HLA class I expression was coloured with a blue chromogen, whereas non-epithelial tissue was visualized with a brown chromogen. We subsequently developed a semi-automated image analysis method that identified tumour epithelium as well as the percentage of HLA class I-positive tumour epithelium. Using this technique, we compared HCA2/HC10 and EMR8-5 antibodies for the assessment of HLA class I tumour expression and concluded that EMR8-5 is the superior antibody for this purpose. This IHC double staining can in principle be used for scoring of any biomarker expressed by tumour epithelium.

Quantitative imaging for development of companion diagnostics to drugs targeting HGF/MET

The limited clinical success of anti-HGF/MET drugs can be attributed to the lack of predictive biomarkers that adequately select patients for treatment. We demonstrate here that quantitative digital imaging of formalin fixed paraffin embedded tissues stained by immunohistochemistry can be used to measure signals from weakly staining antibodies and provides new opportunities to develop assays for detection of MET receptor activity. To establish a biomarker panel of MET activation, we employed seven antibodies measuring protein expression in the HGF/MET pathway in 20 cases and up to 80 cores from 18 human cancer types. The antibodies bind to epitopes in the extra (EC)- and intracellular (IC) domains of MET (MET4^EC, SP44_MET^IC, D1C2_MET^IC), to MET-pY1234/pY1235, a marker of MET kinase activation, as well as to HGF, pSFK or pMAPK. Expression of HGF was determined in tumour cells (T_HGF) as well as in stroma surrounding cancer (St_HGF). Remarkably, MET4^EC correlated more strongly with pMET (r = 0.47) than SP44_MET^IC (r = 0.21) or D1C2_MET^IC (r = 0.08) across 18 cancer types. In addition, correlation coefficients of pMET and T_HGF (r = 0.38) and pMET and pSFK (r = 0.56) were high. Prediction models of MET activation reveal cancer-type specific differences in performance of MET4^EC, SP44_MET^IC and anti-HGF antibodies. Thus, we conclude that assays to predict the response to HGF/MET inhibitors require a cancer-type specific antibody selection and should be developed in those cancer types in which they are employed clinically.

Computational pathology of pre-treatment biopsies identifies lymphocyte density as a predictor of response to neoadjuvant chemotherapy in breast cancer

BACKGROUND

There is a need to improve prediction of response to chemotherapy in breast cancer in order to improve clinical management and this may be achieved by harnessing computational metrics of tissue pathology. We investigated the association between quantitative image metrics derived from computational analysis of digital pathology slides and response to chemotherapy in women with breast cancer who received neoadjuvant chemotherapy.

METHODS

We digitised tissue sections of both diagnostic and surgical samples of breast tumours from 768 patients enrolled in the Neo-tAnGo randomized controlled trial. We subjected digital images to systematic analysis optimised for detection of single cells. Machine-learning methods were used to classify cells as cancer, stromal or lymphocyte and we computed estimates of absolute numbers, relative fractions and cell densities using these data. Pathological complete response (pCR), a histological indicator of chemotherapy response, was the primary endpoint. Fifteen image metrics were tested for their association with pCR using univariate and multivariate logistic regression.

RESULTS

Median lymphocyte density proved most strongly associated with pCR on univariate analysis (OR 4.46, 95 % CI 2.34-8.50, p < 0.0001; observations = 614) and on multivariate analysis (OR 2.42, 95 % CI 1.08-5.40, p = 0.03; observations = 406) after adjustment for clinical factors. Further exploratory analyses revealed that in approximately one quarter of cases there was an increase in lymphocyte density in the tumour removed at surgery compared to diagnostic biopsies. A reduction in lymphocyte density at surgery was strongly associated with pCR (OR 0.28, 95 % CI 0.17-0.47, p < 0.0001; observations = 553).

CONCLUSIONS

A data-driven analysis of computational pathology reveals lymphocyte density as an independent predictor of pCR. Paradoxically an increase in lymphocyte density, following exposure to chemotherapy, is associated with a lack of pCR. Computational pathology can provide objective, quantitative and reproducible tissue metrics and represents a viable means of outcome prediction in breast cancer.

TRIAL REGISTRATION

ClinicalTrials.gov NCT00070278 ; 03/10/2003.

Human cell structure-driven model construction for predicting protein subcellular location from biological images

MOTIVATION

The systematic study of subcellular location pattern is very important for fully characterizing the human proteome. Nowadays, with the great advances in automated microscopic imaging, accurate bioimage-based classification methods to predict protein subcellular locations are highly desired. All existing models were constructed on the independent parallel hypothesis, where the cellular component classes are positioned independently in a multi-class classification engine. The important structural information of cellular compartments is missed. To deal with this problem for developing more accurate models, we proposed a novel cell structure-driven classifier construction approach (SC-PSorter) by employing the prior biological structural information in the learning model. Specifically, the structural relationship among the cellular components is reflected by a new codeword matrix under the error correcting output coding framework. Then, we construct multiple SC-PSorter-based classifiers corresponding to the columns of the error correcting output coding codeword matrix using a multi-kernel support vector machine classification approach. Finally, we perform the classifier ensemble by combining those multiple SC-PSorter-based classifiers via majority voting.

RESULTS

We evaluate our method on a collection of 1636 immunohistochemistry images from the Human Protein Atlas database. The experimental results show that our method achieves an overall accuracy of 89.0%, which is 6.4% higher than the state-of-the-art method.

AVAILABILITY AND IMPLEMENTATION

The dataset and code can be downloaded from https://github.com/shaoweinuaa/.

CONTACT

dqzhang@nuaa.edu.cn

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Syndecan-1 up-regulates microRNA-331-3p and mediates epithelial-to-mesenchymal transition in prostate cancer

MicroRNAs (miRNAs) are small noncoding RNAs with a length of approximately 19-24 nucleotides that regulate gene expression through translational inhibition and contribute to the progression of various tumors including prostate cancer. Aberrant expression of miRNAs has been implicated in the progression and metastasis of prostate cancer. The present study aimed to investigate whether miR-331-3p controlled by syndecan-1 positively affects the epithelial-to-mesenchymal transition (EMT). Overexpression of miR-331-3p upregulated mesenchymal markers such as vimentin, N-cadherin, and snail and downregulated epithelial markers such as E-cadherin and desmoplakin in the prostate cancer cell line PC3. We identified Neuropilin 2 and nucleus accumbens-associated protein 1 as putative target molecules in silico, as they were closely associated with the expression of miR-331-3p and TGF-β/Smad 4 signals. In situ hybridization and immunohistochemistry of radical prostatectomy samples revealed miR-331-3p in cancer cells with high Gleason patterns, in which EMT was demonstrated by decreased E-cadherin, and increased vimentin staining. Syndecan-1 gene silencing decreased levels of Dicer, which is involved in miRNA maturation. MiR-331-3p-mediated miRNA maturation and enhanced EMT via effects on TGF-β/Smad 4 and Dicer are essential for the development of prostate cancer mediated by syndecan-1. © 2015 Wiley Periodicals, Inc.

The potential clinical impact of the tissue-based map of the human proteome

Since the first draft of the human genome sequence was published, several attempts have been made to map the human proteome, the functional representation of the genome. One such initiative is the Human Protein Atlas project, which recently released a tissue-based map of the human proteome. The Human Protein Atlas is based on the combination of transcriptomics and antibody-based proteomics for mapping the human proteome down to the single cell level. The comprehensive publicly available database contains more than 13 million unique immunohistochemistry images and provides an excellent resource for exploration and investigation of future drug targets and disease biomarkers.