Adjacent Cell Marker Lateral Spillover Compensation and Reinforcement for Multiplexed Images

From surfing to diving into the tumor microenvironment through multiparametric imaging mass cytometry

The tumor microenvironment (TME) is a complex ecosystem where malignant and non-malignant cells cooperate and interact determining cancer progression. Cell abundance, phenotype and localization within the TME vary over tumor development and in response to therapeutic interventions. Therefore, increasing our knowledge of the spatiotemporal changes in the tumor ecosystem architecture is of importance to better understand the etiologic development of the neoplastic diseases. Imaging Mass Cytometry (IMC) represents the elective multiplexed imaging technology enabling the in-situ analysis of up to 43 different protein markers for in-depth phenotypic and spatial investigation of cells in their preserved microenvironment. IMC is currently applied in cancer research to define the composition of the cellular landscape and to identify biomarkers of predictive and prognostic significance with relevance in mechanisms of drug resistance. Herein, we describe the general principles and experimental workflow of IMC raising the informative potential in preclinical and clinical cancer research.

Highly Efficient Calibration-Free Color Compensation Algorithm for Imaging Flow Cytometry

As an emerging platform gaining significant attention from the biomedical community, multiplexed fluorescent imaging from imaging flow cytometry enables simultaneous detection of numerous biological targets within a single cell. Due to the spectral overlap, signals from one fluorophore can bleed into other detection channels, leading to spillover artifacts, which cause erroneous results and false discoveries. Existing color compensation algorithms use special samples to calibrate the fluorophores individually, a time-consuming and laborious process that is cumbersome and hard to scale. While recent developments in calibration-free algorithms produce promising results in multi-color microscope images, these algorithms, when applied to single-cell images with all the fluorophores within a small and constrained area, tend to cause overcorrection by treating real signals as crosstalk and triggering stability problems during the iterative computation process. Here we demonstrate a simple and intuitive algorithm that greatly reduces overcorrection and is computationally efficient. While designed for imaging flow cytometers, our calibration-free crosstalk removal algorithm can be readily applied to microscopy as well. We have validated its effectiveness on various datasets, including simulated cell images, 2D and 3D imaging flow cytometry images, and microscopic images. Our algorithm offers an effective solution for multi-parameter single-cell images where channels are often both spectrally and spatially overlapped within the limited area of a single cell.

Integrated workflow for analysis of immune enriched spatial proteomic data with IMmuneCite

Spatial proteomics enable detailed analysis of tissue at single cell resolution. However, creating reliable segmentation masks and assigning accurate cell phenotypes to discrete cellular phenotypes can be challenging. We introduce IMmuneCite, a computational framework for comprehensive image pre-processing and single-cell dataset creation, focused on defining complex immune landscapes when using spatial proteomics platforms. We demonstrate that IMmuneCite facilitates the identification of 32 discrete immune cell phenotypes using data from human liver samples while substantially reducing nonbiological cell clusters arising from co-localization of markers for different cell lineages. We established its versatility and ability to accommodate any antibody panel and different species by applying IMmuneCite to data from murine liver tissue. This approach enabled deep characterization of different functional states in each immune compartment, uncovering key features of the immune microenvironment in clinical liver transplantation and murine hepatocellular carcinoma. In conclusion, we demonstrated that IMmuneCite is a user-friendly, integrated computational platform that facilitates investigation of the immune microenvironment across species, while ensuring the creation of an immune focused, spatially resolved single-cell proteomic dataset to provide high fidelity, biologically relevant analyses.

Addressing persistent challenges in digital image analysis of cancer tissue: resources developed from a hackathon

The National Cancer Institute (NCI) supports numerous research consortia that rely on imaging technologies to study cancerous tissues. To foster collaboration and innovation in this field, the Image Analysis Working Group (IAWG) was created in 2019. As multiplexed imaging techniques grow in scale and complexity, more advanced computational methods are required beyond traditional approaches like segmentation and pixel intensity quantification. In 2022, the IAWG held a virtual hackathon focused on addressing challenges in analyzing complex, high-dimensional datasets from fixed cancer tissues. The hackathon addressed key challenges in three areas: (1) cell type classification and assessment, (2) spatial data visualization and translation, and (3) scaling image analysis for large, multi-terabyte datasets. Participants explored the limitations of current automated analysis tools, developed potential solutions, and made significant progress during the hackathon. Here we provide a summary of the efforts and resultant resources and highlight remaining challenges facing the research community as emerging technologies are integrated into diverse imaging modalities and data analysis platforms.

Segmentation aware probabilistic phenotyping of single-cell spatial protein expression data

Spatial protein expression technologies can map cellular content and organization by simultaneously quantifying the expression of >40 proteins at subcellular resolution within intact tissue sections and cell lines. However, necessary image segmentation to single cells is challenging and error prone, easily confounding the interpretation of cellular phenotypes and cell clusters. To address these limitations, we present STARLING, a probabilistic machine learning model designed to quantify cell populations from spatial protein expression data while accounting for segmentation errors. To evaluate performance, we develop a comprehensive benchmarking workflow by generating highly multiplexed imaging data of cell line pellet standards with controlled cell content and marker expression and additionally established a score to quantify the biological plausibility of discovered cellular phenotypes on patient-derived tissue sections. Moreover, we generate spatial expression data of the human tonsil-a densely packed tissue prone to segmentation errors-and demonstrate cellular states captured by STARLING identify known cell types not visible with other methods and enable quantification of intra- and inter- individual heterogeneity.

Multiplexed Immunophenotyping of Lymphoma Tissue Samples

High-plex imaging techniques enable the detection and quantification of a multitude of markers in tissue biopsies at single-cell or near-single-cell resolution. In lymphoma, this can facilitate the detection and characterization of cellular phenotypes and interactions, describing both tumor and microenvironmental cells. In combination with other techniques, high-plex imaging allows the investigation of biological mechanisms and clinically relevant biomarkers. CO-Detection by IndEXing (CODEX), one of such techniques, is based on antibodies labeled with unique DNA oligonucleotides that can be visualized by complementary reporter oligonucleotides coupled to a fluorophore. Here, we provide an overview of the key steps of a CODEX-based project, including (1) antibody panel design, (2) cohort selection, (3) staining and imaging, (4) data analysis. By sharing our CODEX protocol and our experience with FFPE tissue samples, we aim to encourage wider use of this powerful technique in lymphoma research and improve insight into cellular composition and spatial dynamics for improved diagnostics and therapy.

ImmuNet: a segmentation-free machine learning pipeline for immune landscape phenotyping in tumors by multiplex imaging

Tissue specimens taken from primary tumors or metastases contain important information for diagnosis and treatment of cancer patients. Multiplex imaging allows in situ visualization of heterogeneous cell populations, such as immune cells, in tissue samples. Most image processing pipelines first segment cell boundaries and then measure marker expression to assign cell phenotypes. In dense tissue environments, this segmentation-first approach can be inaccurate due to segmentation errors or overlapping cells. Here, we introduce the machine-learning pipeline "ImmuNet", which identifies positions and phenotypes of cells without segmenting them. ImmuNet is easy to train: human annotators only need to click on an immune cell and score its expression of each marker-drawing a full cell outline is not required. We trained and evaluated ImmuNet on multiplex images from human tonsil, lung cancer, prostate cancer, melanoma, and bladder cancer tissue samples and found it to consistently achieve error rates below 5%-10% across tissue types, cell types, and tissue densities, outperforming a segmentation-based baseline method. Furthermore, we externally validate ImmuNet results by comparing them to flow cytometric cell count measurements from the same tissue. In summary, ImmuNet is an effective, simpler alternative to segmentation-based approaches when only cell positions and phenotypes, but not their shapes, are required for downstream analyses. Thus, ImmuNet helps researchers to analyze cell positions in multiplex tissue images more easily and accurately.

Quality control for single-cell analysis of high-plex tissue profiles using CyLinter

Tumors are complex assemblies of cellular and acellular structures patterned on spatial scales from microns to centimeters. Study of these assemblies has advanced dramatically with the introduction of high-plex spatial profiling. Image-based profiling methods reveal the intensities and spatial distributions of 20-100 proteins at subcellular resolution in 103-107 cells per specimen. Despite extensive work on methods for extracting single-cell data from these images, all tissue images contain artifacts such as folds, debris, antibody aggregates, optical aberrations and image processing errors that arise from imperfections in specimen preparation, data acquisition, image assembly and feature extraction. Here we show that these artifacts dramatically impact single-cell data analysis, obscuring meaningful biological interpretation. We describe an interactive quality control software tool, CyLinter, that identifies and removes data associated with imaging artifacts. CyLinter greatly improves single-cell analysis, especially for archival specimens sectioned many years before data collection, such as those from clinical trials.

Pseudo-spectral angle mapping for pixel and cell classification in highly multiplexed immunofluorescence images

Purpose

The rapid development of highly multiplexed microscopy has enabled the study of cells embedded within their native tissue. The rich spatial data provided by these techniques have yielded exciting insights into the spatial features of human disease. However, computational methods for analyzing these high-content images are still emerging; there is a need for more robust and generalizable tools for evaluating the cellular constituents and stroma captured by high-plex imaging. To address this need, we have adapted spectral angle mapping-an algorithm developed for hyperspectral image analysis-to compress the channel dimension of high-plex immunofluorescence (IF) images.

Approach

Here, we present pseudo-spectral angle mapping (pSAM), a robust and flexible method for determining the most likely class of each pixel in a high-plex image. The class maps calculated through pSAM yield pixel classifications which can be combined with instance segmentation algorithms to classify individual cells.

Results

In a dataset of colon biopsies imaged with a 13-plex staining panel, 16 pSAM class maps were computed to generate pixel classifications. Instance segmentations of cells with Cellpose2.0 ( F 1 -score of 0.83 ± 0.13 ) were combined with these class maps to provide cell class predictions for 13 cell classes. In addition, in a separate unseen dataset of kidney biopsies imaged with a 44-plex staining panel, pSAM plus Cellpose2.0 ( F 1 -score of 0.86 ± 0.11 ) detected a diverse set of 38 classes of structural and immune cells.

Conclusions

In summary, pSAM is a powerful and generalizable tool for evaluating high-plex IF image data and classifying cells in these high-dimensional images.

IMmuneCite: an integrated workflow for analysis of immune enriched spatial proteomic data

Spatial proteomics enable detailed analysis of tissue at single cell resolution. However, creating reliable segmentation masks and assigning accurate cell phenotypes to discrete cellular phenotypes can be challenging. We introduce IMmuneCite, a computational framework for comprehensive image pre-processing and single-cell dataset creation, focused on defining complex immune landscapes when using spatial proteomics platforms. We demonstrate that IMmuneCite facilitates the identification of 32 discrete immune cell phenotypes using data from human liver samples while substantially reducing nonbiological cell clusters arising from co-localization of markers for different cell lineages. We established its versatility and ability to accommodate any antibody panel and different species by applying IMmuneCite to data from murine liver tissue. This approach enabled deep characterization of different functional states in each immune compartment, uncovering key features of the immune microenvironment in clinical liver transplantation and murine hepatocellular carcinoma. In conclusion, we demonstrated that IMmuneCite is a user-friendly, integrated computational platform that facilitates investigation of the immune microenvironment across species, while ensuring the creation of an immune focused, spatially resolved single-cell proteomic dataset to provide high fidelity, biologically relevant analyses.

Quantitative image analysis pipeline for detecting circulating hybrid cells in immunofluorescence images with human-level accuracy

Circulating hybrid cells (CHCs) are a newly discovered, tumor-derived cell population found in the peripheral blood of cancer patients and are thought to contribute to tumor metastasis. However, identifying CHCs by immunofluorescence (IF) imaging of patient peripheral blood mononuclear cells (PBMCs) is a time-consuming and subjective process that currently relies on manual annotation by laboratory technicians. Additionally, while IF is relatively easy to apply to tissue sections, its application to PBMC smears presents challenges due to the presence of biological and technical artifacts. To address these challenges, we present a robust image analysis pipeline to automate the detection and analysis of CHCs in IF images. The pipeline incorporates quality control to optimize specimen preparation protocols and remove unwanted artifacts, leverages a β-variational autoencoder (VAE) to learn meaningful latent representations of single-cell images, and employs a support vector machine (SVM) classifier to achieve human-level CHC detection. We created a rigorously labeled IF CHC data set including nine patients and two disease sites with the assistance of 10 annotators to evaluate the pipeline. We examined annotator variation and bias in CHC detection and provided guidelines to optimize the accuracy of CHC annotation. We found that all annotators agreed on CHC identification for only 65% of the cells in the data set and had a tendency to underestimate CHC counts for regions of interest (ROIs) containing relatively large amounts of cells (>50,000) when using the conventional enumeration method. On the other hand, our proposed approach is unbiased to ROI size. The SVM classifier trained on the β-VAE embeddings achieved an F1 score of 0.80, matching the average performance of human annotators. Our pipeline enables researchers to explore the role of CHCs in cancer progression and assess their potential as a clinical biomarker for metastasis. Further, we demonstrate that the pipeline can identify discrete cellular phenotypes among PBMCs, highlighting its utility beyond CHCs.

Development of 42 marker panel for in-depth study of cancer associated fibroblast niches in breast cancer using imaging mass cytometry

Imaging Mass Cytometry (IMC) is a novel, and formidable high multiplexing imaging method emerging as a promising tool for in-depth studying of tissue architecture and intercellular communications. Several studies have reported various IMC antibody panels mainly focused on studying the immunological landscape of the tumor microenvironment (TME). With this paper, we wanted to address cancer associated fibroblasts (CAFs), a component of the TME very often underrepresented and not emphasized enough in present IMC studies. Therefore, we focused on the development of a comprehensive IMC panel that can be used for a thorough description of the CAF composition of breast cancer TME and for an in-depth study of different CAF niches in relation to both immune and breast cancer cell communication. We established and validated a 42 marker panel using a variety of control tissues and rigorous quantification methods. The final panel contained 6 CAF-associated markers (aSMA, FAP, PDGFRa, PDGFRb, YAP1, pSMAD2). Breast cancer tissues (4 cases of luminal, 5 cases of triple negative breast cancer) and a modified CELESTA pipeline were used to demonstrate the utility of our IMC panel for detailed profiling of different CAF, immune and cancer cell phenotypes.

Quantitative performance assessment of Ultivue multiplex panels in formalin-fixed, paraffin-embedded human and murine tumor specimens

We present a rigorous validation strategy to evaluate the performance of Ultivue multiplex immunofluorescence panels. We have quantified the accuracy and precision of four different multiplex panels (three human and one mouse) in tumor specimens with varying levels of T cell density. Our results show that Ultivue panels are typically accurate wherein the relative difference in cell proportion between a multiplex image and a 1-plex image is less than 20% for a given biomarker. Ultivue panels exhibited relatively high intra-run precision (CV ≤ 25%) and relatively low inter-run precision (CV >> 25%) which can be remedied by using local intensity thresholding to gate biomarker positivity. We also evaluated the reproducibility of cell-cell distance estimates measured from multiplex images which show high intra- and inter-run precision. We introduce a new metric, multiplex labeling efficiency, which can be used to benchmark the overall fidelity of the multiplex data across multiple batch runs. Taken together our results provide a comprehensive characterization of Ultivue panels and offer practical guidelines for analyzing multiplex images.

Quality Control for Single Cell Analysis of High-plex Tissue Profiles using CyLinter

Tumors are complex assemblies of cellular and acellular structures patterned on spatial scales from microns to centimeters. Study of these assemblies has advanced dramatically with the introduction of high-plex spatial profiling. Image-based profiling methods reveal the intensities and spatial distributions of 20-100 proteins at subcellular resolution in 103-107 cells per specimen. Despite extensive work on methods for extracting single-cell data from these images, all tissue images contain artefacts such as folds, debris, antibody aggregates, optical aberrations and image processing errors that arise from imperfections in specimen preparation, data acquisition, image assembly, and feature extraction. We show that these artefacts dramatically impact single-cell data analysis, obscuring meaningful biological interpretation. We describe an interactive quality control software tool, CyLinter, that identifies and removes data associated with imaging artefacts. CyLinter greatly improves single-cell analysis, especially for archival specimens sectioned many years prior to data collection, such as those from clinical trials.

A Hitchhiker's guide to high-dimensional tissue imaging with multiplexed ion beam imaging

Advancements in multiplexed tissue imaging technologies are vital in shaping our understanding of tissue microenvironmental influences in disease contexts. These technologies now allow us to relate the phenotype of individual cells to their higher-order roles in tissue organization and function. Multiplexed Ion Beam Imaging (MIBI) is one of such technologies, which uses metal isotope-labeled antibodies and secondary ion mass spectrometry (SIMS) to image more than 40 protein markers simultaneously within a single tissue section. Here, we describe an optimized MIBI workflow for high-plex analysis of Formalin-Fixed Paraffin-Embedded (FFPE) tissues following antigen retrieval, metal isotope-conjugated antibody staining, imaging using the MIBI instrument, and subsequent data processing and analysis. While this workflow is focused on imaging human FFPE samples using the MIBI, this workflow can be easily extended to model systems, biological questions, and multiplexed imaging modalities.

Epstein-Barr Virus Orchestrates Spatial Reorganization and Immunomodulation within the Classic Hodgkin Lymphoma Tumor Microenvironment

Classic Hodgkin Lymphoma (cHL) is a tumor composed of rare malignant Hodgkin and Reed-Sternberg (HRS) cells nested within a T-cell rich inflammatory immune infiltrate. cHL is associated with Epstein-Barr Virus (EBV) in 25% of cases. The specific contributions of EBV to the pathogenesis of cHL remain largely unknown, in part due to technical barriers in dissecting the tumor microenvironment (TME) in high detail. Herein, we applied multiplexed ion beam imaging (MIBI) spatial pro-teomics on 6 EBV-positive and 14 EBV-negative cHL samples. We identify key TME features that distinguish between EBV-positive and EBV-negative cHL, including the relative predominance of memory CD8 T cells and increased T-cell dysfunction as a function of spatial proximity to HRS cells. Building upon a larger multi-institutional cohort of 22 EBV-positive and 24 EBV-negative cHL samples, we orthogonally validated our findings through a spatial multi-omics approach, coupling whole transcriptome capture with antibody-defined cell types for tu-mor and T-cell populations within the cHL TME. We delineate contrasting transcriptomic immunological signatures between EBV-positive and EBV-negative cases that differently impact HRS cell proliferation, tumor-immune interactions, and mecha-nisms of T-cell dysregulation and dysfunction. Our multi-modal framework enabled a comprehensive dissection of EBV-linked reorganization and immune evasion within the cHL TME, and highlighted the need to elucidate the cellular and molecular fac-tors of virus-associated tumors, with potential for targeted therapeutic strategies.

Pseudo-spectral angle mapping for automated pixel-level analysis of highly multiplexed tissue image data

The rapid development of highly multiplexed microscopy systems has enabled the study of cells embedded within their native tissue, which is providing exciting insights into the spatial features of human disease [1]. However, computational methods for analyzing these high-content images are still emerging, and there is a need for more robust and generalizable tools for evaluating the cellular constituents and underlying stroma captured by high-plex imaging [2]. To address this need, we have adapted spectral angle mapping - an algorithm used widely in hyperspectral image analysis - to compress the channel dimension of high-plex immunofluorescence images. As many high-plex immunofluorescence imaging experiments probe unique sets of protein markers, existing cell and pixel classification models do not typically generalize well. Pseudospectral angle mapping (pSAM) uses reference pseudospectra - or pixel vectors - to assign each pixel in an image a similarity score to several cell class reference vectors, which are defined by each unique staining panel. Here, we demonstrate that the class maps provided by pSAM can directly provide insight into the prevalence of each class defined by reference pseudospectra. In a dataset of high-plex images of colon biopsies from patients with gut autoimmune conditions, sixteen pSAM class representation maps were combined with instance segmentation of cells to provide cell class predictions. Finally, pSAM detected a diverse set of structure and immune cells when applied to a novel dataset of kidney biopsies imaged with a 43-marker panel. In summary, pSAM provides a powerful and readily generalizable method for evaluating high-plex immunofluorescence image data.

MAPS: pathologist-level cell type annotation from tissue images through machine learning

Highly multiplexed protein imaging is emerging as a potent technique for analyzing protein distribution within cells and tissues in their native context. However, existing cell annotation methods utilizing high-plex spatial proteomics data are resource intensive and necessitate iterative expert input, thereby constraining their scalability and practicality for extensive datasets. We introduce MAPS (Machine learning for Analysis of Proteomics in Spatial biology), a machine learning approach facilitating rapid and precise cell type identification with human-level accuracy from spatial proteomics data. Validated on multiple in-house and publicly available MIBI and CODEX datasets, MAPS outperforms current annotation techniques in terms of speed and accuracy, achieving pathologist-level precision even for typically challenging cell types, including tumor cells of immune origin. By democratizing rapidly deployable and scalable machine learning annotation, MAPS holds significant potential to expedite advances in tissue biology and disease comprehension.

NIPMAP: niche-phenotype mapping of multiplex histology data by community ecology

Advances in multiplex histology allow surveying millions of cells, dozens of cell types, and up to thousands of phenotypes within the spatial context of tissue sections. This leads to a combinatorial challenge in (a) summarizing the cellular and phenotypic architecture of tissues and (b) identifying phenotypes with interesting spatial architecture. To address this, we combine ideas from community ecology and machine learning into niche-phenotype mapping (NIPMAP). NIPMAP takes advantage of geometric constraints on local cellular composition imposed by the niche structure of tissues in order to automatically segment tissue sections into niches and their interfaces. Projecting phenotypes on niches and their interfaces identifies previously-reported and previously-unreported spatially-driven phenotypes, concisely summarizes the phenotypic architecture of tissues, and reveals fundamental properties of tissue architecture. NIPMAP is applicable to both protein and RNA multiplex histology of healthy and diseased tissue. An open-source R/Python package implements NIPMAP.

Dual-modality imaging of immunofluorescence and imaging mass cytometry for whole-slide imaging and accurate segmentation

Imaging mass cytometry (IMC) is a powerful technique capable of detecting over 30 markers on a single slide. It has been increasingly used for single-cell-based spatial phenotyping in a wide range of samples. However, it only acquires a rectangle field of view (FOV) with a relatively small size and low image resolution, which hinders downstream analysis. Here, we reported a highly practical dual-modality imaging method that combines high-resolution immunofluorescence (IF) and high-dimensional IMC on the same tissue slide. Our computational pipeline uses the whole-slide image (WSI) of IF as a spatial reference and integrates small-FOV IMC into a WSI of IMC. The high-resolution IF images enable accurate single-cell segmentation to extract robust high-dimensional IMC features for downstream analysis. We applied this method in esophageal adenocarcinoma of different stages, identified the single-cell pathology landscape via reconstruction of WSI IMC images, and demonstrated the advantage of the dual-modality imaging strategy.

LOCATOR: feature extraction and spatial analysis of the cancer tissue microenvironment using mass cytometry imaging technologies

Motivation

Recent advances in highly multiplexed imaging have provided unprecedented insights into the complex cellular organization of tissues, with many applications in translational medicine. However, downstream analyses of multiplexed imaging data face several technical limitations, and although some computational methods and bioinformatics tools are available, deciphering the complex spatial organization of cellular ecosystems remains a challenging problem.

Results

To mitigate this problem, we develop a novel computational tool, LOCATOR (anaLysis Of CAncer Tissue micrOenviRonment), for spatial analysis of cancer tissue microenvironments using data acquired from mass cytometry imaging technologies. LOCATOR introduces a graph-based representation of tissue images to describe features of the cellular organization and deploys downstream analysis and visualization utilities that can be used for data-driven patient-risk stratification. Our case studies using mass cytometry imaging data from two well-annotated breast cancer cohorts re-confirmed that the spatial organization of the tumour-immune microenvironment is strongly associated with the clinical outcome in breast cancer. In addition, we report interesting potential associations between the spatial organization of macrophages and patients' survival. Our work introduces an automated and versatile analysis tool for mass cytometry imaging data with many applications in future cancer research projects.

Availability and implementation

Datasets and codes of LOCATOR are publicly available at https://github.com/RezvanEhsani/LOCATOR.

The impact of similarity metrics on cell-type clustering in highly multiplexed in situ imaging cytometry data

Motivation

The advent of highly multiplexed in situ imaging cytometry assays has revolutionized the study of cellular systems, offering unparalleled detail in observing cellular activities and characteristics. These assays provide comprehensive insights by concurrently profiling the spatial distribution and molecular features of numerous cells. In navigating this complex data landscape, unsupervised machine learning techniques, particularly clustering algorithms, have become essential tools. They enable the identification and categorization of cell types and subsets based on their molecular characteristics. Despite their widespread adoption, most clustering algorithms in use were initially developed for cell suspension technologies, leading to a potential mismatch in application. There is a critical gap in the systematic evaluation of these methods, particularly in determining the properties that make them optimal for in situ imaging assays. Addressing this gap is vital for ensuring accurate, reliable analyses and fostering advancements in cellular biology research.

Results

In our extensive investigation, we evaluated a range of similarity metrics, which are crucial in determining the relationships between cells during the clustering process. Our findings reveal substantial variations in clustering performance, contingent on the similarity metric employed. These variations underscore the importance of selecting appropriate metrics to ensure accurate cell type and subset identification. In response to these challenges, we introduce FuseSOM, a novel ensemble clustering algorithm that integrates hierarchical multiview learning of similarity metrics with self-organizing maps. Through a rigorous stratified subsampling analysis framework, we demonstrate that FuseSOM outperforms existing best-practice clustering methods specifically tailored for in situ imaging cytometry data. Our work not only provides critical insights into the performance of clustering algorithms in this novel context but also offers a robust solution, paving the way for more accurate and reliable in situ imaging cytometry data analysis.

Availability and implementation

The FuseSOM R package is available on Bioconductor and is available under the GPL-3 license. All the codes for the analysis performed can be found at Github.

Multiplex imaging in immuno-oncology

Multiplex imaging has emerged as an invaluable tool for immune-oncologists and translational researchers, enabling them to examine intricate interactions among immune cells, stroma, matrix, and malignant cells within the tumor microenvironment (TME). It holds significant promise in the quest to discover improved biomarkers for treatment stratification and identify novel therapeutic targets. Nonetheless, several challenges exist in the realms of study design, experiment optimization, and data analysis. In this review, our aim is to present an overview of the utilization of multiplex imaging in immuno-oncology studies and inform novice researchers about the fundamental principles at each stage of the imaging and analysis process.

Quantitative image analysis pipeline for detecting circulating hybrid cells in immunofluorescence images with human-level accuracy

Circulating hybrid cells (CHCs) are a newly discovered, tumor-derived cell population identified in the peripheral blood of cancer patients and are thought to contribute to tumor metastasis. However, identifying CHCs by immunofluorescence (IF) imaging of patient peripheral blood mononuclear cells (PBMCs) is a time-consuming and subjective process that currently relies on manual annotation by laboratory technicians. Additionally, while IF is relatively easy to apply to tissue sections, its application on PBMC smears presents challenges due to the presence of biological and technical artifacts. To address these challenges, we present a robust image analysis pipeline to automate the detection and analyses of CHCs in IF images. The pipeline incorporates quality control to optimize specimen preparation protocols and remove unwanted artifacts, leverages a β-variational autoencoder (VAE) to learn meaningful latent representations of single-cell images and employs a support vector machine (SVM) classifier to achieve human-level CHC detection. We created a rigorously labeled IF CHC dataset including 9 patients and 2 disease sites with the assistance of 10 annotators to evaluate the pipeline. We examined annotator variation and bias in CHC detection and then provided guidelines to optimize the accuracy of CHC annotation. We found that all annotators agreed on CHC identification for only 65% of the cells in the dataset and had a tendency to underestimate CHC counts for regions of interest (ROI) containing relatively large amounts of cells (>50,000) when using conventional enumeration methods. On the other hand, our proposed approach is unbiased to ROI size. The SVM classifier trained on the β-VAE encodings achieved an F1 score of 0.80, matching the average performance of annotators. Our pipeline enables researchers to explore the role of CHCs in cancer progression and assess their potential as a clinical biomarker for metastasis. Further, we demonstrate that the pipeline can identify discrete cellular phenotypes among PBMCs, highlighting its utility beyond CHCs.

Robust phenotyping of highly multiplexed tissue imaging data using pixel-level clustering

While technologies for multiplexed imaging have provided an unprecedented understanding of tissue composition in health and disease, interpreting this data remains a significant computational challenge. To understand the spatial organization of tissue and how it relates to disease processes, imaging studies typically focus on cell-level phenotypes. However, images can capture biologically important objects that are outside of cells, such as the extracellular matrix. Here, we describe a pipeline, Pixie, that achieves robust and quantitative annotation of pixel-level features using unsupervised clustering and show its application across a variety of biological contexts and multiplexed imaging platforms. Furthermore, current cell phenotyping strategies that rely on unsupervised clustering can be labor intensive and require large amounts of manual cluster adjustments. We demonstrate how pixel clusters that lie within cells can be used to improve cell annotations. We comprehensively evaluate pre-processing steps and parameter choices to optimize clustering performance and quantify the reproducibility of our method. Importantly, Pixie is open source and easily customizable through a user-friendly interface.

Addressing persistent challenges in digital image analysis of cancerous tissues

The National Cancer Institute (NCI) supports many research programs and consortia, many of which use imaging as a major modality for characterizing cancerous tissue. A trans-consortia Image Analysis Working Group (IAWG) was established in 2019 with a mission to disseminate imaging-related work and foster collaborations. In 2022, the IAWG held a virtual hackathon focused on addressing challenges of analyzing high dimensional datasets from fixed cancerous tissues. Standard image processing techniques have automated feature extraction, but the next generation of imaging data requires more advanced methods to fully utilize the available information. In this perspective, we discuss current limitations of the automated analysis of multiplexed tissue images, the first steps toward deeper understanding of these limitations, what possible solutions have been developed, any new or refined approaches that were developed during the Image Analysis Hackathon 2022, and where further effort is required. The outstanding problems addressed in the hackathon fell into three main themes: 1) challenges to cell type classification and assessment, 2) translation and visual representation of spatial aspects of high dimensional data, and 3) scaling digital image analyses to large (multi-TB) datasets. We describe the rationale for each specific challenge and the progress made toward addressing it during the hackathon. We also suggest areas that would benefit from more focus and offer insight into broader challenges that the community will need to address as new technologies are developed and integrated into the broad range of image-based modalities and analytical resources already in use within the cancer research community.

CellSighter: a neural network to classify cells in highly multiplexed images

Multiplexed imaging enables measurement of multiple proteins in situ, offering an unprecedented opportunity to chart various cell types and states in tissues. However, cell classification, the task of identifying the type of individual cells, remains challenging, labor-intensive, and limiting to throughput. Here, we present CellSighter, a deep-learning based pipeline to accelerate cell classification in multiplexed images. Given a small training set of expert-labeled images, CellSighter outputs the label probabilities for all cells in new images. CellSighter achieves over 80% accuracy for major cell types across imaging platforms, which approaches inter-observer concordance. Ablation studies and simulations show that CellSighter is able to generalize its training data and learn features of protein expression levels, as well as spatial features such as subcellular expression patterns. CellSighter's design reduces overfitting, and it can be trained with only thousands or even hundreds of labeled examples. CellSighter also outputs a prediction confidence, allowing downstream experts control over the results. Altogether, CellSighter drastically reduces hands-on time for cell classification in multiplexed images, while improving accuracy and consistency across datasets.

MAPS: Pathologist-level cell type annotation from tissue images through machine learning

Highly multiplexed protein imaging is emerging as a potent technique for analyzing protein distribution within cells and tissues in their native context. However, existing cell annotation methods utilizing high-plex spatial proteomics data are resource intensive and necessitate iterative expert input, thereby constraining their scalability and practicality for extensive datasets. We introduce MAPS (Machine learning for Analysis of Proteomics in Spatial biology), a machine learning approach facilitating rapid and precise cell type identification with human-level accuracy from spatial proteomics data. Validated on multiple in-house and publicly available MIBI and CODEX datasets, MAPS outperforms current annotation techniques in terms of speed and accuracy, achieving pathologist-level precision even for challenging cell types, including tumor cells of immune origin. By democratizing rapidly deployable and scalable machine learning annotation, MAPS holds significant potential to expedite advances in tissue biology and disease comprehension.

Different approaches to Imaging Mass Cytometry data analysis

Imaging Mass Cytometry (IMC) is a novel, high multiplexing imaging platform capable of simultaneously detecting and visualizing up to 40 different protein targets. It is a strong asset available for in-depth study of histology and pathophysiology of the tissues. Bearing in mind the robustness of this technique and the high spatial context of the data it gives, it is especially valuable in studying the biology of cancer and tumor microenvironment. IMC-derived data are not classical micrographic images, and due to the characteristics of the data obtained using IMC, the image analysis approach, in this case, can diverge to a certain degree from the classical image analysis pipelines. As the number of publications based on the IMC is on the rise, this trend is also followed by an increase in the number of available methodologies designated solely to IMC-derived data analysis. This review has for an aim to give a systematic synopsis of all the available classical image analysis tools and pipelines useful to be employed for IMC data analysis and give an overview of tools intentionally developed solely for this purpose, easing the choice to researchers of selecting the most suitable methodologies for a specific type of analysis desired.

Single-cell high-dimensional imaging mass cytometry: one step beyond in oncology

Solid tumors have a dynamic ecosystem in which malignant and non-malignant (endothelial, stromal, and immune) cell types constantly interact. Importantly, the abundance, localization, and functional orientation of each cell component within the tumor microenvironment vary significantly over time and in response to treatment. Such intratumoral heterogeneity influences the tumor course and its sensitivity to treatments. Recently, high-dimensional imaging mass cytometry (IMC) has been developed to explore the tumor ecosystem at the single-cell level. In the last years, several studies demonstrated that IMC is a powerful tool to decipher the tumor complexity. In this review, we summarize the potential of this technology and how it may be useful for cancer research (from preclinical to clinical studies).

Highly multiplexed spatial profiling with CODEX: bioinformatic analysis and application in human disease

Multiplexed imaging, which enables spatial localization of proteins and RNA to cells within tissues, complements existing multi-omic technologies and has deepened our understanding of health and disease. CODEX, a multiplexed single-cell imaging technology, utilizes a microfluidics system that incorporates DNA barcoded antibodies to visualize 50 + cellular markers at the single-cell level. Here, we discuss the latest applications of CODEX to studies of cancer, autoimmunity, and infection as well as current bioinformatics approaches for analysis of multiplexed imaging data from preprocessing to cell segmentation and marker quantification to spatial analysis techniques. We conclude with a commentary on the challenges and future developments for multiplexed spatial profiling.

Spatial omics technologies at multimodal and single cell/subcellular level

Spatial omics technologies enable a deeper understanding of cellular organizations and interactions within a tissue of interest. These assays can identify specific compartments or regions in a tissue with differential transcript or protein abundance, delineate their interactions, and complement other methods in defining cellular phenotypes. A variety of spatial methodologies are being developed and commercialized; however, these techniques differ in spatial resolution, multiplexing capability, scale/throughput, and coverage. Here, we review the current and prospective landscape of single cell to subcellular resolution spatial omics technologies and analysis tools to provide a comprehensive picture for both research and clinical applications.

Spatial epitope barcoding reveals clonal tumor patch behaviors

Intratumoral heterogeneity is a seminal feature of human tumors contributing to tumor progression and response to treatment. Current technologies are still largely unsuitable to accurately track phenotypes and clonal evolution within tumors, especially in response to genetic manipulations. Here, we developed epitopes for imaging using combinatorial tagging (EpicTags), which we coupled to multiplexed ion beam imaging (EpicMIBI) for in situ tracking of barcodes within tissue microenvironments. Using EpicMIBI, we dissected the spatial component of cell lineages and phenotypes in xenograft models of small cell lung cancer. We observed emergent properties from mixed clones leading to the preferential expansion of clonal patches for both neuroendocrine and non-neuroendocrine cancer cell states in these models. In a tumor model harboring a fraction of PTEN-deficient cancer cells, we observed a non-autonomous increase of clonal patch size in PTEN wild-type cancer cells. EpicMIBI facilitates in situ interrogation of cell-intrinsic and cell-extrinsic processes involved in intratumoral heterogeneity.

Intravital and high-content multiplex imaging of the immune system

Highly motile and functionally diverse immune cells orchestrate effective immune responses through complex and dynamic cooperative behavior. Multiphoton intravital microscopy (MP-IVM) presents a unique and powerful tool to study the coordinated action of immune cell interactions in situ. Here, we review the current state of intravital microscopy in deepening our understanding of the immune system and discuss its fundamental limitations. In addition, we draw insights from recent technical advances in multiplex static tissue-imaging methods and propose an approach that could enable simultaneous visualization of cellular dynamics, deep phenotyping, and transcriptional states through a new type of correlative microscopy that combines these imaging technologies with advances in complex data analysis.

Drawing Polycyclic Molecules

A personal perspective on the practice of drawing polycyclic molecules and its implications for understanding and undertaking chemistry is presented.