MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging

Quantitative image analysis pipeline for detecting circulating hybrid cells in immunofluorescence images with human-level accuracy

Circulating hybrid cells (CHCs) are a newly discovered, tumor-derived cell population found in the peripheral blood of cancer patients and are thought to contribute to tumor metastasis. However, identifying CHCs by immunofluorescence (IF) imaging of patient peripheral blood mononuclear cells (PBMCs) is a time-consuming and subjective process that currently relies on manual annotation by laboratory technicians. Additionally, while IF is relatively easy to apply to tissue sections, its application to PBMC smears presents challenges due to the presence of biological and technical artifacts. To address these challenges, we present a robust image analysis pipeline to automate the detection and analysis of CHCs in IF images. The pipeline incorporates quality control to optimize specimen preparation protocols and remove unwanted artifacts, leverages a β-variational autoencoder (VAE) to learn meaningful latent representations of single-cell images, and employs a support vector machine (SVM) classifier to achieve human-level CHC detection. We created a rigorously labeled IF CHC data set including nine patients and two disease sites with the assistance of 10 annotators to evaluate the pipeline. We examined annotator variation and bias in CHC detection and provided guidelines to optimize the accuracy of CHC annotation. We found that all annotators agreed on CHC identification for only 65% of the cells in the data set and had a tendency to underestimate CHC counts for regions of interest (ROIs) containing relatively large amounts of cells (>50,000) when using the conventional enumeration method. On the other hand, our proposed approach is unbiased to ROI size. The SVM classifier trained on the β-VAE embeddings achieved an F1 score of 0.80, matching the average performance of human annotators. Our pipeline enables researchers to explore the role of CHCs in cancer progression and assess their potential as a clinical biomarker for metastasis. Further, we demonstrate that the pipeline can identify discrete cellular phenotypes among PBMCs, highlighting its utility beyond CHCs.

psudo: Exploring Multi-Channel Biomedical Image Data with Spatially and Perceptually Optimized Pseudocoloring

Over the past century, multichannel fluorescence imaging has been pivotal in myriad scientific breakthroughs by enabling the spatial visualization of proteins within a biological sample. With the shift to digital methods and visualization software, experts can now flexibly pseudocolor and combine image channels, each corresponding to a different protein, to explore their spatial relationships. We thus propose psudo, an interactive system that allows users to create optimal color palettes for multichannel spatial data. In psudo, a novel optimization method generates palettes that maximize the perceptual differences between channels while mitigating confusing color blending in overlapping channels. We integrate this method into a system that allows users to explore multi-channel image data and compare and evaluate color palettes for their data. An interactive lensing approach provides on-demand feedback on channel overlap and a color confusion metric while giving context to the underlying channel values. Color palettes can be applied globally or, using the lens, to local regions of interest. We evaluate our palette optimization approach using three graphical perception tasks in a crowdsourced user study with 150 participants, showing that users are more accurate at discerning and comparing the underlying data using our approach. Additionally, we showcase psudo in a case study exploring the complex immune responses in cancer tissue data with a biologist.

Phase II Study of Eribulin plus Pembrolizumab in Metastatic Soft-tissue Sarcomas: Clinical Outcomes and Biological Correlates

PURPOSE

Eribulin modulates the tumor-immune microenvironment via cGAS-STING signaling in preclinical models. This non-randomized phase II trial evaluated the combination of eribulin and pembrolizumab in patients with soft-tissue sarcomas (STS).

PATIENTS AND METHODS

Patients enrolled in one of three cohorts: leiomyosarcoma (LMS), liposarcomas (LPS), or other STS that may benefit from PD-1 inhibitors, including undifferentiated pleomorphic sarcoma (UPS). Eribulin was administered at 1.4 mg/m2 i.v. (days 1 and 8) with fixed-dose pembrolizumab 200 mg i.v. (day 1) of each 21-day cycle, until progression, unacceptable toxicity, or completion of 2 years of treatment. The primary endpoint was the 12-week progression-free survival rate (PFS-12) in each cohort. Secondary endpoints included the objective response rate, median PFS, safety profile, and overall survival (OS). Pretreatment and on-treatment blood specimens were evaluated in patients who achieved durable disease control (DDC) or progression within 12 weeks [early progression (EP)]. Multiplexed immunofluorescence was performed on archival LPS samples from patients with DDC or EP.

RESULTS

Fifty-seven patients enrolled (LMS, n = 19; LPS, n = 20; UPS/Other, n = 18). The PFS-12 was 36.8% (90% confidence interval: 22.5-60.4) for LMS, 69.6% (54.5-89.0) for LPS, and 52.6% (36.8-75.3) for UPS/Other cohorts. All 3 patients in the UPS/Other cohort with angiosarcoma achieved RECIST responses. Toxicity was manageable. Higher IFNα and IL4 serum levels were associated with clinical benefit. Immune aggregates expressing PD-1 and PD-L1 were observed in a patient that completed 2 years of treatment.

CONCLUSIONS

The combination of eribulin and pembrolizumab demonstrated promising activity in LPS and angiosarcoma.

SpatialCells: automated profiling of tumor microenvironments with spatially resolved multiplexed single-cell data

Cancer is a complex cellular ecosystem where malignant cells coexist and interact with immune, stromal and other cells within the tumor microenvironment (TME). Recent technological advancements in spatially resolved multiplexed imaging at single-cell resolution have led to the generation of large-scale and high-dimensional datasets from biological specimens. This underscores the necessity for automated methodologies that can effectively characterize molecular, cellular and spatial properties of TMEs for various malignancies. This study introduces SpatialCells, an open-source software package designed for region-based exploratory analysis and comprehensive characterization of TMEs using multiplexed single-cell data. The source code and tutorials are available at https://semenovlab.github.io/SpatialCells. SpatialCells efficiently streamlines the automated extraction of features from multiplexed single-cell data and can process samples containing millions of cells. Thus, SpatialCells facilitates subsequent association analyses and machine learning predictions, making it an essential tool in advancing our understanding of tumor growth, invasion and metastasis.

Quality Control for Single Cell Analysis of High-plex Tissue Profiles using CyLinter

Tumors are complex assemblies of cellular and acellular structures patterned on spatial scales from microns to centimeters. Study of these assemblies has advanced dramatically with the introduction of high-plex spatial profiling. Image-based profiling methods reveal the intensities and spatial distributions of 20-100 proteins at subcellular resolution in 103-107 cells per specimen. Despite extensive work on methods for extracting single-cell data from these images, all tissue images contain artefacts such as folds, debris, antibody aggregates, optical aberrations and image processing errors that arise from imperfections in specimen preparation, data acquisition, image assembly, and feature extraction. We show that these artefacts dramatically impact single-cell data analysis, obscuring meaningful biological interpretation. We describe an interactive quality control software tool, CyLinter, that identifies and removes data associated with imaging artefacts. CyLinter greatly improves single-cell analysis, especially for archival specimens sectioned many years prior to data collection, such as those from clinical trials.

Multiplex protein imaging in tumour biology

Tissue imaging has become much more colourful in the past decade. Advances in both experimental and analytical methods now make it possible to image protein markers in tissue samples in high multiplex. The ability to routinely image 40-50 markers simultaneously, at single-cell or subcellular resolution, has opened up new vistas in the study of tumour biology. Cellular phenotypes, interaction, communication and spatial organization have become amenable to molecular-level analysis, and application to patient cohorts has identified clinically relevant cellular and tissue features in several cancer types. Here, we review the use of multiplex protein imaging methods to study tumour biology, discuss ongoing attempts to combine these approaches with other forms of spatial omics, and highlight challenges in the field.

Semi-automated approaches for interrogating spatial heterogeneity of tissue samples

Tissues are spatially orchestrated ecosystems composed of heterogeneous cell populations and non-cellular elements. Tissue components' interactions shape the biological processes that govern homeostasis and disease, thus comprehensive insights into tissues' composition are crucial for understanding their biology. Recently, advancements in the spatial biology field enabled the in-depth analyses of tissue architecture at single-cell resolution, while preserving the structural context. The increasing number of biomarkers analyzed, together with whole tissue imaging, generate datasets approaching several hundreds of gigabytes in size, which are rich sources of valuable knowledge but require investments in infrastructure and resources for extracting quantitative information. The analysis of multiplex whole-tissue images requires extensive training and experience in data analysis. Here, we showcase how a set of open-source tools can allow semi-automated image data extraction to study the spatial composition of tissues with a focus on tumor microenvironment (TME). With the use of Lunaphore COMET platform, we interrogated lung cancer specimens where we examined the expression of 20 biomarkers. Subsequently, the tissue composition was interrogated using an in-house optimized nuclei detection algorithm followed by a newly developed image artifact exclusion approach. Thereafter, the data was processed using several publicly available tools, highlighting the compatibility of COMET-derived data with currently available image analysis frameworks. In summary, we showcased an innovative semi-automated workflow that highlights the ease of adoption of multiplex imaging to explore TME composition at single-cell resolution using a simple slide in, data out approach. Our workflow is easily transferrable to various cohorts of specimens to provide a toolset for spatial cellular dissection of the tissue composition.

Extended correlation functions for spatial analysis of multiplex imaging data

Imaging platforms for generating highly multiplexed histological images are being continually developed and improved. Significant improvements have also been made in the accuracy of methods for automated cell segmentation and classification. However, less attention has focused on the quantification and analysis of the resulting point clouds, which describe the spatial coordinates of individual cells. We focus here on a particular spatial statistical method, the cross-pair correlation function (cross-PCF), which can identify positive and negative spatial correlation between cells across a range of length scales. However, limitations of the cross-PCF hinder its widespread application to multiplexed histology. For example, it can only consider relations between pairs of cells, and cells must be classified using discrete categorical labels (rather than labeling continuous labels such as stain intensity). In this paper, we present three extensions to the cross-PCF which address these limitations and permit more detailed analysis of multiplex images: topographical correlation maps can visualize local clustering and exclusion between cells; neighbourhood correlation functions can identify colocalization of two or more cell types; and weighted-PCFs describe spatial correlation between points with continuous (rather than discrete) labels. We apply the extended PCFs to synthetic and biological datasets in order to demonstrate the insight that they can generate.

Epigenome and early selection determine the tumour-immune evolutionary trajectory of colorectal cancer

Immune system control is a major hurdle that cancer evolution must circumvent. The relative timing and evolutionary dynamics of subclones that have escaped immune control remain incompletely characterized, and how immune-mediated selection shapes the epigenome has received little attention. Here, we infer the genome- and epigenome-driven evolutionary dynamics of tumour-immune coevolution within primary colorectal cancers (CRCs). We utilise our existing CRC multi-region multi-omic dataset that we supplement with high-resolution spatially-resolved neoantigen sequencing data and highly multiplexed imaging of the tumour microenvironment (TME). Analysis of somatic chromatin accessibility alterations (SCAAs) reveals frequent somatic loss of accessibility at antigen presenting genes, and that SCAAs contribute to silencing of neoantigens. We observe that strong immune escape and exclusion occur at the outset of CRC formation, and that within tumours, including at the microscopic level of individual tumour glands, additional immune escape alterations have negligible consequences for the immunophenotype of cancer cells. Further minor immuno-editing occurs during local invasion and is associated with TME reorganisation, but that evolutionary bottleneck is relatively weak. Collectively, we show that immune evasion in CRC follows a "Big Bang" evolutionary pattern, whereby genetic, epigenetic and TME-driven immune evasion acquired by the time of transformation defines subsequent cancer-immune evolution.

A perspective on FAIR quality control in multiplexed imaging data processing

Multiplexed imaging approaches are getting increasingly adopted for imaging of large tissue areas, yielding big imaging datasets both in terms of the number of samples and the size of image data per sample. The processing and analysis of these datasets is complex owing to frequent technical artifacts and heterogeneous profiles from a high number of stained targets To streamline the analysis of multiplexed images, automated pipelines making use of state-of-the-art algorithms have been developed. In these pipelines, the output quality of one processing step is typically dependent on the output of the previous step and errors from each step, even when they appear minor, can propagate and confound the results. Thus, rigorous quality control (QC) at each of these different steps of the image processing pipeline is of paramount importance both for the proper analysis and interpretation of the analysis results and for ensuring the reusability of the data. Ideally, QC should become an integral and easily retrievable part of the imaging datasets and the analysis process. Yet, limitations of the currently available frameworks make integration of interactive QC difficult for large multiplexed imaging data. Given the increasing size and complexity of multiplexed imaging datasets, we present the different challenges for integrating QC in image analysis pipelines as well as suggest possible solutions that build on top of recent advances in bioimage analysis.

Harmonizing the Generation and Pre-publication Stewardship of FAIR Image data

Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled "Enabling Global Image Data Sharing in the Life Sciences," which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data (arXiv:2401.13023 [q-bio.OT], https://doi.org/10.48550/arXiv.2401.13023). In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges, and democratize access to common practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location.

Tissue niches and immunopathology through the lens of spatial tissue profiling techniques

Spatial organization plays a fundamental role in biology, influencing the function of biological structures at various levels. The immune system, in particular, relies on the orchestrated interactions of immune cells with their microenvironment to mount protective or pathogenic immune responses. The COVID-19 pandemic has underscored the significance of studying immunity within target organs to understand disease progression and severity. To achieve this, multiplex histology and spatial transcriptomics have proven indispensable in providing a spatial context to protein and gene expression patterns. By combining these techniques, researchers gain a more comprehensive understanding of the complex interactions at the cellular and molecular level in distinct tissue niches, key functional units modulating health and disease. In this review, we discuss recent advances in spatial tissue profiling techniques, highlighting their advantages over traditional histopathology studies. The insights gained from these approaches have the potential to revolutionize the diagnosis and treatment of various diseases including cancer, autoimmune disorders, and infectious diseases. However, we also acknowledge their challenges and limitations. Despite these, spatial tissue profiling offers promising opportunities to improve our understanding of how tissue niches direct regional immunity, and their relevance in tissue immunopathology, as a basis for novel therapeutic strategies and personalized medicine.

Spatial Morphoproteomic Features Predict Uniqueness of Immune Microarchitectures and Responses in Lymphoid Follicles

Multiplex imaging technologies allow the characterization of single cells in their cellular environments. Understanding the organization of single cells within their microenvironment and quantifying disease-status related biomarkers is essential for multiplex datasets. Here we proposed SNOWFLAKE, a graph neural network framework pipeline for the prediction of disease-status from combined multiplex cell expression and morphology in human B-cell follicles. We applied SNOWFLAKE to a multiplex dataset related to COVID-19 infection in humans and showed better predictive power of the SNOWFLAKE pipeline compared to other machine learning and deep learning methods. Moreover, we combined morphological features inside graph edge features to utilize attribution methods for extracting disease-relevant motifs from single-cell spatial graphs. The underlying subgraphs were further analyzed and associated with disease status across the dataset. We showed that SNOWFLAKE successfully extracted significant low dimensional embedding from subgraphs with a clear separation between disease status and helped characterize unique cellular interactions in the subgraphs. SNOWFLAKE is a generalizable pipeline for the analysis of multiplex imaging data modality by extracting disease-relevant subgraphs guided by graph-level prediction.

Qualification of a multiplexed tissue imaging assay and detection of novel patterns of HER2 heterogeneity in breast cancer

Emerging data suggests that HER2 intratumoral heterogeneity (ITH) is associated with therapy resistance, highlighting the need for new strategies to assess HER2 ITH. A promising approach is leveraging multiplexed tissue analysis techniques such as cyclic immunofluorescence (CyCIF), which enable visualization and quantification of 10-60 antigens at single-cell resolution from individual tissue sections. In this study, we qualified a breast cancer-specific antibody panel, including HER2, ER, and PR, for multiplexed tissue imaging. We then compared the performance of these antibodies against established clinical standards using pixel-, cell- and tissue-level analyses, utilizing 866 tissue cores (representing 294 patients). To ensure reliability, the CyCIF antibodies were qualified against HER2 immunohistochemistry (IHC) and fluorescence in situ hybridization (FISH) data from the same samples. Our findings demonstrate the successful qualification of a breast cancer antibody panel for CyCIF, showing high concordance with established clinical antibodies. Subsequently, we employed the qualified antibodies, along with antibodies for CD45, CD68, PD-L1, p53, Ki67, pRB, and AR, to characterize 567 HER2+ invasive breast cancer samples from 189 patients. Through single-cell analysis, we identified four distinct cell clusters within HER2+ breast cancer exhibiting heterogeneous HER2 expression. Furthermore, these clusters displayed variations in ER, PR, p53, AR, and PD-L1 expression. To quantify the extent of heterogeneity, we calculated heterogeneity scores based on the diversity among these clusters. Our analysis revealed expression patterns that are relevant to breast cancer biology, with correlations to HER2 ITH and potential relevance to clinical outcomes.

Imaging Mass Cytometry for In Situ Immune Profiling

The complexities and cellular heterogeneity associated with tissues necessitate the concurrent detection of markers beyond the limitations of conventional imaging approaches in order to spatially resolve the relationships between immune cell populations and their environments. This is a necessary complement to single-cell suspension-based methods to inform a better understanding of the events that may underlie pathological conditions. Imaging mass cytometry is a high-dimensional imaging modality that allows for the concurrent detection of up to 40 protein markers of interest across tissues at subcellular resolution. Here, we present an optimized staining protocol for imaging mass cytometry with modifications that integrate RNAscope. This unique addition enables combined protein and single-molecule RNA detection, thereby expanding the utility of imaging mass cytometry to researchers investigating low abundance or noncoding targets. In general, the procedure described is broadly applicable for comprehensive immune profiling of host-pathogen interactions, tumor microenvironments and inflammatory conditions, all within the tissue contexture.

Cell Cycle Mapping Using Multiplexed Immunofluorescence

The development of technologies that allow measurement of the cell cycle at the single-cell level has revealed novel insights into the mechanisms that regulate cell cycle commitment and progression through DNA replication and cell division. These studies have also provided evidence of heterogeneity in cell cycle regulation among individual cells, even within a genetically identical population. Cell cycle mapping combines highly multiplexed imaging with manifold learning to visualize the diversity of "paths" that cells can take through the proliferative cell cycle or into various states of cell cycle arrest. In this chapter, we describe a general protocol of the experimental and computational components of cell cycle mapping. We also provide a comprehensive guide for the design and analysis of experiments, discussing key considerations in detail (e.g., antibody library preparation, analysis strategies, etc.) that may vary depending on the research question being addressed.

OPTIMAL: An OPTimized Imaging Mass cytometry AnaLysis framework for benchmarking segmentation and data exploration

Analysis of imaging mass cytometry (IMC) data and other low-resolution multiplexed tissue imaging technologies is often confounded by poor single-cell segmentation and suboptimal approaches for data visualization and exploration. This can lead to inaccurate identification of cell phenotypes, states, or spatial relationships compared to reference data from single-cell suspension technologies. To this end we have developed the "OPTimized Imaging Mass cytometry AnaLysis (OPTIMAL)" framework to benchmark any approaches for cell segmentation, parameter transformation, batch effect correction, data visualization/clustering, and spatial neighborhood analysis. Using a panel of 27 metal-tagged antibodies recognizing well-characterized phenotypic and functional markers to stain the same Formalin-Fixed Paraffin Embedded (FFPE) human tonsil sample tissue microarray over 12 temporally distinct batches we tested several cell segmentation models, a range of different arcsinh cofactor parameter transformation values, 5 different dimensionality reduction algorithms, and 2 clustering methods. Finally, we assessed the optimal approach for performing neighborhood analysis. We found that single-cell segmentation was improved by the use of an Ilastik-derived probability map but that issues with poor segmentation were only really evident after clustering and cell type/state identification and not always evident when using "classical" bivariate data display techniques. The optimal arcsinh cofactor for parameter transformation was 1 as it maximized the statistical separation between negative and positive signal distributions and a simple Z-score normalization step after arcsinh transformation eliminated batch effects. Of the five different dimensionality reduction approaches tested, PacMap gave the best data structure with FLOWSOM clustering out-performing phenograph in terms of cell type identification. We also found that neighborhood analysis was influenced by the method used for finding neighboring cells with a "disc" pixel expansion outperforming a "bounding box" approach combined with the need for filtering objects based on size and image-edge location. Importantly, OPTIMAL can be used to assess and integrate with any existing approach to IMC data analysis and, as it creates .FCS files from the segmentation output and allows for single-cell exploration to be conducted using a wide variety of accessible software and algorithms familiar to conventional flow cytometrists.

Imputing Single-Cell Protein Abundance in Multiplex Tissue Imaging

Multiplex tissue imaging are a collection of increasingly popular single-cell spatial proteomics and transcriptomics assays for characterizing biological tissues both compositionally and spatially. However, several technical issues limit the utility of multiplex tissue imaging, including the limited number of RNAs and proteins that can be assayed, tissue loss, and protein probe failure. In this work, we demonstrate how machine learning methods can address these limitations by imputing protein abundance at the single-cell level using multiplex tissue imaging datasets from a breast cancer cohort. We first compared machine learning methods' strengths and weaknesses for imputing single-cell protein abundance. Machine learning methods used in this work include regularized linear regression, gradient-boosted regression trees, and deep learning autoencoders. We also incorporated cellular spatial information to improve imputation performance. Using machine learning, single-cell protein expression can be imputed with mean absolute error ranging between 0.05-0.3 on a [0,1] scale. Our results demonstrate (1) the feasibility of imputing single-cell abundance levels for many proteins using machine learning to overcome the technical constraints of multiplex tissue imaging and (2) how including cellular spatial information can substantially enhance imputation results.

Decoding the tumor microenvironment with spatial technologies

Visualization of the cellular heterogeneity and spatial architecture of the tumor microenvironment (TME) is becoming increasingly important to understand mechanisms of disease progression and therapeutic response. This is particularly relevant in the era of cancer immunotherapy, in which the contexture of immune cell positioning within the tumor landscape has been proven to affect efficacy. Although single-cell technologies have mostly replaced conventional approaches to analyze specific cellular subsets within tumors, those that integrate a spatial dimension are now on the rise. In this Review, we assess the strengths and limitations of emerging spatial technologies with a focus on their applications in tumor immunology, as well as forthcoming opportunities for artificial intelligence (AI) and the value of integrating multiomics datasets to achieve a holistic picture of the TME.

Cell Spotter (CSPOT): A machine-learning approach to automated cell spotting and quantification of highly multiplexed tissue images

Highly multiplexed tissue imaging and in situ spatial profiling aim to extract single-cell data from specimens containing closely packed cells of diverse morphology. This is challenging due to the difficulty of accurately assigning boundaries between cells (segmentation) and then generating per-cell staining intensities. Existing methods use gating to convert per-cell intensity data to positive and negative scores; this is a common approach in flow cytometry, but one that is problematic in imaging. In contrast, human experts identify cells in crowded environments using morphological, neighborhood, and intensity information. Here we describe a computational approach (Cell Spotter or CSPOT) that uses supervised machine learning in combination with classical segmentation to perform automated cell type calling. CSPOT is robust to artifacts that commonly afflict tissue imaging and can replace conventional gating. The end-to-end Python implementation of CSPOT can be integrated into cloud-based image processing pipelines to substantially improve the speed, accuracy, and reproducibility of single-cell spatial data.

SpatialCells: Automated Profiling of Tumor Microenvironments with Spatially Resolved Multiplexed Single-Cell Data

Background

Cancer is a complex cellular ecosystem where malignant cells coexist and interact with immune, stromal, and other cells within the tumor microenvironment. Recent technological advancements in spatially resolved multiplexed imaging at single-cell resolution have led to the generation of large-scale and high-dimensional datasets from biological specimens. This underscores the necessity for automated methodologies that can effectively characterize the molecular, cellular, and spatial properties of tumor microenvironments for various malignancies.

Results

This study introduces SpatialCells, an open-source software package designed for region-based exploratory analysis and comprehensive characterization of tumor microenvironments using multiplexed single-cell data.

Conclusions

SpatialCells efficiently streamlines the automated extraction of features from multiplexed single-cell data and can process samples containing millions of cells. Thus, SpatialCells facilitates subsequent association analyses and machine learning predictions, making it an essential tool in advancing our understanding of tumor growth, invasion, and metastasis.

Enablers and challenges of spatial omics, a melting pot of technologies

Spatial omics has emerged as a rapidly growing and fruitful field with hundreds of publications presenting novel methods for obtaining spatially resolved information for any omics data type on spatial scales ranging from subcellular to organismal. From a technology development perspective, spatial omics is a highly interdisciplinary field that integrates imaging and omics, spatial and molecular analyses, sequencing and mass spectrometry, and image analysis and bioinformatics. The emergence of this field has not only opened a window into spatial biology, but also created multiple novel opportunities, questions, and challenges for method developers. Here, we provide the perspective of technology developers on what makes the spatial omics field unique. After providing a brief overview of the state of the art, we discuss technological enablers and challenges and present our vision about the future applications and impact of this melting pot.

An end-to-end workflow for multiplexed image processing and analysis

Multiplexed imaging enables the simultaneous spatial profiling of dozens of biological molecules in tissues at single-cell resolution. Extracting biologically relevant information, such as the spatial distribution of cell phenotypes from multiplexed tissue imaging data, involves a number of computational tasks, including image segmentation, feature extraction and spatially resolved single-cell analysis. Here, we present an end-to-end workflow for multiplexed tissue image processing and analysis that integrates previously developed computational tools to enable these tasks in a user-friendly and customizable fashion. For data quality assessment, we highlight the utility of napari-imc for interactively inspecting raw imaging data and the cytomapper R/Bioconductor package for image visualization in R. Raw data preprocessing, image segmentation and feature extraction are performed using the steinbock toolkit. We showcase two alternative approaches for segmenting cells on the basis of supervised pixel classification and pretrained deep learning models. The extracted single-cell data are then read, processed and analyzed in R. The protocol describes the use of community-established data containers, facilitating the application of R/Bioconductor packages for dimensionality reduction, single-cell visualization and phenotyping. We provide instructions for performing spatially resolved single-cell analysis, including community analysis, cellular neighborhood detection and cell-cell interaction testing using the imcRtools R/Bioconductor package. The workflow has been previously applied to imaging mass cytometry data, but can be easily adapted to other highly multiplexed imaging technologies. This protocol can be implemented by researchers with basic bioinformatics training, and the analysis of the provided dataset can be completed within 5-6 h. An extended version is available at https://bodenmillergroup.github.io/IMCDataAnalysis/ .

Delineating spatial cell-cell interactions in the solid tumour microenvironment through the lens of highly multiplexed imaging

The growth and metastasis of solid tumours is known to be facilitated by the tumour microenvironment (TME), which is composed of a highly diverse collection of cell types that interact and communicate with one another extensively. Many of these interactions involve the immune cell population within the TME, referred to as the tumour immune microenvironment (TIME). These non-cell autonomous interactions exert substantial influence over cell behaviour and contribute to the reprogramming of immune and stromal cells into numerous pro-tumourigenic phenotypes. The study of some of these interactions, such as the PD-1/PD-L1 axis that induces CD8+ T cell exhaustion, has led to the development of breakthrough therapeutic advances. Yet many common analyses of the TME either do not retain the spatial data necessary to assess cell-cell interactions, or interrogate few (<10) markers, limiting the capacity for cell phenotyping. Recently developed digital pathology technologies, together with sophisticated bioimage analysis programs, now enable the high-resolution, highly-multiplexed analysis of diverse immune and stromal cell markers within the TME of clinical specimens. In this article, we review the tumour-promoting non-cell autonomous interactions in the TME and their impact on tumour behaviour. We additionally survey commonly used image analysis programs and highly-multiplexed spatial imaging technologies, and we discuss their relative advantages and limitations. The spatial organization of the TME varies enormously between patients, and so leveraging these technologies in future studies to further characterize how non-cell autonomous interactions impact tumour behaviour may inform the personalization of cancer treatment.​.

LOCATOR: feature extraction and spatial analysis of the cancer tissue microenvironment using mass cytometry imaging technologies

Motivation

Recent advances in highly multiplexed imaging have provided unprecedented insights into the complex cellular organization of tissues, with many applications in translational medicine. However, downstream analyses of multiplexed imaging data face several technical limitations, and although some computational methods and bioinformatics tools are available, deciphering the complex spatial organization of cellular ecosystems remains a challenging problem.

Results

To mitigate this problem, we develop a novel computational tool, LOCATOR (anaLysis Of CAncer Tissue micrOenviRonment), for spatial analysis of cancer tissue microenvironments using data acquired from mass cytometry imaging technologies. LOCATOR introduces a graph-based representation of tissue images to describe features of the cellular organization and deploys downstream analysis and visualization utilities that can be used for data-driven patient-risk stratification. Our case studies using mass cytometry imaging data from two well-annotated breast cancer cohorts re-confirmed that the spatial organization of the tumour-immune microenvironment is strongly associated with the clinical outcome in breast cancer. In addition, we report interesting potential associations between the spatial organization of macrophages and patients' survival. Our work introduces an automated and versatile analysis tool for mass cytometry imaging data with many applications in future cancer research projects.

Availability and implementation

Datasets and codes of LOCATOR are publicly available at https://github.com/RezvanEhsani/LOCATOR.

Multiplex imaging in immuno-oncology

Multiplex imaging has emerged as an invaluable tool for immune-oncologists and translational researchers, enabling them to examine intricate interactions among immune cells, stroma, matrix, and malignant cells within the tumor microenvironment (TME). It holds significant promise in the quest to discover improved biomarkers for treatment stratification and identify novel therapeutic targets. Nonetheless, several challenges exist in the realms of study design, experiment optimization, and data analysis. In this review, our aim is to present an overview of the utilization of multiplex imaging in immuno-oncology studies and inform novice researchers about the fundamental principles at each stage of the imaging and analysis process.

GammaGateR: semi-automated marker gating for single-cell multiplexed imaging

Motivation

Multiplexed immunofluorescence (mIF) is an emerging assay for multichannel protein imaging that can decipher cell-level spatial features in tissues. However, existing automated cell phenotyping methods, such as clustering, face challenges in achieving consistency across experiments and often require subjective evaluation. As a result, mIF analyses often revert to marker gating based on manual thresholding of raw imaging data.

Results

To address the need for an evaluable semi-automated algorithm, we developed GammaGateR, an R package for interactive marker gating designed specifically for segmented cell-level data from mIF images. Based on a novel closed-form gamma mixture model, GammaGateR provides estimates of marker-positive cell proportions and soft clustering of marker-positive cells. The model incorporates user-specified constraints that provide a consistent but slide-specific model fit. We compared GammaGateR against the newest unsupervised approach for annotating mIF data, employing two colon datasets and one ovarian cancer dataset for the evaluation. We showed that GammaGateR produces highly similar results to a silver standard established through manual annotation. Furthermore, we demonstrated its effectiveness in identifying biological signals, achieved by mapping known spatial interactions between CD68 and MUC5AC cells in the colon and by accurately predicting survival in ovarian cancer patients using the phenotype probabilities as input for machine learning methods. GammaGateR is a highly efficient tool that can improve the replicability of marker gating results, while reducing the time of manual segmentation.

Availability and Implementation

The R package is available at https://github.com/JiangmeiRubyXiong/GammaGateR.

Mapping human natural killer cell development in pediatric tonsil by imaging mass cytometry and high-resolution microscopy

Natural killer (NK) cells develop from CD34+ progenitors in a stage-specific manner defined by changes in cell surface receptor expression and function. Secondary lymphoid tissues, including tonsil, are sites of human NK cell development. Here we present new insights into human NK cell development in pediatric tonsil using cyclic immunofluorescence and imaging mass cytometry. We show that NK cell subset localization and interactions are dependent on NK cell developmental stage and tissue residency. NK cell progenitors are found in the interfollicular domain in proximity to cytokine-expressing stromal cells that promote proliferation and maturation. Mature NK cells are primarily found in the T-cell rich parafollicular domain engaging in cell-cell interactions that differ depending on their stage and tissue residency. The presence of local inflammation results in changes in NK cell interactions, abundance, and localization. This study provides the first comprehensive atlas of human NK cell development in secondary lymphoid tissue.

Machine learning implementation strategy in imaging and impedance flow cytometry

Imaging and impedance flow cytometry is a label-free technique that has shown promise as a potential replacement for standard flow cytometry. This is due to its ability to provide rich information and archive high-throughput analysis. Recently, significant efforts have been made to leverage machine learning for processing the abundant data generated by those techniques, enabling rapid and accurate analysis. Harnessing the power of machine learning, imaging and impedance flow cytometry has demonstrated its capability to address various complex phenotyping scenarios. Herein, we present a comprehensive overview of the detailed strategies for implementing machine learning in imaging and impedance flow cytometry. We initiate the discussion by outlining the commonly employed setup to acquire the data (i.e., image or signal) from the cell. Subsequently, we delve into the necessary processes for extracting features from the acquired image or signal data. Finally, we discuss how these features can be utilized for cell phenotyping through the application of machine learning algorithms. Furthermore, we discuss the existing challenges and provide insights for future perspectives of intelligent imaging and impedance flow cytometry.

Quantitative image analysis pipeline for detecting circulating hybrid cells in immunofluorescence images with human-level accuracy

Circulating hybrid cells (CHCs) are a newly discovered, tumor-derived cell population identified in the peripheral blood of cancer patients and are thought to contribute to tumor metastasis. However, identifying CHCs by immunofluorescence (IF) imaging of patient peripheral blood mononuclear cells (PBMCs) is a time-consuming and subjective process that currently relies on manual annotation by laboratory technicians. Additionally, while IF is relatively easy to apply to tissue sections, its application on PBMC smears presents challenges due to the presence of biological and technical artifacts. To address these challenges, we present a robust image analysis pipeline to automate the detection and analyses of CHCs in IF images. The pipeline incorporates quality control to optimize specimen preparation protocols and remove unwanted artifacts, leverages a β-variational autoencoder (VAE) to learn meaningful latent representations of single-cell images and employs a support vector machine (SVM) classifier to achieve human-level CHC detection. We created a rigorously labeled IF CHC dataset including 9 patients and 2 disease sites with the assistance of 10 annotators to evaluate the pipeline. We examined annotator variation and bias in CHC detection and then provided guidelines to optimize the accuracy of CHC annotation. We found that all annotators agreed on CHC identification for only 65% of the cells in the dataset and had a tendency to underestimate CHC counts for regions of interest (ROI) containing relatively large amounts of cells (>50,000) when using conventional enumeration methods. On the other hand, our proposed approach is unbiased to ROI size. The SVM classifier trained on the β-VAE encodings achieved an F1 score of 0.80, matching the average performance of annotators. Our pipeline enables researchers to explore the role of CHCs in cancer progression and assess their potential as a clinical biomarker for metastasis. Further, we demonstrate that the pipeline can identify discrete cellular phenotypes among PBMCs, highlighting its utility beyond CHCs.

Robust phenotyping of highly multiplexed tissue imaging data using pixel-level clustering

While technologies for multiplexed imaging have provided an unprecedented understanding of tissue composition in health and disease, interpreting this data remains a significant computational challenge. To understand the spatial organization of tissue and how it relates to disease processes, imaging studies typically focus on cell-level phenotypes. However, images can capture biologically important objects that are outside of cells, such as the extracellular matrix. Here, we describe a pipeline, Pixie, that achieves robust and quantitative annotation of pixel-level features using unsupervised clustering and show its application across a variety of biological contexts and multiplexed imaging platforms. Furthermore, current cell phenotyping strategies that rely on unsupervised clustering can be labor intensive and require large amounts of manual cluster adjustments. We demonstrate how pixel clusters that lie within cells can be used to improve cell annotations. We comprehensively evaluate pre-processing steps and parameter choices to optimize clustering performance and quantify the reproducibility of our method. Importantly, Pixie is open source and easily customizable through a user-friendly interface.

Virtual alignment of pathology image series for multi-gigapixel whole slide images

Interest in spatial omics is on the rise, but generation of highly multiplexed images remains challenging, due to cost, expertise, methodical constraints, and access to technology. An alternative approach is to register collections of whole slide images (WSI), generating spatially aligned datasets. WSI registration is a two-part problem, the first being the alignment itself and the second the application of transformations to huge multi-gigapixel images. To address both challenges, we developed Virtual Alignment of pathoLogy Image Series (VALIS), software which enables generation of highly multiplexed images by aligning any number of brightfield and/or immunofluorescent WSI, the results of which can be saved in the ome.tiff format. Benchmarking using publicly available datasets indicates VALIS provides state-of-the-art accuracy in WSI registration and 3D reconstruction. Leveraging existing open-source software tools, VALIS is written in Python, providing a free, fast, scalable, robust, and easy-to-use pipeline for registering multi-gigapixel WSI, facilitating downstream spatial analyses.

Addressing persistent challenges in digital image analysis of cancerous tissues

The National Cancer Institute (NCI) supports many research programs and consortia, many of which use imaging as a major modality for characterizing cancerous tissue. A trans-consortia Image Analysis Working Group (IAWG) was established in 2019 with a mission to disseminate imaging-related work and foster collaborations. In 2022, the IAWG held a virtual hackathon focused on addressing challenges of analyzing high dimensional datasets from fixed cancerous tissues. Standard image processing techniques have automated feature extraction, but the next generation of imaging data requires more advanced methods to fully utilize the available information. In this perspective, we discuss current limitations of the automated analysis of multiplexed tissue images, the first steps toward deeper understanding of these limitations, what possible solutions have been developed, any new or refined approaches that were developed during the Image Analysis Hackathon 2022, and where further effort is required. The outstanding problems addressed in the hackathon fell into three main themes: 1) challenges to cell type classification and assessment, 2) translation and visual representation of spatial aspects of high dimensional data, and 3) scaling digital image analyses to large (multi-TB) datasets. We describe the rationale for each specific challenge and the progress made toward addressing it during the hackathon. We also suggest areas that would benefit from more focus and offer insight into broader challenges that the community will need to address as new technologies are developed and integrated into the broad range of image-based modalities and analytical resources already in use within the cancer research community.

CellSighter: a neural network to classify cells in highly multiplexed images

Multiplexed imaging enables measurement of multiple proteins in situ, offering an unprecedented opportunity to chart various cell types and states in tissues. However, cell classification, the task of identifying the type of individual cells, remains challenging, labor-intensive, and limiting to throughput. Here, we present CellSighter, a deep-learning based pipeline to accelerate cell classification in multiplexed images. Given a small training set of expert-labeled images, CellSighter outputs the label probabilities for all cells in new images. CellSighter achieves over 80% accuracy for major cell types across imaging platforms, which approaches inter-observer concordance. Ablation studies and simulations show that CellSighter is able to generalize its training data and learn features of protein expression levels, as well as spatial features such as subcellular expression patterns. CellSighter's design reduces overfitting, and it can be trained with only thousands or even hundreds of labeled examples. CellSighter also outputs a prediction confidence, allowing downstream experts control over the results. Altogether, CellSighter drastically reduces hands-on time for cell classification in multiplexed images, while improving accuracy and consistency across datasets.

A Galaxy of informatics resources for MS-based proteomics

INTRODUCTION

Continuous advances in mass spectrometry (MS) technologies have enabled deeper and more reproducible proteome characterization and a better understanding of biological systems when integrated with other 'omics data. Bioinformatic resources meeting the analysis requirements of increasingly complex MS-based proteomic data and associated multi-omic data are critically needed. These requirements included availability of software that would span diverse types of analyses, scalability for large-scale, compute-intensive applications, and mechanisms to ease adoption of the software.

AREAS COVERED

The Galaxy ecosystem meets these requirements by offering a multitude of open-source tools for MS-based proteomics analyses and applications, all in an adaptable, scalable, and accessible computing environment. A thriving global community maintains these software and associated training resources to empower researcher-driven analyses.

EXPERT OPINION

The community-supported Galaxy ecosystem remains a crucial contributor to basic biological and clinical studies using MS-based proteomics. In addition to the current status of Galaxy-based resources, we describe ongoing developments for meeting emerging challenges in MS-based proteomic informatics. We hope this review will catalyze increased use of Galaxy by researchers employing MS-based proteomics and inspire software developers to join the community and implement new tools, workflows, and associated training content that will add further value to this already rich ecosystem.

High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers

Precision medicine is critically dependent on better methods for diagnosing and staging disease and predicting drug response. Histopathology using hematoxylin and eosin (H&E)-stained tissue (not genomics) remains the primary diagnostic method in cancer. Recently developed highly multiplexed tissue imaging methods promise to enhance research studies and clinical practice with precise, spatially resolved single-cell data. Here, we describe the 'Orion' platform for collecting H&E and high-plex immunofluorescence images from the same cells in a whole-slide format suitable for diagnosis. Using a retrospective cohort of 74 colorectal cancer resections, we show that immunofluorescence and H&E images provide human experts and machine learning algorithms with complementary information that can be used to generate interpretable, multiplexed image-based models predictive of progression-free survival. Combining models of immune infiltration and tumor-intrinsic features achieves a 10- to 20-fold discrimination between rapid and slow (or no) progression, demonstrating the ability of multimodal tissue imaging to generate high-performance biomarkers.

Recent advancements in tumour microenvironment landscaping for target selection and response prediction in immune checkpoint therapies achieved through spatial protein multiplexing analysis

Immune checkpoint therapies have significantly advanced cancer treatment. Nevertheless, the high costs and potential adverse effects associated with these therapies highlight the need for better predictive biomarkers to identify patients who are most likely to benefit from treatment. Unfortunately, the existing biomarkers are insufficient to identify such patients. New high-dimensional spatial technologies have emerged as a valuable tool for discovering novel biomarkers by analysing multiple protein markers at a single-cell resolution in tissue samples. These technologies provide a more comprehensive map of tissue composition, cell functionality, and interactions between different cell types in the tumour microenvironment. In this review, we provide an overview of how spatial protein-based multiplexing technologies have fuelled biomarker discovery and advanced the field of immunotherapy. In particular, we will focus on how these technologies contributed to (i) characterise the tumour microenvironment, (ii) understand the role of tumour heterogeneity, (iii) study the interplay of the immune microenvironment and tumour progression, (iv) discover biomarkers for immune checkpoint therapies (v) suggest novel therapeutic strategies.

Human T follicular helper clones seed the germinal center-resident regulatory pool

The mechanisms by which FOXP3+ T follicular regulatory (Tfr) cells simultaneously steer antibody formation toward microbe or vaccine recognition and away from self-reactivity remain incompletely understood. To explore underappreciated heterogeneity in human Tfr cell development, function, and localization, we used paired TCRVA/TCRVB sequencing to distinguish tonsillar Tfr cells that are clonally related to natural regulatory T cells (nTfr) from those likely induced from T follicular helper (Tfh) cells (iTfr). The proteins iTfr and nTfr cells differentially expressed were used to pinpoint their in situ locations via multiplex microscopy and establish their divergent functional roles. In silico analyses and in vitro tonsil organoid tracking models corroborated the existence of separate Treg-to-nTfr and Tfh-to-iTfr developmental trajectories. Our results identify human iTfr cells as a distinct CD38+, germinal center-resident, Tfh-descended subset that gains suppressive function while retaining the capacity to help B cells, whereas CD38- nTfr cells are elite suppressors primarily localized in follicular mantles. Interventions differentially targeting specific Tfr cell subsets may provide therapeutic opportunities to boost immunity or more precisely treat autoimmune diseases.

IMC-Denoise: a content aware denoising pipeline to enhance Imaging Mass Cytometry

Imaging Mass Cytometry (IMC) is an emerging multiplexed imaging technology for analyzing complex microenvironments using more than 40 molecularly-specific channels. However, this modality has unique data processing requirements, particularly for patient tissue specimens where signal-to-noise ratios for markers can be low, despite optimization, and pixel intensity artifacts can deteriorate image quality and downstream analysis. Here we demonstrate an automated content-aware pipeline, IMC-Denoise, to restore IMC images deploying a differential intensity map-based restoration (DIMR) algorithm for removing hot pixels and a self-supervised deep learning algorithm for shot noise image filtering (DeepSNiF). IMC-Denoise outperforms existing methods for adaptive hot pixel and background noise removal, with significant image quality improvement in modeled data and datasets from multiple pathologies. This includes in technically challenging human bone marrow; we achieve noise level reduction of 87% for a 5.6-fold higher contrast-to-noise ratio, and more accurate background noise removal with approximately 2 × improved F1 score. Our approach enhances manual gating and automated phenotyping with cell-scale downstream analyses. Verified by manual annotations, spatial and density analysis for targeted cell groups reveal subtle but significant differences of cell populations in diseased bone marrow. We anticipate that IMC-Denoise will provide similar benefits across mass cytometric applications to more deeply characterize complex tissue microenvironments.

SEG: Segmentation Evaluation in absence of Ground truth labels

Identifying individual cells or nuclei is often the first step in the analysis of multiplex tissue imaging (MTI) data. Recent efforts to produce plug-and-play, end-to-end MTI analysis tools such as MCMICRO1- though groundbreaking in their usability and extensibility - are often unable to provide users guidance regarding the most appropriate models for their segmentation task among an endless proliferation of novel segmentation methods. Unfortunately, evaluating segmentation results on a user's dataset without ground truth labels is either purely subjective or eventually amounts to the task of performing the original, time-intensive annotation. As a consequence, researchers rely on models pre-trained on other large datasets for their unique tasks. Here, we propose a methodological approach for evaluating MTI nuclei segmentation methods in absence of ground truth labels by scoring relatively to a larger ensemble of segmentations. To avoid potential sensitivity to collective bias from the ensemble approach, we refine the ensemble via weighted average across segmentation methods, which we derive from a systematic model ablation study. First, we demonstrate a proof-of-concept and the feasibility of the proposed approach to evaluate segmentation performance in a small dataset with ground truth annotation. To validate the ensemble and demonstrate the importance of our method-specific weighting, we compare the ensemble's detection and pixel-level predictions - derived without supervision - with the data's ground truth labels. Second, we apply the methodology to an unlabeled larger tissue microarray (TMA) dataset, which includes a diverse set of breast cancer phenotypes, and provides decision guidelines for the general user to more easily choose the most suitable segmentation methods for their own dataset by systematically evaluating the performance of individual segmentation approaches in the entire dataset.

Topological-Preserving Membrane Skeleton Segmentation in Multiplex Immunofluorescence Imaging

Multiplex immunofluorescence (MxIF) is an emerging imaging technology whose downstream molecular analytics highly rely upon the effectiveness of cell segmentation. In practice, multiple membrane markers (e.g., NaKATPase, PanCK and β-catenin) are employed to stain membranes for different cell types, so as to achieve a more comprehensive cell segmentation since no single marker fits all cell types. However, prevalent watershed-based image processing might yield inferior capability for modeling complicated relationships between markers. For example, some markers can be misleading due to questionable stain quality. In this paper, we propose a deep learning based membrane segmentation method to aggregate complementary information that is uniquely provided by large scale MxIF markers. We aim to segment tubular membrane structure in MxIF data using global (membrane markers z-stack projection image) and local (separate individual markers) information to maximize topology preservation with deep learning. Specifically, we investigate the feasibility of four SOTA 2D deep networks and four volumetric-based loss functions. We conducted a comprehensive ablation study to assess the sensitivity of the proposed method with various combinations of input channels. Beyond using adjusted rand index (ARI) as the evaluation metric, which was inspired by the clDice, we propose a novel volumetric metric that is specific for skeletal structure, denoted as c l D i c e S K E L . In total, 80 membrane MxIF images were manually traced for 5-fold cross-validation. Our model outperforms the baseline with a 20.2% and 41.3% increase in c l D i c e S K E L and ARI performance, which is significant (p<0.05) using the Wilcoxon signed rank test. Our work explores a promising direction for advancing MxIF imaging cell segmentation with deep learning membrane segmentation. Tools are available at https://github.com/MASILab/MxIF_Membrane_Segmentation.

Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer

Advanced solid cancers are complex assemblies of tumor, immune, and stromal cells characterized by high intratumoral variation. We use highly multiplexed tissue imaging, 3D reconstruction, spatial statistics, and machine learning to identify cell types and states underlying morphological features of known diagnostic and prognostic significance in colorectal cancer. Quantitation of these features in high-plex marker space reveals recurrent transitions from one tumor morphology to the next, some of which are coincident with long-range gradients in the expression of oncogenes and epigenetic regulators. At the tumor invasive margin, where tumor, normal, and immune cells compete, T cell suppression involves multiple cell types and 3D imaging shows that seemingly localized 2D features such as tertiary lymphoid structures are commonly interconnected and have graded molecular properties. Thus, while cancer genetics emphasizes the importance of discrete changes in tumor state, whole-specimen imaging reveals large-scale morphological and molecular gradients analogous to those in developing tissues.

Visinity: Visual Spatial Neighborhood Analysis for Multiplexed Tissue Imaging Data

New highly-multiplexed imaging technologies have enabled the study of tissues in unprecedented detail. These methods are increasingly being applied to understand how cancer cells and immune response change during tumor development, progression, and metastasis, as well as following treatment. Yet, existing analysis approaches focus on investigating small tissue samples on a per-cell basis, not taking into account the spatial proximity of cells, which indicates cell-cell interaction and specific biological processes in the larger cancer microenvironment. We present Visinity, a scalable visual analytics system to analyze cell interaction patterns across cohorts of whole-slide multiplexed tissue images. Our approach is based on a fast regional neighborhood computation, leveraging unsupervised learning to quantify, compare, and group cells by their surrounding cellular neighborhood. These neighborhoods can be visually analyzed in an exploratory and confirmatory workflow. Users can explore spatial patterns present across tissues through a scalable image viewer and coordinated views highlighting the neighborhood composition and spatial arrangements of cells. To verify or refine existing hypotheses, users can query for specific patterns to determine their presence and statistical significance. Findings can be interactively annotated, ranked, and compared in the form of small multiples. In two case studies with biomedical experts, we demonstrate that Visinity can identify common biological processes within a human tonsil and uncover novel white-blood cell networks and immune-tumor interactions.

Highly multiplexed spatial profiling with CODEX: bioinformatic analysis and application in human disease

Multiplexed imaging, which enables spatial localization of proteins and RNA to cells within tissues, complements existing multi-omic technologies and has deepened our understanding of health and disease. CODEX, a multiplexed single-cell imaging technology, utilizes a microfluidics system that incorporates DNA barcoded antibodies to visualize 50 + cellular markers at the single-cell level. Here, we discuss the latest applications of CODEX to studies of cancer, autoimmunity, and infection as well as current bioinformatics approaches for analysis of multiplexed imaging data from preprocessing to cell segmentation and marker quantification to spatial analysis techniques. We conclude with a commentary on the challenges and future developments for multiplexed spatial profiling.

Statistical Analysis of Multiplex Immunofluorescence and Immunohistochemistry Imaging Data

Advances in multiplexed single-cell immunofluorescence (mIF) and multiplex immunohistochemistry (mIHC) imaging technologies have enabled the analysis of cell-to-cell spatial relationships that promise to revolutionize our understanding of tissue-based diseases and autoimmune disorders. Multiplex images are collected as multichannel TIFF files; then denoised, segmented to identify cells and nuclei, normalized across slides with protein markers to correct for batch effects, and phenotyped; and then tissue composition and spatial context at the cellular level are analyzed. This chapter discusses methods and software infrastructure for image processing and statistical analysis of mIF/mIHC data.

Spatial biology of cancer evolution

The natural history of cancers can be understood through the lens of evolution given that the driving forces of cancer development are mutation and selection of fitter clones. Cancer growth and progression are spatial processes that involve the breakdown of normal tissue organization, invasion and metastasis. For these reasons, spatial patterns are an integral part of histological tumour grading and staging as they measure the progression from normal to malignant disease. Furthermore, tumour cells are part of an ecosystem of tumour cells and their surrounding tumour microenvironment. A range of new spatial genomic, transcriptomic and proteomic technologies offers new avenues for the study of cancer evolution with great molecular and spatial detail. These methods enable precise characterizations of the tumour microenvironment, cellular interactions therein and micro-anatomical structures. In conjunction with spatial genomics, it emerges that tumours and microenvironments co-evolve, which helps explain observable patterns of heterogeneity and offers new routes for therapeutic interventions.

UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues

Upcoming technologies enable routine collection of highly multiplexed (20–60 channel), subcellular resolution images of mammalian tissues for research and diagnosis. Extracting single cell data from such images requires accurate image segmentation, a challenging problem commonly tackled with deep learning. In this paper, we report two findings that substantially improve image segmentation of tissues using a range of machine learning architectures. First, we unexpectedly find that the inclusion of intentionally defocused and saturated images in training data substantially improves subsequent image segmentation. Such real augmentation outperforms computational augmentation (Gaussian blurring). In addition, we find that it is practical to image the nuclear envelope in multiple tissues using an antibody cocktail thereby better identifying nuclear outlines and improving segmentation. The two approaches cumulatively and substantially improve segmentation on a wide range of tissue types. We speculate that the use of real augmentations will have applications in image processing outside of microscopy.

Multiplexed and reproducible high content screening of live and fixed cells using Dye Drop

High-throughput measurement of cells perturbed using libraries of small molecules, gene knockouts, or different microenvironmental factors is a key step in functional genomics and pre-clinical drug discovery. However, it remains difficult to perform accurate single-cell assays in 384-well plates, limiting many studies to well-average measurements (e.g., CellTiter-Glo®). Here we describe a public domain Dye Drop method that uses sequential density displacement and microscopy to perform multi-step assays on living cells. We use Dye Drop cell viability and DNA replication assays followed by immunofluorescence imaging to collect single-cell dose-response data for 67 investigational and clinical-grade small molecules in 58 breast cancer cell lines. By separating the cytostatic and cytotoxic effects of drugs computationally, we uncover unexpected relationships between the two. Dye Drop is rapid, reproducible, customizable, and compatible with manual or automated laboratory equipment. Dye Drop improves the tradeoff between data content and cost, enabling the collection of information-rich perturbagen-response datasets.

Physiologically Based Pharmacokinetic Modeling of Nanoparticle Biodistribution: A Review of Existing Models, Simulation Software, and Data Analysis Tools

Cancer treatment and pharmaceutical development require targeted treatment and less toxic therapeutic intervention to achieve real progress against this disease. In this scenario, nanomedicine emerged as a reliable tool to improve drug pharmacokinetics and to translate to the clinical biologics based on large molecules. However, the ability of our body to recognize foreign objects together with carrier transport heterogeneity derived from the combination of particle physical and chemical properties, payload and surface modification, make the designing of effective carriers very difficult. In this scenario, physiologically based pharmacokinetic modeling can help to design the particles and eventually predict their ability to reach the target and treat the tumor. This effort is performed by scientists with specific expertise and skills and familiarity with artificial intelligence tools such as advanced software that are not usually in the “cords” of traditional medical or material researchers. The goal of this review was to highlight the advantages that computational modeling could provide to nanomedicine and bring together scientists with different background by portraying in the most simple way the work of computational developers through the description of the tools that they use to predict nanoparticle transport and tumor targeting in our body.