Transformer for one stop interpretable cell type annotation

TransAnno-Net: A Deep Learning Framework for Accurate Cell Type Annotation of Mouse Lung Tissue Using Self-supervised Pretraining

BACKGROUND

Single-cell RNA sequencing (scRNA-seq) has become a significant tool for addressing complex issuess in the field of biology. In the context of scRNA-seq analysis, it is imperative to accurately determine the type of each cell. However, conventional supervised or semi-supervised methodologies are contingent on expert labels and incur substantial labeling costs, In contrast self-supervised pre-training strategies leverage unlabeled data during the pre-training phase and utilise a limited amount of labeled data in the fine-tuning phase, thereby greatly reducing labor costs. Furthermore, the fine-tuning does not need to learn the feature representations from scratch, enhancing the efficiency and transferability of the model.

METHODS

The proposed methodology is outlined below. The deep learning framework, TransAnno-Net, is based on transfer learning and a Transformer architecture. It has been designed for efficient and accurate cell type annotations in large-scale scRNA-seq datasets of mouse lung organs. Specifically, TransAnno-Net is pre-trained on the scRNA-seq lung data of approximately 100,000 cells to acquire gene-gene similarities via self-supervised learning. It is then migrated to a relatively small number of datasets to fine-tune specific cell type annotation tasks. To address the issue of imbalance in cell types commonly observed in scRNA-seq data, we applied a random oversampling technique is applied to the fine-tuned dataset. This is done to mitigate the impact of distributional imbalance on the annotation outcomes.

RESULTS

The experimental findings demonstrate that TransAnno-Net exhibits superior performance with an AUC of 0.979, 0.901, and 0.982, respectively, on three mouse lung datasets, outperforming eight state-of-the-art (SOTA) methods. In addition, TransAnno-Net demonstrates robust performance on cross-organ, cross-platform datasets, and is competitive with the fully supervised learning-based method.

CONCLUSION

The TransAnno-Net method is a highly effective cross-platform and cross-data set single-cell type annotation method for mouse lung tissues and supports cross-organ cell type annotation. This approach is expected to enhance the efficiency of research on the biological mechanisms of complex biological systems and diseases.

An automatic annotation tool and reference database for T cell subtypes and states at single-cell resolution

T cells have various subtypes and states with different functions. However, a reference list and automated annotation tool for T cell subtypes and states are lacking, which is critical for analyzing and comparing T cells under various conditions. We constructed the largest human T cell reference, containing 1,348,268 T cells from 35 conditions and 16 tissues. We classified T cells into 33 subtypes and further stratified them into 68 categories according to subtype and state. Based on this reference, we developed a tool named STCAT to automatically annotate T cells from scRNA-seq data by hierarchical models and marker correction. The accuracy of STCAT was 28% higher than that of existing tools validated on six independent datasets, including cancer and healthy samples. Using STCAT, we consistently discovered that CD4+ Th17 cells were enriched in late-stage lung cancer patients in multiple datasets, whereas MAIT cells were prevalent in milder-stage COVID-19 patients. We also confirmed a decrease in Treg cytotoxicity in post-treatment ovarian cancer. Systematic landscape analyses of CD4+ and CD8+ T cell references revealed that CD4+ Treg cells were enriched in tumor samples and that CD8+ naive-related cells were abundant in healthy individuals. Finally, we deposited all the T cell references and annotations into a TCellAtlas (https://guolab.wchscu.cn/TCellAtlas) database, which allows users to browse T cell expression profiles and analyze customized scRNA-seq data by STCAT. In conclusion, comprehensive human T cell subtypes and states reference, automated annotation tool, and database will greatly facilitate research on T cell immunity and tumor immunology.

HL-BscPF: Hybrid learning facilitates brain cell auto-identification in multiple pathologies

AIMS

The rapidly growing scale and complexity of single-cell transcriptomic data in brain research make it increasingly difficult for traditional methods to extract meaningful insights efficiently, highlighting the need for artificial intelligence.

MATERIALS AND METHODS

We presented the Hybrid Learning-based Brain single-cell Prediction Framework (HL-BscPF), designed to automate cell type classification and reveal disease-related pathways in the brain. HL-BscPF integrates ItClust and TOSICA models, combining autoencoder-based dimensionality reduction with transformer architecture to enhance predictive accuracy. HL-BscPF was evaluated using brain scRNA-seq datasets representing various neuropathological states, and its predictive performance was benchmarked against ground-truth annotations.

KEY FINDINGS

Applied to four brain-specific single-cell datasets, including aging, Alzheimer's disease, postoperative cognitive dysfunction, and stroke, HL-BscPF accurately classified cell types and uncovered key functional alterations in neuronal and glial populations. TOSICA showed higher accuracy in large-scale datasets due to its multi-head self-attention capabilities, whereas ItClust performed optimally in cases with lower cell diversity, demonstrating their complementary strengths. By providing precise cell identification and novel insights into brain-specific pathway dysregulation, HL-BscPF offers a powerful tool for extracting meaningful insights from vast single-cell datasets, enabling a deeper understanding of the complex neuropathologies.

SIGNIFICANCE

HL-BscPF demonstrates exceptional accuracy and interpretability in cell type annotation and functional analysis, uncovering critical disease-related mechanisms. This framework offers a powerful tool for advancing single-cell research in brain pathologies.

Mature tertiary lymphoid structures evoke intra-tumoral T and B cell responses via progenitor exhausted CD4+ T cells in head and neck cancer

Tumor tertiary lymphoid structures (TLS), especially mature TLS (mTLS), have been associated with better prognosis and improved responses to immune checkpoint blockade (ICB), but the underlying mechanisms remain incompletely understood. Here, by performing single-cell RNA, antigen receptor sequencing and spatial transcriptomics on tumor tissue from head and neck squamous cell carcinoma (HNSCC) patients with different statuses of TLS, we observe that mTLS are enriched with stem-like T cells, and B cells at various maturation stages. Notably, progenitor exhausted CD4+ T cells, with features resembling follicular helper T cells, support these responses, by activating B cells to produce plasma cells in the germinal center, and interacting with DC-LAMP+ dendritic cells to support CD8+ T cell activation. Conversely, non-mTLS tumors do not promote local anti-tumor immunity which is abundant of immunosuppressive cells or a lack of stem-like B and T cells. Furthermore, patients with mTLS manifest improved overall survival and response to ICB compared to those with non-mTLS. Overall, our study provides insights into mechanisms underlying mTLS-mediated intra-tumoral immunity events against cancer.

Advances and applications in single-cell and spatial genomics

The applications of single-cell and spatial technologies in recent times have revolutionized the present understanding of cellular states and the cellular heterogeneity inherent in complex biological systems. These advancements offer unprecedented resolution in the examination of the functional genomics of individual cells and their spatial context within tissues. In this review, we have comprehensively discussed the historical development and recent progress in the field of single-cell and spatial genomics. We have reviewed the breakthroughs in single-cell multi-omics technologies, spatial genomics methods, and the computational strategies employed toward the analyses of single-cell atlas data. Furthermore, we have highlighted the advances made in constructing cellular atlases and their clinical applications, particularly in the context of disease. Finally, we have discussed the emerging trends, challenges, and opportunities in this rapidly evolving field.

Self-Supervised Graph Representation Learning for Single-Cell Classification

Accurately identifying cell types in single-cell RNA sequencing data is critical for understanding cellular differentiation and pathological mechanisms in downstream analysis. As traditional biological approaches are laborious and time-intensive, it is imperative to develop computational biology methods for cell classification. However, it remains a challenge for existing methods to adequately utilize the potential gene expression information within the vast amount of unlabeled cell data, which limits their classification and generalization performance. Therefore, we propose a novel self-supervised graph representation learning framework for single-cell classification, named scSSGC. Specifically, in the pre-training stage of self-supervised learning, multiple K-means clustering tasks conducted on unlabeled cell data are jointly employed for model training, thereby mitigating the issue of limited labeled data. To effectively capture the potential interactions among cells, we introduce a locally augmented graph neural network to enhance the information aggregation capability for nodes with fewer neighbors in the cell graph. A range of benchmark experiments demonstrates that scSSGC outperforms existing state-of-the-art cell classification methods. More importantly, scSSGC provides stable performance when faced with cross-datasets, indicating better generalization ability.

Mapping Cell Identity from scRNA-seq: A primer on computational methods

Single cell (sc) technologies mark a conceptual and methodological breakthrough in our way to study cells, the base units of life. Thanks to these technological developments, large-scale initiatives are currently ongoing aimed at mapping of all the cell types in the human body, with the ambitious aim to gain a cell-level resolution of physiological development and disease. Since its broad applicability and ease of interpretation scRNA-seq is probably the most common sc-based application. This assay uses high throughput RNA sequencing to capture gene expression profiles at the sc-level. Subsequently, under the assumption that differences in transcriptional programs correspond to distinct cellular identities, ad-hoc computational methods are used to infer cell types from gene expression patterns. A wide array of computational methods were developed for this task. However, depending on the underlying algorithmic approach and associated computational requirements, each method might have a specific range of application, with implications that are not always clear to the end user. Here we will provide a concise overview on state-of-the-art computational methods for cell identity annotation in scRNA-seq, tailored for new users and non-computational scientists. To this end, we classify existing tools in five main categories, and discuss their key strengths, limitations and range of application.

Application of bioinformatic tools in cell type classification for single-cell RNA-seq data

The advancements in single-cell RNA sequencing (scRNAseq) technology have significantly transformed genomics research, enabling the handling of thousands of cells in each experiment. As of now, 32,068 research studies have been cataloged in the Pubmed database. The primary aim of scRNAseq investigations is to identify cell types, understand the antitumor immune response, and identify new and uncommon cell types. Traditional techniques for identifying cell types include microscopy, histology, and pathological characteristics. However, the complexity of instruments and the need for precise experimental design make it difficult to fully capture the overall heterogeneity. Unsupervised clustering and supervised classification methods have been used to solve this task. Supervised cell type classification methods have gained popularity as large-scale, high-quality, well-annotated and more robust results compared to clustering methods. A recent study showed that support vector machine (SVM) gives a high-quality classification performance in different scenarios. In this article, we compare and evaluate the performance of four different kernels (sigmoid, linear, radial, polynomial) of SVM. The results of the experiments on three standard scRNA-seq datasets indicate that SVM with linear and SVM with sigmoid kernel classify the cells more accurately (approx. 99 %) where SVM linear kernel method has remarkably fast computation time and we also evaluate the results using some single cell specific evaluation matrices F-1 score, MCC, AUC value. Additionally, it sheds light on the potential use of kernels of SVM to give underlying information of single-cell RNA-Seq data more effectively.

UniMap: Type-Level Integration Enhances Biological Preservation and Interpretability in Single-Cell Annotation

Integrating single-cell datasets from multiple studies provides a cost-effective way to build comprehensive cell atlases, granting deeper insights into cellular characteristics across diverse biological systems. However, current data integration methods struggle with interference in partially overlapping datasets and varying annotation granularities. Here, a multiselective adversarial network is introduced for the first time and present UniMap, which functions as a "discerner" to identify and exclude interfering cells from various data sources during dataset integration. Compared to other state-of-the-art methods, UniMap emphasizes type-level integration and proves to be the best model for preserving biological variability, achieving noticeably higher accuracy in single-cell automated annotation under various circumstances. Additionally, it enhances interpretability by revealing shared and domain-specific cell types and providing prediction confidence. The efficacy of UniMap is demonstrated in terms of identifying new cell types, creating high-resolution cell atlases, annotating cells along developmental trajectories, and performing cross-species analysis, underscoring its potential as a robust tool for single-cell research.

Foundation models in bioinformatics

With the adoption of foundation models (FMs), artificial intelligence (AI) has become increasingly significant in bioinformatics and has successfully addressed many historical challenges, such as pre-training frameworks, model evaluation and interpretability. FMs demonstrate notable proficiency in managing large-scale, unlabeled datasets, because experimental procedures are costly and labor intensive. In various downstream tasks, FMs have consistently achieved noteworthy results, demonstrating high levels of accuracy in representing biological entities. A new era in computational biology has been ushered in by the application of FMs, focusing on both general and specific biological issues. In this review, we introduce recent advancements in bioinformatics FMs employed in a variety of downstream tasks, including genomics, transcriptomics, proteomics, drug discovery and single-cell analysis. Our aim is to assist scientists in selecting appropriate FMs in bioinformatics, according to four model types: language FMs, vision FMs, graph FMs and multimodal FMs. In addition to understanding molecular landscapes, AI technology can establish the theoretical and practical foundation for continued innovation in molecular biology.

Simple and effective embedding model for single-cell biology built from ChatGPT

Large-scale gene-expression data are being leveraged to pretrain models that implicitly learn gene and cellular functions. However, such models require extensive data curation and training. Here we explore a much simpler alternative: leveraging ChatGPT embeddings of genes based on the literature. We used GPT-3.5 to generate gene embeddings from text descriptions of individual genes and to then generate single-cell embeddings by averaging the gene embeddings weighted by each gene's expression level. We also created a sentence embedding for each cell by using only the gene names ordered by their expression level. On many downstream tasks used to evaluate pretrained single-cell embedding models-particularly, tasks of gene-property and cell-type classifications-our model, which we named GenePT, achieved comparable or better performance than models pretrained from gene-expression profiles of millions of cells. GenePT shows that large-language-model embeddings of the literature provide a simple and effective path to encoding single-cell biological knowledge.

scTrans: Sparse attention powers fast and accurate cell type annotation in single-cell RNA-seq data

Cell type annotation is crucial in single-cell RNA sequencing data analysis because it enables significant biological discoveries and deepens our understanding of tissue biology. Given the high-dimensional and highly sparse nature of single-cell RNA sequencing data, most existing annotation tools focus on highly variable genes to reduce dimensionality and computational load. However, this approach inevitably results in information loss, potentially weakening the model's generalization performance and adaptability to novel datasets. To mitigate this issue, we developed scTrans, a single cell Transformer-based model, which employs sparse attention to utilize all non-zero genes, thereby effectively reducing the input data dimensionality while minimizing information loss. We validated the speed and accuracy of scTrans by performing cell type annotation on 31 different tissues within the Mouse Cell Atlas. Remarkably, even with datasets nearing a million cells, scTrans efficiently perform cell type annotation in limited computational resources. Furthermore, scTrans demonstrates strong generalization capabilities, accurately annotating cells in novel datasets and generating high-quality latent representations, which are essential for precise clustering and trajectory analysis.

High resolution tissue and cell type identification via single cell transcriptomic profiling

Tissue identification can be instrumental in reconstructing a crime scene but remains a challenging task in forensic investigations. Conventionally, identifying the presence of certain tissue from tissue mixture by predefined cell type markers in bulk fashion is challenging due to limitations in sensitivity and accuracy. In contrast, single-cell RNA sequencing (scRNA-Seq) is a promising technology that has the potential to enhance or even revolutionize tissue and cell type identification. In this study, we developed a high sensitive general purpose single cell annotation pipeline, scTissueID, to accurately evaluate the single cell profile quality and precisely determine the cell and tissue types based on scRNA profiles. By incorporating a crucial and unique reference cell quality differentiation phase of targeting only high confident cells as reference, scTissueID achieved better and consistent performance in determining cell and tissue types compared to 8 state-of-art single cell annotation pipelines and 6 widely adopted machine learning algorithms, as demonstrated through a large-scale and comprehensive comparison study using both forensic-relevant and Human Cell Atlas (HCA) data. We highlighted the significance of cell quality differentiation, a previously undervalued factor. Thus, this study offers a tool capable of accurately and efficiently identifying cell and tissue types, with broad applicability to forensic investigations and other biomedical research endeavors.

TransGeneSelector: using a transformer approach to mine key genes from small transcriptomic datasets in plant responses to various environments

Gene mining is crucial for understanding the regulatory mechanisms underlying complex biological processes, particularly in plants responding to environmental conditions. Traditional machine learning methods, while useful, often overlook important gene relationships due to their reliance on manual feature selection and limited ability to capture complex inter-gene regulatory dynamics. Deep learning approaches, while powerful, are often unsuitable for small sample sizes. This study introduces TransGeneSelector, the first deep learning framework specifically designed for mining key genes from small transcriptomic datasets. By integrating a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for sample generation and a Transformer-based network for classification, TransGeneSelector efficiently addresses the challenges of small-sample transcriptomic data, capturing both global gene regulatory interactions and specific biological processes. Evaluated in Arabidopsis thaliana, the model achieved high classification accuracy in predicting seed germination and heat stress conditions, outperforming traditional methods like Random Forest and Support Vector Machines (SVM). Moreover, Shapley Additive Explanations (SHAP) analysis and gene regulatory network construction revealed that TransGeneSelector effectively identified genes that appear to have upstream regulatory functions based on our analyses, enriching them in multiple key pathways which are critical for seed germination and heat stress response. RT-qPCR validation further confirmed the model's gene selection accuracy, demonstrating consistent expression patterns across varying germination conditions. The findings underscore the potential of TransGeneSelector as a robust tool for gene mining, offering deeper insights into gene regulation and organism adaptation under diverse environmental conditions. This work provides a framework that leverages deep learning for key gene identification in small transcriptomic datasets.

SENSITIVITY BASED MODEL AGNOSTIC SCALABLE EXPLANATIONS OF DEEP LEARNING

Deep neural networks (DNNs) are powerful tools for data-driven predictive machine learning, but their complex architecture obscures mechanistic relations that they have learned from data. This information is critical to the scientific method of hypotheses development, experiment design, and model validation, especially when DNNs are used for biological and clinical predictions that affect human health. We design SensX, a model agnostic explainable AI (XAI) framework that outperformed current state-of-the-art XAI in accuracy (up to 52% higher) and computation time (up to 158 times faster), with higher consistency in all cases. It also determines an optimal subset of important input features, reducing dimensionality of further analyses. SensX scaled to explain vision transformer (ViT) models with more than 150,000 features, which is computationally infeasible for current state-of-the-art XAI. SensX validated that ViT models learned justifiable features as important for different facial attributes of different human faces. SensX revealed biases inherent to the ViT architecture, an observation possible only when importance of each feature is explained. We trained DNNs to annotate biological cell types using single-cell RNA-seq data and SensX determined the sets of genes that the DNNs learned to be important to different cell types.

WCSGNet: a graph neural network approach using weighted cell-specific networks for cell-type annotation in scRNA-seq

Single-cell RNA sequencing (scRNA-seq) has emerged as a powerful tool for understanding cellular heterogeneity, providing unprecedented resolution in molecular regulation analysis. Existing supervised learning approaches for cell type annotation primarily utilize gene expression profiles from scRNA-seq data. Although some methods incorporated gene interaction network information, they fail to use cell-specific gene association networks. This limitation overlooks the unique gene interaction patterns within individual cells, potentially compromising the accuracy of cell type classification. We introduce WCSGNet, a graph neural network-based algorithm for automatic cell-type annotation that leverages Weighted Cell-Specific Networks (WCSNs). These networks are constructed based on highly variable genes and inherently capture both gene expression patterns and gene association network structure features. Extensive experimental validation demonstrates that WCSGNet consistently achieves superior cell type classification performance, ranking among the top-performing methods while maintaining robust stability across diverse datasets. Notably, WCSGNet exhibits a distinct advantage in handling imbalanced datasets, outperforming existing methods in these challenging scenarios. All datasets and codes for reproducing this work were deposited in a GitHub repository (https://github.com/Yi-ellen/WCSGNet).

scCobra allows contrastive cell embedding learning with domain adaptation for single cell data integration and harmonization

The rapid advancement of single-cell technologies has created an urgent need for effective methods to integrate and harmonize single-cell data. Technical and biological variations across studies complicate data integration, while conventional tools often struggle with reliance on gene expression distribution assumptions and over-correction. Here, we present scCobra, a deep generative neural network designed to overcome these challenges through contrastive learning with domain adaptation. scCobra effectively mitigates batch effects, minimizes over-correction, and ensures biologically meaningful data integration without assuming specific gene expression distributions. It enables online label transfer across datasets with batch effects, allowing continuous integration of new data without retraining. Additionally, scCobra supports batch effect simulation, advanced multi-omic integration, and scalable processing of large datasets. By integrating and harmonizing datasets from similar studies, scCobra expands the available data for investigating specific biological problems, improving cross-study comparability, and revealing insights that may be obscured in isolated datasets.

Computing hematopoiesis plasticity in response to genetic mutations and environmental stimulations

Cell plasticity (CP), describing a dynamic cell state, plays a crucial role in maintaining homeostasis during organ morphogenesis, regeneration, and trauma-to-repair biological process. Single-cell-omics datasets provide an unprecedented resource to empower CP analysis. Hematopoiesis offers fertile opportunities to develop quantitative methods for understanding CP. In this study, we generated high-quality lineage-negative single-cell RNA-sequencing datasets under various conditions and introduced a working pipeline named scPlasticity to interrogate naïve and disturbed plasticity of hematopoietic stem and progenitor cells with mutational or environmental challenges. Using embedding methods UMAP or FA, a continuum of hematopoietic development is visually observed in wild type where the pipeline confirms a low proportion of hybrid cells ( P hc , with bias range: 0.4∼0.6) on a transition trajectory. Upon Tet2 mutation, a driver of leukemia, or treatment of DSS, an inducer of colitis, P hc is increased and plasticity of hematopoietic stem and progenitor cells was enhanced. We prioritized several transcription factors and signaling pathways, which are responsible for P hc alterations. In silico perturbation suggests knocking out EGR regulons or pathways of IL-1R1 and β-adrenoreceptor partially reverses P hc promoted by Tet2 mutation and inflammation.

A cell atlas foundation model for scalable search of similar human cells

Single-cell RNA sequencing has profiled hundreds of millions of human cells across organs, diseases, development and perturbations to date. Mining these growing atlases could reveal cell-disease associations, identify cell states in unexpected tissue contexts and relate in vivo biology to in vitro models. These require a common measure of cell similarity across the body and an efficient way to search. Here we develop SCimilarity, a metric-learning framework to learn a unified and interpretable representation that enables rapid queries of tens of millions of cell profiles from diverse studies for cells that are transcriptionally similar to an input cell profile or state. We use SCimilarity to query a 23.4-million-cell atlas of 412 single-cell RNA-sequencing studies for macrophage and fibroblast profiles from interstitial lung disease1 and reveal similar cell profiles across other fibrotic diseases and tissues. The top scoring in vitro hit for the macrophage query was a 3D hydrogel system2, which we experimentally demonstrated reproduces this cell state. SCimilarity serves as a foundation model for single-cell profiles that enables researchers to query for similar cellular states across the human body, providing a powerful tool for generating biological insights from the Human Cell Atlas.

Advancing bioinformatics with large language models: components, applications and perspectives

Large language models (LLMs) are a class of artificial intelligence models based on deep learning, which have great performance in various tasks, especially in natural language processing (NLP). Large language models typically consist of artificial neural networks with numerous parameters, trained on large amounts of unlabeled input using self-supervised or semi-supervised learning. However, their potential for solving bioinformatics problems may even exceed their proficiency in modeling human language. In this review, we will provide a comprehensive overview of the essential components of large language models (LLMs) in bioinformatics, spanning genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis. Key aspects covered include tokenization methods for diverse data types, the architecture of transformer models, the core attention mechanism, and the pre-training processes underlying these models. Additionally, we will introduce currently available foundation models and highlight their downstream applications across various bioinformatics domains. Finally, drawing from our experience, we will offer practical guidance for both LLM users and developers, emphasizing strategies to optimize their use and foster further innovation in the field.

uniLIVER: a human liver cell atlas for data-driven cellular state mapping

The liver performs several vital functions such as metabolism, toxin removal, and glucose storage through the coordination of various cell types. With the recent breakthrough of the single-cell/single-nucleus RNA-seq (sc/snRNA-seq) techniques, there is a great opportunity to establish a reference cell map of the liver at single-cell resolution with transcriptome-wise features. In this study, we build a unified liver cell atlas uniLIVER (http://lifeome.net/database/uniliver) by integrative analysis of a large-scale sc/snRNA-seq data collection of normal human liver with 331,125 cells and 79 samples from 6 datasets. Moreover, we introduce LiverCT, a novel machine learning based method for mapping any query dataset to the liver reference map by introducing the definition of "variant" cellular states analogy to the sequence variants in genomic analysis. Applying LiverCT on liver cancer datasets, we find that the "deviated" states of T cells are highly correlated with the stress pathway activities in hepatocellular carcinoma, and the enrichments of tumor cells with the hepatocyte-cholangiocyte "intermediate" states significantly indicate poor prognosis. Besides, we find that the tumor cells of different patients have different zonation tendencies and this zonation tendency is also significantly associated with the prognosis. This reference atlas mapping framework can also be extended to any other tissues.

CSI-GEP: A GPU-based unsupervised machine learning approach for recovering gene expression programs in atlas-scale single-cell RNA-seq data

Exploratory analysis of single-cell RNA sequencing (scRNA-seq) typically relies on hard clustering over two-dimensional projections like uniform manifold approximation and projection (UMAP). However, such methods can severely distort the data and have many arbitrary parameter choices. Methods that can model scRNA-seq data as non-discrete "gene expression programs" (GEPs) can better preserve the data's structure, but currently, they are often not scalable, not consistent across repeated runs, and lack an established method for choosing key parameters. Here, we developed a GPU-based unsupervised learning approach, "consensus and scalable inference of gene expression programs" (CSI-GEP). We show that CSI-GEP can recover ground truth GEPs in real and simulated atlas-scale scRNA-seq datasets, significantly outperforming cutting-edge methods, including GPT-based neural networks. We applied CSI-GEP to a whole mouse brain atlas of 2.2 million cells, disentangling endothelial cell types missed by other methods, and to an integrated scRNA-seq atlas of human tumors and cell lines, discovering mesenchymal-like GEPs unique to cancer cells growing in culture.

Analyzing scRNA-seq data by CCP-assisted UMAP and tSNE

Single-cell RNA sequencing (scRNA-seq) is widely used to reveal heterogeneity in cells, which has given us insights into cell-cell communication, cell differentiation, and differential gene expression. However, analyzing scRNA-seq data is a challenge due to sparsity and the large number of genes involved. Therefore, dimensionality reduction and feature selection are important for removing spurious signals and enhancing downstream analysis. Correlated clustering and projection (CCP) was recently introduced as an effective method for preprocessing scRNA-seq data. CCP utilizes gene-gene correlations to partition the genes and, based on the partition, employs cell-cell interactions to obtain super-genes. Because CCP is a data-domain approach that does not require matrix diagonalization, it can be used in many downstream machine learning tasks. In this work, we utilize CCP as an initialization tool for uniform manifold approximation and projection (UMAP) and t-distributed stochastic neighbor embedding (tSNE). By using 21 publicly available datasets, we have found that CCP significantly improves UMAP and tSNE visualization and dramatically improve their accuracy. More specifically, CCP improves UMAP by 22% in ARI, 14% in NMI and 15% in ECM, and improves tSNE by 11% in ARI, 9% in NMI and 8% in ECM.

Evaluating the Utilities of Foundation Models in Single-cell Data Analysis

Foundation Models (FMs) have made significant strides in both industrial and scientific domains. In this paper, we evaluate the performance of FMs for single-cell sequencing data analysis through comprehensive experiments across eight downstream tasks pertinent to single-cell data. Overall, the top FMs include scGPT, Geneformer, and CellPLM by considering model performances and user accessibility among ten single-cell FMs. However, by comparing these FMs with task-specific methods, we found that single-cell FMs may not consistently excel than task-specific methods in all tasks, which challenges the necessity of developing foundation models for single-cell analysis. In addition, we evaluated the effects of hyper-parameters, initial settings, and stability for training single-cell FMs based on a proposed scEval framework, and provide guidelines for pre-training and fine-tuning, to enhance the performances of single-cell FMs. Our work summarizes the current state of single-cell FMs, points to their constraints and avenues for future development, and offers a freely available evaluation pipeline to benchmark new models and improve method development.

Multiomics Research: Principles and Challenges in Integrated Analysis

Multiomics research is a transformative approach in the biological sciences that integrates data from genomics, transcriptomics, proteomics, metabolomics, and other omics technologies to provide a comprehensive understanding of biological systems. This review elucidates the fundamental principles of multiomics, emphasizing the necessity of data integration to uncover the complex interactions and regulatory mechanisms underlying various biological processes. We explore the latest advances in computational methodologies, including deep learning, graph neural networks (GNNs), and generative adversarial networks (GANs), which facilitate the effective synthesis and interpretation of multiomics data. Additionally, this review addresses the critical challenges in this field, such as data heterogeneity, scalability, and the need for robust, interpretable models. We highlight the potential of large language models to enhance multiomics analysis through automated feature extraction, natural language generation, and knowledge integration. Despite the important promise of multiomics, the review acknowledges the substantial computational resources required and the complexity of model tuning, underscoring the need for ongoing innovation and collaboration in the field. This comprehensive analysis aims to guide researchers in navigating the principles and challenges of multiomics research to foster advances in integrative biological analysis.

A multi-task deep learning model based on comprehensive feature integration and self-attention mechanism for predicting response to anti-PD1/PD-L1

BACKGROUND

Immune checkpoint inhibitor (ICI) has been widely used in the treatment of advanced cancers, but predicting their efficacy remains challenging. Traditional biomarkers are numerous but exhibit heterogeneity within populations. For comprehensively utilizing the ICI-related biomarkers, we aim to conduct multidimensional feature selection and deep learning model construction.

METHODS

We used statistical and machine learning methods to map features of different levels to next-generation sequencing gene expression. We integrated genes from different sources into the feature input of a deep learning model, by means of self-attention mechanism.

RESULTS

We performed feature selection at the single-cell sequencing level, PD-L1 (CD274) analysis level, tumor mutational burden (TMB)/mismatch repair (MMR) level, and somatic copy number alteration (SCNA) level, obtaining 96 feature genes. Based on the pan-cancer dataset, we trained a multi-task deep learning model. We tested the model in the bladder urothelial carcinoma testing set 1 (AUC = 0.62, n = 298), bladder urothelial carcinoma testing set 2 (AUC = 0.66, n = 89), non-small cell lung cancer testing set (AUC = 0.85, n = 27), and skin cutaneous melanoma testing set (AUC = 0.71, n = 27).

CONCLUSION

Our study demonstrates the potential of the deep learning model for integrating multidimensional features in predicting the outcome of ICI. Our study also provides a potential methodological case for medical scenarios requiring the integration of multiple levels of features.

GeneCompass: deciphering universal gene regulatory mechanisms with a knowledge-informed cross-species foundation model

Deciphering universal gene regulatory mechanisms in diverse organisms holds great potential for advancing our knowledge of fundamental life processes and facilitating clinical applications. However, the traditional research paradigm primarily focuses on individual model organisms and does not integrate various cell types across species. Recent breakthroughs in single-cell sequencing and deep learning techniques present an unprecedented opportunity to address this challenge. In this study, we built an extensive dataset of over 120 million human and mouse single-cell transcriptomes. After data preprocessing, we obtained 101,768,420 single-cell transcriptomes and developed a knowledge-informed cross-species foundation model, named GeneCompass. During pre-training, GeneCompass effectively integrated four types of prior biological knowledge to enhance our understanding of gene regulatory mechanisms in a self-supervised manner. By fine-tuning for multiple downstream tasks, GeneCompass outperformed state-of-the-art models in diverse applications for a single species and unlocked new realms of cross-species biological investigations. We also employed GeneCompass to search for key factors associated with cell fate transition and showed that the predicted candidate genes could successfully induce the differentiation of human embryonic stem cells into the gonadal fate. Overall, GeneCompass demonstrates the advantages of using artificial intelligence technology to decipher universal gene regulatory mechanisms and shows tremendous potential for accelerating the discovery of critical cell fate regulators and candidate drug targets.

VICTOR: Validation and inspection of cell type annotation through optimal regression

Single-cell RNA sequencing provides unprecedent opportunities to explore the heterogeneity and dynamics inherent in cellular biology. An essential step in the data analysis involves the automatic annotation of cells. Despite development of numerous tools for automated cell annotation, assessing the reliability of predicted annotations remains challenging, particularly for rare and unknown cell types. Here, we introduce VICTOR: Validation and inspection of cell type annotation through optimal regression. VICTOR aims to gauge the confidence of cell annotations by an elastic-net regularized regression with optimal thresholds. We demonstrated that VICTOR performed well in identifying inaccurate annotations, surpassing existing methods in diagnostic ability across various single-cell datasets, including within-platform, cross-platform, cross-studies, and cross-omics settings.

scMoMtF: An interpretable multitask learning framework for single-cell multi-omics data analysis

With the rapidly development of biotechnology, it is now possible to obtain single-cell multi-omics data in the same cell. However, how to integrate and analyze these single-cell multi-omics data remains a great challenge. Herein, we introduce an interpretable multitask framework (scMoMtF) for comprehensively analyzing single-cell multi-omics data. The scMoMtF can simultaneously solve multiple key tasks of single-cell multi-omics data including dimension reduction, cell classification and data simulation. The experimental results shows that scMoMtF outperforms current state-of-the-art algorithms on these tasks. In addition, scMoMtF has interpretability which allowing researchers to gain a reliable understanding of potential biological features and mechanisms in single-cell multi-omics data.

Advancing plant single-cell genomics with foundation models

Single-cell genomics, combined with advanced AI models, hold transformative potential for understanding complex biological processes in plants. This article reviews deep-learning approaches in single-cell genomics, focusing on foundation models, a type of large-scale, pretrained, multi-purpose generative AI models. We explore how these models, such as Generative Pre-trained Transformers (GPT), Bidirectional Encoder Representations from Transformers (BERT), and other Transformer-based architectures, are applied to extract meaningful biological insights from diverse single-cell datasets. These models address challenges in plant single-cell genomics, including improved cell-type annotation, gene network modeling, and multi-omics integration. Moreover, we assess the use of Generative Adversarial Networks (GANs) and diffusion models, focusing on their capacity to generate high-fidelity synthetic single-cell data, mitigate dropout events, and handle data sparsity and imbalance. Together, these AI-driven approaches hold immense potential to enhance research in plant genomics, facilitating discoveries in crop resilience, productivity, and stress adaptation.

scGO: interpretable deep neural network for cell status annotation and disease diagnosis

Machine learning has emerged as a transformative tool for elucidating cellular heterogeneity in single-cell RNA sequencing. However, a significant challenge lies in the "black box" nature of deep learning models, which obscures the decision-making process and limits interpretability in cell status annotation. In this study, we introduced scGO, a Gene Ontology (GO)-inspired deep learning framework designed to provide interpretable cell status annotation for scRNA-seq data. scGO employs sparse neural networks to leverage the intrinsic biological relationships among genes, transcription factors, and GO terms, significantly augmenting interpretability and reducing computational cost. scGO outperforms state-of-the-art methods in the precise characterization of cell subtypes across diverse datasets. Our extensive experimentation across a spectrum of scRNA-seq datasets underscored the remarkable efficacy of scGO in disease diagnosis, prediction of developmental stages, and evaluation of disease severity and cellular senescence status. Furthermore, we incorporated in silico individual gene manipulations into the scGO model, introducing an additional layer for discovering therapeutic targets. Our results provide an interpretable model for accurately annotating cell status, capturing latent biological knowledge, and informing clinical practice.

Graph contrastive learning of subcellular-resolution spatial transcriptomics improves cell type annotation and reveals critical molecular pathways

Imaging-based spatial transcriptomics (iST), such as MERFISH, CosMx SMI, and Xenium, quantify gene expression level across cells in space, but more importantly, they directly reveal the subcellular distribution of RNA transcripts at the single-molecule resolution. The subcellular localization of RNA molecules plays a crucial role in the compartmentalization-dependent regulation of genes within individual cells. Understanding the intracellular spatial distribution of RNA for a particular cell type thus not only improves the characterization of cell identity but also is of paramount importance in elucidating unique subcellular regulatory mechanisms specific to the cell type. However, current cell type annotation approaches of iST primarily utilize gene expression information while neglecting the spatial distribution of RNAs within cells. In this work, we introduce a semi-supervised graph contrastive learning method called Focus, the first method, to the best of our knowledge, that explicitly models RNA's subcellular distribution and community to improve cell type annotation. Focus demonstrates significant improvements over state-of-the-art algorithms across a range of spatial transcriptomics platforms, achieving improvements up to 27.8% in terms of accuracy and 51.9% in terms of F1-score for cell type annotation. Furthermore, Focus enjoys the advantages of intricate cell type-specific subcellular spatial gene patterns and providing interpretable subcellular gene analysis, such as defining the gene importance score. Importantly, with the importance score, Focus identifies genes harboring strong relevance to cell type-specific pathways, indicating its potential in uncovering novel regulatory programs across numerous biological systems.

scRGCL: a cell type annotation method for single-cell RNA-seq data using residual graph convolutional neural network with contrastive learning

Cell type annotation is a critical step in analyzing single-cell RNA sequencing (scRNA-seq) data. A large number of deep learning (DL)-based methods have been proposed to annotate cell types of scRNA-seq data and have achieved impressive results. However, there are several limitations to these methods. First, they do not fully exploit cell-to-cell differential features. Second, they are developed based on shallow features and lack of flexibility in integrating high-order features in the data. Finally, the low-dimensional gene features may lead to overfitting in neural networks. To overcome those limitations, we propose a novel DL-based model, cell type annotation of single-cell RNA-seq data using residual graph convolutional neural network with contrastive learning (scRGCL), based on residual graph convolutional neural network and contrastive learning for cell type annotation of single-cell RNA-seq data. scRGCL mainly consists of a residual graph convolutional neural network, contrastive learning, and weight freezing. A residual graph convolutional neural network is utilized to extract complex high-order features from data. Contrastive learning can help the model learn meaningful cell-to-cell differential features. Weight freezing can avoid overfitting and help the model discover the impact of specific gene expression on cell type annotation. To verify the effectiveness of scRGCL, we compared its performance with six methods (three shallow learning algorithms and three state-of-the-art DL-based methods) on eight single-cell benchmark datasets from two species (seven in human and one in mouse). Experimental results not only show that scRGCL outperforms competing methods but also demonstrate the generalizability of scRGCL for cell type annotation. scRGCL is available at https://github.com/nathanyl/scRGCL.

Cytometry masked autoencoder: An accurate and interpretable automated immunophenotyper

Single-cell cytometry data are crucial for understanding the role of the immune system in diseases and responses to treatment. However, traditional methods for annotating cytometry data face challenges in scalability, robustness, and accuracy. We propose a cytometry masked autoencoder (cyMAE), which automates immunophenotyping tasks including cell type annotation. The model upholds user-defined cell type definitions, facilitating interpretability and cross-study comparisons. The training of cyMAE has a self-supervised phase, which leverages large amounts of unlabeled data, followed by fine-tuning on specialized tasks using smaller amounts of annotated data. The cost of training a new model is amortized over repeated inferences on new datasets using the same panel. Through validation across multiple studies using the same panel, we demonstrate that cyMAE delivers accurate and interpretable cellular immunophenotyping and improves the prediction of subject-level metadata. This proof of concept marks a significant step forward for large-scale immunology studies.

DeSide: A unified deep learning approach for cellular deconvolution of tumor microenvironment

Cellular deconvolution via bulk RNA sequencing (RNA-seq) presents a cost-effective and efficient alternative to experimental methods such as flow cytometry and single-cell RNA-seq (scRNA-seq) for analyzing the complex cellular composition of tumor microenvironments. Despite challenges due to heterogeneity within and among tumors, our innovative deep learning-based approach, DeSide, shows exceptional accuracy in estimating the proportions of 16 distinct cell types and subtypes within solid tumors. DeSide integrates biological pathways and assesses noncancerous cell types first, effectively sidestepping the issue of highly variable gene expression profiles (GEPs) associated with cancer cells. By leveraging scRNA-seq data from six cancer types and 185 cancer cell lines across 22 cancer types as references, our method introduces distinctive sampling and filtering techniques to generate a high-quality training set that closely replicates real tumor GEPs, based on The Cancer Genome Atlas (TCGA) bulk RNA-seq data. With this model and high-quality training set, DeSide outperforms existing methods in estimating tumor purity and the proportions of noncancerous cells within solid tumors. Our model precisely predicts cellular compositions across 19 cancer types from TCGA and proves its effectiveness with multiple additional external datasets. Crucially, DeSide enables the identification and analysis of combinatorial cell type pairs, facilitating the stratification of cancer patients into prognostically significant groups. This approach not only provides deeper insights into the dynamics of tumor biology but also highlights potential therapeutic targets by underscoring the importance of specific cell type or subtype interactions.

scGraphformer: unveiling cellular heterogeneity and interactions in scRNA-seq data using a scalable graph transformer network

The precise classification of cell types from single-cell RNA sequencing (scRNA-seq) data is pivotal for dissecting cellular heterogeneity in biological research. Traditional graph neural network (GNN) models are constrained by reliance on predefined graphs, limiting the exploration of complex cell-to-cell relationships. We introduce scGraphformer, a transformer-based GNN that transcends these limitations by learning an all-encompassing cell-cell relational network directly from scRNA-seq data. Through an iterative refinement process, scGraphformer constructs a dense graph structure that captures the full spectrum of cellular interactions. This comprehensive approach enables the identification of subtle and previously obscured cellular patterns and relationships. Evaluated on multiple datasets, scGraphformer demonstrates superior performance in cell type identification compared to existing methods and showcases its scalability with large-scale datasets. Our method not only provides enhanced cell type classification ability but also reveals the underlying cell interactions, offering deeper insights into functional cellular relationships. The scGraphformer thus holds the potential to significantly advance the field of single-cell analysis and contribute to a more nuanced understanding of cellular behavior.

Knowledge-based inductive bias and domain adaptation for cell type annotation

Measurement techniques often result in domain gaps among batches of cellular data from a specific modality. The effectiveness of cross-batch annotation methods is influenced by inductive bias, which refers to a set of assumptions that describe the behavior of model predictions. Different annotation methods possess distinct inductive biases, leading to varying degrees of generalizability and interpretability. Given that certain cell types exhibit unique functional patterns, we hypothesize that the inductive biases of cell annotation methods should align with these biological patterns to produce meaningful predictions. In this study, we propose KIDA, Knowledge-based Inductive bias and Domain Adaptation. The knowledge-based inductive bias constrains the prediction rules learned from the reference dataset, composed of multiple batches, to functional patterns relevant to biology, thereby enhancing the generalization of the model to unseen batches. Since the query dataset also contains gaps from multiple batches, KIDA's domain adaptation employs pseudo labels for self-knowledge distillation, effectively narrowing the distribution gap between model predictions and the query dataset. Benchmark experiments demonstrate that KIDA is capable of achieving accurate cross-batch cell type annotation.

Transfer learning enabled transformer-based generative adversarial networks for modeling and generating terahertz channels

Terahertz communications are envisioned as a promising technology for the sixth generation and beyond wireless systems, which can support wireless links with Terabits-per-second (Tbps) data rates. As the foundation of designing terahertz communications, channel modeling and characterization are crucial to scrutinize the potential of this spectrum. However, current channel modeling in the terahertz band heavily relies on time-consuming and costly measurements. Here, we propose a transfer learning enabled transformer based generative adversarial network to mitigate this problem in terahertz channel modeling. Specifically, as a fundamental building block, a generative adversarial network is exploited to generate channel parameters. To improve the accuracy, a transformer structure with a self-attention mechanism is incorporated in generative adversarial network. Still incurring errors compared with ground-truth measurement, a transfer learning is designed to solve the mismatch between the formulated network and measurement. The proposed method can achieve high accuracy in channel modeling, while requiring only rather limited amount of measurement, which is a promising complement of current channel modeling techniques.

A self-training interpretable cell type annotation framework using specific marker gene

MOTIVATION

Recent advances in sequencing technology provide opportunities to study biological processes at a higher resolution. Cell type annotation is an important step in scRNA-seq analysis, which often relies on established marker genes. However, most of the previous methods divide the identification of cell types into two stages, clustering and assignment, whose performances are susceptible to the clustering algorithm, and the marker information cannot effectively guide the clustering process. Furthermore, their linear heuristic-based cell assignment process is often insufficient to capture potential dependencies between cells and types.

RESULTS

Here, we present Interpretable Cell Type Annotation based on self-training (sICTA), a marker-based cell type annotation method that combines the self-training strategy with pseudo-labeling and the nonlinear association capturing capability of Transformer. In addition, we incorporate biological priori knowledge of genes and pathways into the classifier through an attention mechanism to enhance the transparency of the model. A benchmark analysis on 11 publicly available single-cell datasets demonstrates the superiority of sICTA compared to state-of-the-art methods. The robustness of our method is further validated by evaluating the prediction accuracy of the model on different cell types for each single-cell data. Moreover, ablation studies show that self-training and the ability to capture potential dependencies between cells and cell types, both of which are mutually reinforcing, work together to improve model performance. Finally, we apply sICTA to the pancreatic dataset, exemplifying the interpretable attention matrix captured by sICTA.

AVAILABILITY AND IMPLEMENTATION

The source code of sICTA is available in public at https://github.com/nbnbhwyy/sICTA. The processed datasets can be found at https://drive.google.com/drive/folders/1jbqSxacL_IDIZ4uPjq220C9Kv024m9eL. The final version of the model will be permanently available at https://doi.org/10.5281/zenodo.13474010.

scGAA: a general gated axial-attention model for accurate cell-type annotation of single-cell RNA-seq data

Single-cell RNA sequencing (scRNA-seq) is a key technology for investigating cell development and analysing cell diversity across various diseases. However, the high dimensionality and extreme sparsity of scRNA-seq data pose great challenges for accurate cell type annotation. To address this, we developed a new cell-type annotation model called scGAA (general gated axial-attention model for accurate cell-type annotation of scRNA-seq). Based on the transformer framework, the model decomposes the traditional self-attention mechanism into horizontal and vertical attention, considerably improving computational efficiency. This axial attention mechanism can process high-dimensional data more efficiently while maintaining reasonable model complexity. Additionally, the gated unit was integrated into the model to enhance the capture of relationships between genes, which is crucial for achieving an accurate cell type annotation. The results revealed that our improved transformer model is a promising tool for practical applications. This theoretical innovation increased the model performance and provided new insights into analytical tools for scRNA-seq data.

PathMethy: an interpretable AI framework for cancer origin tracing based on DNA methylation

Despite advanced diagnostics, 3%-5% of cases remain classified as cancer of unknown primary (CUP). DNA methylation, an important epigenetic feature, is essential for determining the origin of metastatic tumors. We presented PathMethy, a novel Transformer model integrated with functional categories and crosstalk of pathways, to accurately trace the origin of tumors in CUP samples based on DNA methylation. PathMethy outperformed seven competing methods in F1-score across nine cancer datasets and predicted accurately the molecular subtypes within nine primary tumor types. It not only excelled at tracing the origins of both primary and metastatic tumors but also demonstrated a high degree of agreement with previously diagnosed sites in cases of CUP. PathMethy provided biological insights by highlighting key pathways, functional categories, and their interactions. Using functional categories of pathways, we gained a global understanding of biological processes. For broader access, a user-friendly web server for researchers and clinicians is available at https://cup.pathmethy.com.

GexMolGen: cross-modal generation of hit-like molecules via large language model encoding of gene expression signatures

Designing de novo molecules with specific biological activity is an essential task since it holds the potential to bypass the exploration of target genes, which is an initial step in the modern drug discovery paradigm. However, traditional methods mainly screen molecules by comparing the desired molecular effects within the documented experimental results. The data set limits this process, and it is hard to conduct direct cross-modal comparisons. Therefore, we propose a solution based on cross-modal generation called GexMolGen (Gene Expression-based Molecule Generator), which generates hit-like molecules using gene expression signatures alone. These signatures are calculated by inputting control and desired gene expression states. Our model GexMolGen adopts a "first-align-then-generate" strategy, aligning the gene expression signatures and molecules within a mapping space, ensuring a smooth cross-modal transition. The transformed molecular embeddings are then decoded into molecular graphs. In addition, we employ an advanced single-cell large language model for input flexibility and pre-train a scaffold-based molecular model to ensure that all generated molecules are 100% valid. Empirical results show that our model can produce molecules highly similar to known references, whether feeding in- or out-of-domain transcriptome data. Furthermore, it can also serve as a reliable tool for cross-modal screening.

Robust self-supervised learning strategy to tackle the inherent sparsity in single-cell RNA-seq data

Single-cell RNA sequencing (scRNA-seq) is a powerful tool for elucidating cellular heterogeneity and tissue function in various biological contexts. However, the sparsity in scRNA-seq data limits the accuracy of cell type annotation and transcriptomic analysis due to information loss. To address this limitation, we present scRobust, a robust self-supervised learning strategy to tackle the inherent sparsity of scRNA-seq data. Built upon the Transformer architecture, scRobust employs a novel self-supervised learning strategy comprising contrastive learning and gene expression prediction tasks. We demonstrated the effectiveness of scRobust using nine benchmarks, additional dropout scenarios, and combined datasets. scRobust outperformed recent methods in cell-type annotation tasks and generated cell embeddings that capture multi-faceted clustering information (e.g. cell types and HbA1c levels). In addition, cell embeddings of scRobust were useful for detecting specific marker genes related to drug tolerance stages. Furthermore, when we applied scRobust to scATAC-seq data, high-quality cell embedding vectors were generated. These results demonstrate the representational power of scRobust.

Integrative single-cell analysis of human colorectal cancer reveals patient stratification with distinct immune evasion mechanisms

The tumor microenvironment (TME) considerably influences colorectal cancer (CRC) progression, therapeutic response and clinical outcome, but studies of interindividual heterogeneities of the TME in CRC are lacking. Here, by integrating human colorectal single-cell transcriptomic data from approximately 200 donors, we comprehensively characterized transcriptional remodeling in the TME compared to noncancer tissues and identified a rare tumor-specific subset of endothelial cells with T cell recruitment potential. The large sample size enabled us to stratify patients based on their TME heterogeneity, revealing divergent TME subtypes in which cancer cells exploit different immune evasion mechanisms. Additionally, by associating single-cell transcriptional profiling with risk genes identified by genome-wide association studies, we determined that stromal cells are major effector cell types in CRC genetic susceptibility. In summary, our results provide valuable insights into CRC pathogenesis and might help with the development of personalized immune therapies.

scPriorGraph: constructing biosemantic cell-cell graphs with prior gene set selection for cell type identification from scRNA-seq data

Cell type identification is an indispensable analytical step in single-cell data analyses. To address the high noise stemming from gene expression data, existing computational methods often overlook the biologically meaningful relationships between genes, opting to reduce all genes to a unified data space. We assume that such relationships can aid in characterizing cell type features and improving cell type recognition accuracy. To this end, we introduce scPriorGraph, a dual-channel graph neural network that integrates multi-level gene biosemantics. Experimental results demonstrate that scPriorGraph effectively aggregates feature values of similar cells using high-quality graphs, achieving state-of-the-art performance in cell type identification.

Transformers in single-cell omics: a review and new perspectives

Recent efforts to construct reference maps of cellular phenotypes have expanded the volume and diversity of single-cell omics data, providing an unprecedented resource for studying cell properties. Despite the availability of rich datasets and their continued growth, current single-cell models are unable to fully capitalize on the information they contain. Transformers have become the architecture of choice for foundation models in other domains owing to their ability to generalize to heterogeneous, large-scale datasets. Thus, the question arises of whether transformers could set off a similar shift in the field of single-cell modeling. Here we first describe the transformer architecture and its single-cell adaptations and then present a comprehensive review of the existing applications of transformers in single-cell analysis and critically discuss their future potential for single-cell biology. By studying limitations and technical challenges, we aim to provide a structured outlook for future research directions at the intersection of machine learning and single-cell biology.

Assessing parameter efficient methods for pre-trained language model in annotating scRNA-seq data

Annotating cell types of single-cell RNA sequencing (scRNA-seq) data is crucial for studying cellular heterogeneity in the tumor microenvironment. Recently, large-scale pre-trained language models (PLMs) have achieved significant progress in cell-type annotation of scRNA-seq data. This approach effectively addresses previous methods' shortcomings in performance and generalization. However, fine-tuning PLMs for different downstream tasks demands considerable computational resources, rendering it impractical. Hence, a new research branch introduces parameter-efficient fine-tuning (PEFT). This involves optimizing a few parameters while leaving the majority unchanged, leading to substantial reductions in computational expenses. Here, we utilize scBERT, a large-scale pre-trained model, to explore the capabilities of three PEFT methods in scRNA-seq cell type annotation. Extensive benchmark studies across several datasets demonstrate the superior applicability of PEFT methods. Furthermore, downstream analysis using models obtained through PEFT showcases their utility in novel cell type discovery and model interpretability for potential marker genes. Our findings underscore the considerable potential of PEFT in PLM-based cell type annotation, presenting novel perspectives for the analysis of scRNA-seq data.

Large-scale foundation model on single-cell transcriptomics

Large pretrained models have become foundation models leading to breakthroughs in natural language processing and related fields. Developing foundation models for deciphering the 'languages' of cells and facilitating biomedical research is promising yet challenging. Here we developed a large pretrained model scFoundation, also named 'xTrimoscFoundationα', with 100 million parameters covering about 20,000 genes, pretrained on over 50 million human single-cell transcriptomic profiles. scFoundation is a large-scale model in terms of the size of trainable parameters, dimensionality of genes and volume of training data. Its asymmetric transformer-like architecture and pretraining task design empower effectively capturing complex context relations among genes in a variety of cell types and states. Experiments showed its merit as a foundation model that achieved state-of-the-art performances in a diverse array of single-cell analysis tasks such as gene expression enhancement, tissue drug response prediction, single-cell drug response classification, single-cell perturbation prediction, cell type annotation and gene module inference.

scGPT: toward building a foundation model for single-cell multi-omics using generative AI

Generative pretrained models have achieved remarkable success in various domains such as language and computer vision. Specifically, the combination of large-scale diverse datasets and pretrained transformers has emerged as a promising approach for developing foundation models. Drawing parallels between language and cellular biology (in which texts comprise words; similarly, cells are defined by genes), our study probes the applicability of foundation models to advance cellular biology and genetic research. Using burgeoning single-cell sequencing data, we have constructed a foundation model for single-cell biology, scGPT, based on a generative pretrained transformer across a repository of over 33 million cells. Our findings illustrate that scGPT effectively distills critical biological insights concerning genes and cells. Through further adaptation of transfer learning, scGPT can be optimized to achieve superior performance across diverse downstream applications. This includes tasks such as cell type annotation, multi-batch integration, multi-omic integration, perturbation response prediction and gene network inference.

Detecting novel cell type in single-cell chromatin accessibility data via open-set domain adaptation

Recent advances in single-cell technologies enable the rapid growth of multi-omics data. Cell type annotation is one common task in analyzing single-cell data. It is a challenge that some cell types in the testing set are not present in the training set (i.e. unknown cell types). Most scATAC-seq cell type annotation methods generally assign each cell in the testing set to one known type in the training set but neglect unknown cell types. Here, we present OVAAnno, an automatic cell types annotation method which utilizes open-set domain adaptation to detect unknown cell types in scATAC-seq data. Comprehensive experiments show that OVAAnno successfully identifies known and unknown cell types. Further experiments demonstrate that OVAAnno also performs well on scRNA-seq data. Our codes are available online at https://github.com/lisaber/OVAAnno/tree/master.