Meta-analysis of (single-cell method) benchmarks reveals the need for extensibility and interoperability

Challenges and best practices in omics benchmarking

Technological advances enabling massively parallel measurement of biological features - such as microarrays, high-throughput sequencing and mass spectrometry - have ushered in the omics era, now in its third decade. The resulting complex landscape of analytical methods has naturally fostered the growth of an omics benchmarking industry. Benchmarking refers to the process of objectively comparing and evaluating the performance of different computational or analytical techniques when processing and analysing large-scale biological data sets, such as transcriptomics, proteomics and metabolomics. With thousands of omics benchmarking studies published over the past 25 years, the field has matured to the point where the foundations of benchmarking have been established and well described. However, generating meaningful benchmarking data and properly evaluating performance in this complex domain remains challenging. In this Review, we highlight some common oversights and pitfalls in omics benchmarking. We also establish a methodology to bring the issues that can be addressed into focus and to be transparent about those that cannot: this takes the form of a spreadsheet template of guidelines for comprehensive reporting, intended to accompany publications. In addition, a survey of recent developments in benchmarking is provided as well as specific guidance for commonly encountered difficulties.

Defining and benchmarking open problems in single-cell analysis

With the growing number of single-cell analysis tools, benchmarks are increasingly important to guide analysis and method development. However, a lack of standardisation and extensibility in current benchmarks limits their usability, longevity, and relevance to the community. We present Open Problems, a living, extensible, community-guided benchmarking platform including 10 current single-cell tasks that we envision will raise standards for the selection, evaluation, and development of methods in single-cell analysis.

Tissue niches and immunopathology through the lens of spatial tissue profiling techniques

Spatial organization plays a fundamental role in biology, influencing the function of biological structures at various levels. The immune system, in particular, relies on the orchestrated interactions of immune cells with their microenvironment to mount protective or pathogenic immune responses. The COVID-19 pandemic has underscored the significance of studying immunity within target organs to understand disease progression and severity. To achieve this, multiplex histology and spatial transcriptomics have proven indispensable in providing a spatial context to protein and gene expression patterns. By combining these techniques, researchers gain a more comprehensive understanding of the complex interactions at the cellular and molecular level in distinct tissue niches, key functional units modulating health and disease. In this review, we discuss recent advances in spatial tissue profiling techniques, highlighting their advantages over traditional histopathology studies. The insights gained from these approaches have the potential to revolutionize the diagnosis and treatment of various diseases including cancer, autoimmune disorders, and infectious diseases. However, we also acknowledge their challenges and limitations. Despite these, spatial tissue profiling offers promising opportunities to improve our understanding of how tissue niches direct regional immunity, and their relevance in tissue immunopathology, as a basis for novel therapeutic strategies and personalized medicine.

The Quartet Data Portal: integration of community-wide resources for multiomics quality control

The Quartet Data Portal facilitates community access to well-characterized reference materials, reference datasets, and related resources established based on a family of four individuals with identical twins from the Quartet Project. Users can request DNA, RNA, protein, and metabolite reference materials, as well as datasets generated across omics, platforms, labs, protocols, and batches. Reproducible analysis tools allow for objective performance assessment of user-submitted data, while interactive visualization tools support rapid exploration of reference datasets. A closed-loop "distribution-collection-evaluation-integration" workflow enables updates and integration of community-contributed multiomics data. Ultimately, this portal helps promote the advancement of reference datasets and multiomics quality control.