Analysis validation has been neglected in the Age of Reproducibility

Turbulence-enhanced microneedle immunoassay platform (TMIP) for high-precision biomarker detection from skin interstitial fluid

Conventional diagnostic methods for biomarker detection often require invasive procedures and exhibit limited reproducibility and sensitivity. In this study, the turbulence-enhanced microneedle immunoassay platform (TMIP) was designed to enhance the performance and accuracy of biomarker detection in skin interstitial fluid (ISF). TMIP combines a bullet-shaped microneedle (MN) array for minimally invasive biomarker capture, a microfluidic device for MN-mediated immunoassay process simplification, and a star-shaped magnetic stirrer tool (MST) to facilitate efficient washing. By targeting S100 calcium-binding protein B (S100B), a diagnostic biomarker for melanoma, TMIP demonstrated substantial improvements in reproducibility, reducing signal deviations by up to 55 % compared to manual operation. The application of nanoporous MNs (NPMNs) achieved a low detection limit of 20 pg/mL with a high linearity (R2 = 0.9758). Validation using a gelatin phantom mimicking human skin confirmed TMIP's ability to achieve improved reproducibility and sensitivity. Furthermore, TMIP successfully detected S100B with high reproducibility in both the phantom (R2 = 0.97523) and melanoma-expressing mice within a rapid incubation time of 1 min. TMIP enables the detection of biomarkers with remarkable reproducibility and sub-nanogram sensitivity by simplifying the analysis process and enhancing reagent washing through turbulence. These features suggest that TMIP has the potential to serve as an efficient and reliable tool for biomarker detection in skin ISF.

Simulation Tests of Methods in Evolution, Ecology, and Systematics: Pitfalls, Progress, and Principles

Complex statistical methods are continuously developed across the fields of ecology, evolution, and systematics (EES). These fields, however, lack standardized principles for evaluating methods, which has led to high variability in the rigor with which methods are tested, a lack of clarity regarding their limitations, and the potential for misapplication. In this review, we illustrate the common pitfalls of method evaluations in EES, the advantages of testing methods with simulated data, and best practices for method evaluations. We highlight the difference between method evaluation and validation and review how simulations, when appropriately designed, can refine the domain in which a method can be reliably applied. We also discuss the strengths and limitations of different evaluation metrics. The potential for misapplication of methods would be greatly reduced if funding agencies, reviewers, and journals required principled method evaluation.

A Simulation Study of the Ecological Speciation Conditions in the Galician Marine Snail Littorina saxatilis

In the last years, the interest in evolutionary divergence at small spatial scales has increased and so did the study of speciation caused by ecologically based divergent natural selection. The evolutionary interplay between gene flow and local adaptation can lead to low-dispersal locally adapted specialists. When this occurs, the evolutionary interplay between gene flow and local adaptation could eventually lead to speciation. The L. saxatilis system consists of two ecotypes displaying a microhabitat-associated intraspecific dimorphism along the wave-exposed rocky shores of Galicia. Despite being a well-known system, the dynamics of the ecotype formation remain unclear and cannot be studied from empirical evidence alone. In this study, individual-based simulations were used to incorporate relevant ecological, spatial, and genetic information, to check different evolutionary scenarios that could evolve non-random mating preferences and finally may facilitate speciation. As main results, we observed the evolution of intermediate values of choice which matches the estimates from empirical data of L. saxatilis in Galician shores and coincides with previous theoretical outcomes. Also, the use of the mating correlation as a proxy for assortative mating led to spuriously inferring greater reproductive isolation in the middle habitat than in the others, which does not happen when directly considering the choice values from the simulations. We also corroborate the well-known fact that the occurrence of speciation is influenced by the strength of selection. Taken together, this means, also according to other L. saxatilis systems, that speciation is not an immediate consequence of local divergent selection and mating preferences, but a fine tuning among several factors including the ecological conditions in the shore levels, the selection strength, the mate choice stringency, and cost to choosiness. The L. saxatilis system could correspond to a case of incomplete reproductive isolation, where the choice intensity is intermediate and local adaptation within the habitat is strong. These results support previous interpretations of the L. saxatilis model system and indicate that further empirical studies would be interesting to test whether the mate choice mechanism functions as a similarity-like mechanism as has been shown in other littorinids.

Seeing the forest for the trees: Assessing genetic offset predictions from gradient forest

Gradient Forest (GF) is a machine learning algorithm designed to analyze spatial patterns of biodiversity as a function of environmental gradients. An offset measure between the GF-predicted environmental association of adapted alleles and a new environment (GF Offset) is increasingly being used to predict the loss of environmentally adapted alleles under rapid environmental change, but remains mostly untested for this purpose. Here, we explore the robustness of GF Offset to assumption violations, and its relationship to measures of fitness, using SLiM simulations with explicit genome architecture and a spatial metapopulation. We evaluate measures of GF Offset in: (1) a neutral model with no environmental adaptation; (2) a monogenic "population genetic" model with a single environmentally adapted locus; and (3) a polygenic "quantitative genetic" model with two adaptive traits, each adapting to a different environment. We found GF Offset to be broadly correlated with fitness offsets under both single locus and polygenic architectures. However, neutral demography, genomic architecture, and the nature of the adaptive environment can all confound relationships between GF Offset and fitness. GF Offset is a promising tool, but it is important to understand its limitations and underlying assumptions, especially when used in the context of predicting maladaptation.

Introduction to statistical simulations in health research

In health research, statistical methods are frequently used to address a wide variety of research questions. For almost every analytical challenge, different methods are available. But how do we choose between different methods and how do we judge whether the chosen method is appropriate for our specific study? Like in any science, in statistics, experiments can be run to find out which methods should be used under which circumstances. The main objective of this paper is to demonstrate that simulation studies, that is, experiments investigating synthetic data with known properties, are an invaluable tool for addressing these questions. We aim to provide a first introduction to simulation studies for data analysts or, more generally, for researchers involved at different levels in the analyses of health data, who (1) may rely on simulation studies published in statistical literature to choose their statistical methods and who, thus, need to understand the criteria of assessing the validity and relevance of simulation results and their interpretation; and/or (2) need to understand the basic principles of designing statistical simulations in order to efficiently collaborate with more experienced colleagues or start learning to conduct their own simulations. We illustrate the implementation of a simulation study and the interpretation of its results through a simple example inspired by recent literature, which is completely reproducible using the R-script available from online supplemental file 1.

Characterizing the multivariate physiogenomic response to environmental change

Global change is altering the climate that species have historically adapted to - in some cases at a pace not recently experienced in their evolutionary history - with cascading effects on all taxa. A central aim in global change biology is to understand how specific populations may be "primed" for global change, either through acclimation or adaptive standing genetic variation. It is therefore an important goal to link physiological measurements to the degree of stress a population experiences (Annual Review of Marine Science, 2012, 4, 39). Although "omic" approaches such as gene expression are often used as a proxy for the amount of stress experienced, we still have a poor understanding of how gene expression affects ecologically and physiologically relevant traits in non-model organisms. In a From the Cover paper in this issue of Molecular Ecology, Griffiths, Pan and Kelley (Molecular Ecology, 2019, 28) link gene expression to physiological traits in a temperate marine coral. They discover population-specific responses to ocean acidification for two populations that originated from locations with different histories of exposure to acidification. By integrating physiological and gene expression data, they were able to elucidate the mechanisms that explain these population-specific responses. Their results give insight into the physiogenomic feedbacks that may prime organisms or make them unfit for ocean global change.

Defining and Evaluating Microbial Contributions to Metabolite Variation in Microbiome-Metabolome Association Studies

Correlation-based analysis of paired microbiome-metabolome data sets is becoming a widespread research approach, aiming to comprehensively identify microbial drivers of metabolic variation. To date, however, the limitations of this approach and other microbiome-metabolome analysis methods have not been comprehensively evaluated. To address this challenge, we have introduced a mathematical framework to quantify the contribution of each taxon to metabolite variation based on uptake and secretion fluxes. We additionally used a multispecies metabolic model to simulate simplified gut communities, generating idealized microbiome-metabolome data sets. We then compared observed taxon-metabolite correlations in these data sets to calculated ground truth taxonomic contribution values. We found that in simulations of both a representative simple 10-species community and complex human gut microbiota, correlation-based analysis poorly identified key contributors, with an extremely low predictive value despite the idealized setting. We further demonstrate that the predictive value of correlation analysis is strongly influenced by both metabolite and taxon properties, as well as by exogenous environmental variation. We finally discuss the practical implications of our findings for interpreting microbiome-metabolome studies.IMPORTANCE Identifying the key microbial taxa responsible for metabolic differences between microbiomes is an important step toward understanding and manipulating microbiome metabolism. To achieve this goal, researchers commonly conduct microbiome-metabolome association studies, comprehensively measuring both the composition of species and the concentration of metabolites across a set of microbial community samples and then testing for correlations between microbes and metabolites. Here, we evaluated the utility of this general approach by first developing a rigorous mathematical definition of the contribution of each microbial taxon to metabolite variation and then examining these contributions in simulated data sets of microbial community metabolism. We found that standard correlation-based analysis of our simulated microbiome-metabolome data sets can identify true contributions with very low predictive value and that its performance depends strongly on specific properties of both metabolites and microbes, as well as on those of the surrounding environment. Combined, our findings can guide future interpretation and validation of microbiome-metabolome studies.

Genomics of host-pathogen interactions: challenges and opportunities across ecological and spatiotemporal scales

Evolutionary genomics has recently entered a new era in the study of host-pathogen interactions. A variety of novel genomic techniques has transformed the identification, detection and classification of both hosts and pathogens, allowing a greater resolution that helps decipher their underlying dynamics and provides novel insights into their environmental context. Nevertheless, many challenges to a general understanding of host-pathogen interactions remain, in particular in the synthesis and integration of concepts and findings across a variety of systems and different spatiotemporal and ecological scales. In this perspective we aim to highlight some of the commonalities and complexities across diverse studies of host-pathogen interactions, with a focus on ecological, spatiotemporal variation, and the choice of genomic methods used. We performed a quantitative review of recent literature to investigate links, patterns and potential tradeoffs between the complexity of genomic, ecological and spatiotemporal scales undertaken in individual host-pathogen studies. We found that the majority of studies used whole genome resolution to address their research objectives across a broad range of ecological scales, especially when focusing on the pathogen side of the interaction. Nevertheless, genomic studies conducted in a complex spatiotemporal context are currently rare in the literature. Because processes of host-pathogen interactions can be understood at multiple scales, from molecular-, cellular-, and physiological-scales to the levels of populations and ecosystems, we conclude that a major obstacle for synthesis across diverse host-pathogen systems is that data are collected on widely diverging scales with different degrees of resolution. This disparity not only hampers effective infrastructural organization of the data but also data granularity and accessibility. Comprehensive metadata deposited in association with genomic data in easily accessible databases will allow greater inference across systems in the future, especially when combined with open data standards and practices. The standardization and comparability of such data will facilitate early detection of emerging infectious diseases as well as studies of the impact of anthropogenic stressors, such as climate change, on disease dynamics in humans and wildlife.

The Effect of Neutral Recombination Variation on Genome Scans for Selection

Recently, there has been an increasing interest in identifying the role that regions of low recombination or inversion play in adaptation of species to local environments. Many examples of groups of adapted genes located within inversions are arising in the literature, in part inspired by theory that predicts the evolution of these so-called “supergenes.” We still, however, have a poor understanding of how genomic heterogeneity, such as varying rates of recombination, may confound signals of selection. Here, I evaluate the effect of neutral inversions and recombination variation on genome scans for selection, including tests for selective sweeps, differentiation outlier tests, and association tests. There is considerable variation among methods in their performance, with some methods being unaffected and some showing elevated false positive signals within a neutral inversion or region of low recombination. In some cases the false positive signal can be dampened or removed, if it is possible to use a quasi-independent set of SNPs to parameterize the model before performing the test. These results will be helpful to those seeking to understand the importance of regions of low recombination in adaptation.

Preparing next-generation scientists for biomedical big data: artificial intelligence approaches

Personalized medicine is being realized by our ability to measure biological and environmental information about patients. Much of these data are being stored in electronic health records yielding big data that presents challenges for its management and analysis. Here, we review several areas of knowledge that are necessary for next-generation scientists to fully realize the potential of biomedical big data. We begin with an overview of big data and its storage and management. We then review statistics and data science as foundational topics followed by a core curriculum of artificial intelligence, machine learning and natural language processing that are needed to develop predictive models for clinical decision making. We end with some specific training recommendations for preparing next-generation scientists for biomedical big data.