Learning from our GWAS mistakes: from experimental design to scientific method

Open Science Practices in Psychiatric Genetics: A Primer

Open science ensures that research is transparently reported and freely accessible for all to assess and collaboratively build on. Psychiatric genetics has led among the health sciences in implementing some open science practices in common study designs, such as replication as part of genome-wide association studies. However, thorough open science implementation guidelines are limited and largely not specific to data, privacy, and research conduct challenges in psychiatric genetics. Here, we present a primer of open science practices, including selection of a research topic with patients/nonacademic collaborators, equitable authorship and citation practices, design of replicable, reproducible studies, preregistrations, open data, and privacy issues. We provide tips for informative figures and inclusive, precise reporting. We discuss considerations in working with nonacademic collaborators and distributing research through preprints, blogs, social media, and accessible lecture materials. Finally, we provide extra resources to support every step of the research process.

Genome-Environment Associations, an Innovative Tool for Studying Heritable Evolutionary Adaptation in Orphan Crops and Wild Relatives

Leveraging innovative tools to speed up prebreeding and discovery of genotypic sources of adaptation from landraces, crop wild relatives, and orphan crops is a key prerequisite to accelerate genetic gain of abiotic stress tolerance in annual crops such as legumes and cereals, many of which are still orphan species despite advances in major row crops. Here, we review a novel, interdisciplinary approach to combine ecological climate data with evolutionary genomics under the paradigm of a new field of study: genome-environment associations (GEAs). We first exemplify how GEA utilizes in situ georeferencing from genotypically characterized, gene bank accessions to pinpoint genomic signatures of natural selection. We later discuss the necessity to update the current GEA models to predict both regional- and local- or micro-habitat-based adaptation with mechanistic ecophysiological climate indices and cutting-edge GWAS-type genetic association models. Furthermore, to account for polygenic evolutionary adaptation, we encourage the community to start gathering genomic estimated adaptive values (GEAVs) for genomic prediction (GP) and multi-dimensional machine learning (ML) models. The latter two should ideally be weighted by de novo GWAS-based GEA estimates and optimized for a scalable marker subset. We end the review by envisioning avenues to make adaptation inferences more robust through the merging of high-resolution data sources, such as environmental remote sensing and summary statistics of the genomic site frequency spectrum, with the epigenetic molecular functionality responsible for plastic inheritance in the wild. Ultimately, we believe that coupling evolutionary adaptive predictions with innovations in ecological genomics such as GEA will help capture hidden genetic adaptations to abiotic stresses based on crop germplasm resources to assist responses to climate change. "I shall endeavor to find out how nature's forces act upon one another, and in what manner the geographic environment exerts its influence on animals and plants. In short, I must find out about the harmony in nature" Alexander von Humboldt-Letter to Karl Freiesleben, June 1799.

TIGA: target illumination GWAS analytics

MOTIVATION

Genome-wide association studies can reveal important genotype-phenotype associations; however, data quality and interpretability issues must be addressed. For drug discovery scientists seeking to prioritize targets based on the available evidence, these issues go beyond the single study.

RESULTS

Here, we describe rational ranking, filtering and interpretation of inferred gene-trait associations and data aggregation across studies by leveraging existing curation and harmonization efforts. Each gene-trait association is evaluated for confidence, with scores derived solely from aggregated statistics, linking a protein-coding gene and phenotype. We propose a method for assessing confidence in gene-trait associations from evidence aggregated across studies, including a bibliometric assessment of scientific consensus based on the iCite relative citation ratio, and meanRank scores, to aggregate multivariate evidence.This method, intended for drug target hypothesis generation, scoring and ranking, has been implemented as an analytical pipeline, available as open source, with public datasets of results, and a web application designed for usability by drug discovery scientists.

AVAILABILITY AND IMPLEMENTATION

Web application, datasets and source code via https://unmtid-shinyapps.net/tiga/.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Omixer: multivariate and reproducible sample randomization to proactively counter batch effects in omics studies

MOTIVATION

Batch effects heavily impact results in omics studies, causing bias and false positive results, but software to control them preemptively is lacking. Sample randomization prior to measurement is vital for minimizing these effects, but current approaches are often ad hoc, poorly documented and ill-equipped to handle multiple batches and outcomes.

RESULTS

We developed Omixer-a Bioconductor package implementing multivariate and reproducible sample randomization for omics studies. It proactively counters correlations between technical factors and biological variables of interest by optimizing sample distribution across batches.

AVAILABILITYAND IMPLEMENTATION

Omixer is available from Bioconductor at http://bioconductor.org/packages/release/bioc/html/Omixer.html. Scripts and data used to generate figures available upon request.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Harnessing Crop Wild Diversity for Climate Change Adaptation

Warming and drought are reducing global crop production with a potential to substantially worsen global malnutrition. As with the green revolution in the last century, plant genetics may offer concrete opportunities to increase yield and crop adaptability. However, the rate at which the threat is happening requires powering new strategies in order to meet the global food demand. In this review, we highlight major recent 'big data' developments from both empirical and theoretical genomics that may speed up the identification, conservation, and breeding of exotic and elite crop varieties with the potential to feed humans. We first emphasize the major bottlenecks to capture and utilize novel sources of variation in abiotic stress (i.e., heat and drought) tolerance. We argue that adaptation of crop wild relatives to dry environments could be informative on how plant phenotypes may react to a drier climate because natural selection has already tested more options than humans ever will. Because isolated pockets of cryptic diversity may still persist in remote semi-arid regions, we encourage new habitat-based population-guided collections for genebanks. We continue discussing how to systematically study abiotic stress tolerance in these crop collections of wild and landraces using geo-referencing and extensive environmental data. By uncovering the genes that underlie the tolerance adaptive trait, natural variation has the potential to be introgressed into elite cultivars. However, unlocking adaptive genetic variation hidden in related wild species and early landraces remains a major challenge for complex traits that, as abiotic stress tolerance, are polygenic (i.e., regulated by many low-effect genes). Therefore, we finish prospecting modern analytical approaches that will serve to overcome this issue. Concretely, genomic prediction, machine learning, and multi-trait gene editing, all offer innovative alternatives to speed up more accurate pre- and breeding efforts toward the increase in crop adaptability and yield, while matching future global food demands in the face of increased heat and drought. In order for these 'big data' approaches to succeed, we advocate for a trans-disciplinary approach with open-source data and long-term funding. The recent developments and perspectives discussed throughout this review ultimately aim to contribute to increased crop adaptability and yield in the face of heat waves and drought events.

Learning Lessons on Reproducibility and Replicability in Large Scale Genome-Wide Association Studies

Reproducibility and replicability play a pivotal role in science. The article reflects on reproducibility and replicability as they figure in large scale genome-wide association studies. Overall, we emphasize the importance of enhancing data reproducibility, analysis reproducibility, and result replicability. We make recommendations pertaining to the development of study designs that address 1) batch effects and selection bias, 2) the incorporation of discrete discovery and replication phases, and 3) the procurement of a large sample size. We emphasize the importance of systematic and transparent data generation, processing, and quality control pipelines, as well as a rigorous field-specific standardized analysis protocol, We offer guidance with respect to collaborative frameworks, open access analysis tools, and software, and the use of supporting mandates, infrastructure, and repositories for data and resource sharing. Finally, we identify the role of incentives and culture in fueling the production of reproducible and replicable research through partnerships of researchers, funding agencies, and journals.

Predicting Thermal Adaptation by Looking Into Populations' Genomic Past

Molecular evolution offers an insightful theory to interpret the genomic consequences of thermal adaptation to previous events of climate change beyond range shifts. However, disentangling often mixed footprints of selective and demographic processes from those due to lineage sorting, recombination rate variation, and genomic constrains is not trivial. Therefore, here we condense current and historical population genomic tools to study thermal adaptation and outline key developments (genomic prediction, machine learning) that might assist their utilization for improving forecasts of populations' responses to thermal variation. We start by summarizing how recent thermal-driven selective and demographic responses can be inferred by coalescent methods and in turn how quantitative genetic theory offers suitable multi-trait predictions over a few generations via the breeder's equation. We later assume that enough generations have passed as to display genomic signatures of divergent selection to thermal variation and describe how these footprints can be reconstructed using genome-wide association and selection scans or, alternatively, may be used for forward prediction over multiple generations under an infinitesimal genomic prediction model. Finally, we move deeper in time to comprehend the genomic consequences of thermal shifts at an evolutionary time scale by relying on phylogeographic approaches that allow for reticulate evolution and ecological parapatric speciation, and end by envisioning the potential of modern machine learning techniques to better inform long-term predictions. We conclude that foreseeing future thermal adaptive responses requires bridging the multiple spatial scales of historical and predictive environmental change research under modern cohesive approaches such as genomic prediction and machine learning frameworks.

Novel Bead-Based Epitope Assay is a sensitive and reliable tool for profiling epitope-specific antibody repertoire in food allergy

Identification of allergenic IgE epitopes is instrumental for the development of novel diagnostic and prognostic methods in food allergy. In this work, we present the quantification and validation of a Bead-Based Epitope Assay (BBEA) that through multiplexing of epitopes and multiple sample processing enables completion of large experiments in a short period of time, using minimal quantities of patients’ blood. Peptides that are uniquely coupled to beads are incubated with serum or plasma samples, and after a secondary fluorophore-labeled antibody is added, the level of fluorescence is quantified with a Luminex reader. The signal is then normalized and converted to epitope-specific antibody binding values. We show that the effect of technical artifacts, i.e. well position or reading order, is minimal; and batch effects - different individual microplate runs - can be easily estimated and eliminated from the data. Epitope-specific antibody binding quantified with BBEA is highly reliable, reproducible and has greater sensitivity of epitope detection compared to peptide microarrays. IgE directed at allergenic epitopes is a sensitive biomarker of food allergy and can be used to predict allergy severity and phenotypes; and quantification of the relationship between epitope-specific IgE and IgG4 can further improve our understanding of the immune mechanisms behind allergic sensitization.

Perspective: Dimensions of the scientific method

The scientific method has been guiding biological research for a long time. It not only prescribes the order and types of activities that give a scientific study validity and a stamp of approval but also has substantially shaped how we collectively think about the endeavor of investigating nature. The advent of high-throughput data generation, data mining, and advanced computational modeling has thrown the formerly undisputed, monolithic status of the scientific method into turmoil. On the one hand, the new approaches are clearly successful and expect the same acceptance as the traditional methods, but on the other hand, they replace much of the hypothesis-driven reasoning with inductive argumentation, which philosophers of science consider problematic. Intrigued by the enormous wealth of data and the power of machine learning, some scientists have even argued that significant correlations within datasets could make the entire quest for causation obsolete. Many of these issues have been passionately debated during the past two decades, often with scant agreement. It is proffered here that hypothesis-driven, data-mining-inspired, and "allochthonous" knowledge acquisition, based on mathematical and computational models, are vectors spanning a 3D space of an expanded scientific method. The combination of methods within this space will most certainly shape our thinking about nature, with implications for experimental design, peer review and funding, sharing of result, education, medical diagnostics, and even questions of litigation.

PlateDesigner: a web-based application for the design of microplate experiments

SUMMARY

In biological assays, systematic variability, known as a batch effect, can often confound the effects of true biological conditions and has been well documented for a variety of high-throughput technologies. In microplate-based multiplex experiments, such as Luminex or OLINK assays, researchers need to consider both position and plate effects. Those effects can be easily accounted for if the experiments are properly designed, which includes randomization of the samples across multiple experimental runs. However, doing the ad hoc randomization becomes challenging when handling multiple samples. PlateDesigner is the first web-based application that provides randomization for microplate experiments, ensuring that the main principles of the experimental design, such as grouping samples from the same biological units and balancing the distribution of experimental conditions, are applied. Creating randomizations with PlateDesigner is simple and the results can be exported in a variety of formats, and easily integrated with microplate readers and statistical analysis software.

AVAILABILITY AND IMPLEMENTATION

PlateDesigner is written in R/Shiny and is hosted online by the Center of Biostatistics at the Icahn School of Medicine at Mount Sinai. This application is freely available at platedesigner.net.

Batch adjustment by reference alignment (BARA): Improved prediction performance in biological test sets with batch effects

Many biological data acquisition platforms suffer from inadvertent inclusion of biologically irrelevant variance in analyzed data, collectively termed batch effects. Batch effects can lead to difficulties in downstream analysis by lowering the power to detect biologically interesting differences and can in certain instances lead to false discoveries. They are especially troublesome in predictive modelling where samples in training sets and test sets are often completely correlated with batches. In this article, we present BARA, a normalization method for adjusting batch effects in predictive modelling. BARA utilizes a few reference samples to adjust for batch effects in a compressed data space spanned by the training set. We evaluate BARA using a collection of publicly available datasets and three different prediction models, and compare its performance to already existing methods developed for similar purposes. The results show that data normalized with BARA generates high and consistent prediction performances. Further, they suggest that BARA produces reliable performances independent of the examined classifiers. We therefore conclude that BARA has great potential to facilitate the development of predictive assays where test sets and training sets are correlated with batch.

Harnessing Human Microphysiology Systems as Key Experimental Models for Quantitative Systems Pharmacology

Two technologies that have emerged in the last decade offer a new paradigm for modern pharmacology, as well as drug discovery and development. Quantitative systems pharmacology (QSP) is a complementary approach to traditional, target-centric pharmacology and drug discovery and is based on an iterative application of computational and systems biology methods with multiscale experimental methods, both of which include models of ADME-Tox and disease. QSP has emerged as a new approach due to the low efficiency of success in developing therapeutics based on the existing target-centric paradigm. Likewise, human microphysiology systems (MPS) are experimental models complementary to existing animal models and are based on the use of human primary cells, adult stem cells, and/or induced pluripotent stem cells (iPSCs) to mimic human tissues and organ functions/structures involved in disease and ADME-Tox. Human MPS experimental models have been developed to address the relatively low concordance of human disease and ADME-Tox with engineered, experimental animal models of disease. The integration of the QSP paradigm with the use of human MPS has the potential to enhance the process of drug discovery and development.

Genomics of coloration in natural animal populations

Animal coloration has traditionally been the target of genetic and evolutionary studies. However, until very recently, the study of the genetic basis of animal coloration has been mainly restricted to model species, whereas research on non-model species has been either neglected or mainly based on candidate approaches, and thereby limited by the knowledge obtained in model species. Recent high-throughput sequencing technologies allow us to overcome previous limitations, and open new avenues to study the genetic basis of animal coloration in a broader number of species and colour traits, and to address the general relevance of different genetic structures and their implications for the evolution of colour. In this review, we highlight aspects where genome-wide studies could be of major utility to fill in the gaps in our understanding of the biology and evolution of animal coloration. The new genomic approaches have been promptly adopted to study animal coloration although substantial work is still needed to consider a larger range of species and colour traits, such as those exhibiting continuous variation or based on reflective structures. We argue that a robust advancement in the study of animal coloration will also require large efforts to validate the functional role of the genes and variants discovered using genome-wide tools.This article is part of the themed issue 'Animal coloration: production, perception, function and application'.

Proper experimental design requires randomization/balancing of molecular ecology experiments

Properly designed (randomized and/or balanced) experiments are standard in ecological research. Molecular methods are increasingly used in ecology, but studies generally do not report the detailed design of sample processing in the laboratory. This may strongly influence the interpretability of results if the laboratory procedures do not account for the confounding effects of unexpected laboratory events. We demonstrate this with a simple experiment where unexpected differences in laboratory processing of samples would have biased results if randomization in DNA extraction and PCR steps do not provide safeguards. We emphasize the need for proper experimental design and reporting of the laboratory phase of molecular ecology research to ensure the reliability and interpretability of results.

Identifying and mitigating batch effects in whole genome sequencing data

BACKGROUND

Large sample sets of whole genome sequencing with deep coverage are being generated, however assembling datasets from different sources inevitably introduces batch effects. These batch effects are not well understood and can be due to changes in the sequencing protocol or bioinformatics tools used to process the data. No systematic algorithms or heuristics exist to detect and filter batch effects or remove associations impacted by batch effects in whole genome sequencing data.

RESULTS

We describe key quality metrics, provide a freely available software package to compute them, and demonstrate that identification of batch effects is aided by principal components analysis of these metrics. To mitigate batch effects, we developed new site-specific filters that identified and removed variants that falsely associated with the phenotype due to batch effect. These include filtering based on: a haplotype based genotype correction, a differential genotype quality test, and removing sites with missing genotype rate greater than 30% after setting genotypes with quality scores less than 20 to missing. This method removed 96.1% of unconfirmed genome-wide significant SNP associations and 97.6% of unconfirmed genome-wide significant indel associations. We performed analyses to demonstrate that: 1) These filters impacted variants known to be disease associated as 2 out of 16 confirmed associations in an AMD candidate SNP analysis were filtered, representing a reduction in power of 12.5%, 2) In the absence of batch effects, these filters removed only a small proportion of variants across the genome (type I error rate of 3%), and 3) in an independent dataset, the method removed 90.2% of unconfirmed genome-wide SNP associations and 89.8% of unconfirmed genome-wide indel associations.

CONCLUSIONS

Researchers currently do not have effective tools to identify and mitigate batch effects in whole genome sequencing data. We developed and validated methods and filters to address this deficiency.

Copy Number Variants in Alzheimer's Disease

Alzheimer's disease (AD) is a devastating disease mainly afflicting elderly people, characterized by decreased cognition, loss of memory, and eventually death. Although risk and deterministic genes are known, major genetics research programs are underway to gain further insights into the inheritance of AD. In the last years, in particular, new developments in genome-wide scanning methodologies have enabled the association of a number of previously uncharacterized copy number variants (CNVs, gain or loss of DNA) in AD. Because of the exceedingly large number of studies performed, it has become difficult for geneticists as well as clinicians to systematically follow, evaluate, and interpret the growing number of (sometime conflicting) CNVs implicated in AD. In this review, after a brief introduction of this type of structural variation, and a description of available databases, computational analyses, and technologies involved, we provide a systematic review of all published data showing statistical and scientific significance of pathogenic CNVs and discuss the role they might play in AD.

Genomics era and complex disorders: Implications of GWAS with special reference to coronary artery disease, type 2 diabetes mellitus, and cancers

The Human Genome Project (HGP) has identified millions of single nucleotide polymorphisms (SNPs) and their association with several diseases, apart from successfully characterizing the Mendelian/monogenic diseases. However, the dissection of precise etiology of complex genetic disorders still poses a challenge for human geneticists. This review outlines the landmark results of genome-wide association studies (GWAS) with respect to major complex diseases - Coronary artery disease (CAD), type 2 diabetes mellitus (T2DM), and predominant cancers. A brief account on the current Indian scenario is also given. All the relevant publications till mid-2015 were accessed through web databases such as PubMed and Google. Several databases providing genetic information related to these diseases were tabulated and in particular, the list of the most significant SNPs identified through GWAS was made, which may be useful for designing studies in functional validation. Post-GWAS implications and emerging concepts such as epigenomics and pharmacogenomics were also discussed.

BatchQC: interactive software for evaluating sample and batch effects in genomic data

Sequencing and microarray samples often are collected or processed in multiple batches or at different times. This often produces technical biases that can lead to incorrect results in the downstream analysis. There are several existing batch adjustment tools for ‘-omics’ data, but they do not indicate a priori whether adjustment needs to be conducted or how correction should be applied. We present a software pipeline, BatchQC, which addresses these issues using interactive visualizations and statistics that evaluate the impact of batch effects in a genomic dataset. BatchQC can also apply existing adjustment tools and allow users to evaluate their benefits interactively. We used the BatchQC pipeline on both simulated and real data to demonstrate the effectiveness of this software toolkit.

Availability and Implementation: BatchQC is available through Bioconductor: http://bioconductor.org/packages/BatchQC and GitHub: https://github.com/mani2012/BatchQC.

Contact:wej@bu.edu

Supplementary information:Supplementary data are available at Bioinformatics online.

Taking Bioinformatics to Systems Medicine

Systems medicine promotes a range of approaches and strategies to study human health and disease at a systems level with the aim of improving the overall well-being of (healthy) individuals, and preventing, diagnosing, or curing disease. In this chapter we discuss how bioinformatics critically contributes to systems medicine. First, we explain the role of bioinformatics in the management and analysis of data. In particular we show the importance of publicly available biological and clinical repositories to support systems medicine studies. Second, we discuss how the integration and analysis of multiple types of omics data through integrative bioinformatics may facilitate the determination of more predictive and robust disease signatures, lead to a better understanding of (patho)physiological molecular mechanisms, and facilitate personalized medicine. Third, we focus on network analysis and discuss how gene networks can be constructed from omics data and how these networks can be decomposed into smaller modules. We discuss how the resulting modules can be used to generate experimentally testable hypotheses, provide insight into disease mechanisms, and lead to predictive models. Throughout, we provide several examples demonstrating how bioinformatics contributes to systems medicine and discuss future challenges in bioinformatics that need to be addressed to enable the advancement of systems medicine.

Practical impacts of genomic data "cleaning" on biological discovery using surrogate variable analysis

Background

Genomic data production is at its highest level and continues to increase, making available novel primary data and existing public data to researchers for exploration. Here we explore the consequences of “batch” correction for biological discovery in two publicly available expression datasets. We consider this to include the estimation of and adjustment for wide-spread systematic heterogeneity in genomic measurements that is unrelated to the effects under study, whether it be technical or biological in nature.

Methods

We present three illustrative data analyses using surrogate variable analysis (SVA) and describe how to perform artifact discovery in light of natural heterogeneity within biological groups, secondary biological questions of interest, and non-linear treatment effects in a dataset profiling differentiating pluripotent cells (GSE32923) and another from human brain tissue (GSE30272).

Results

Careful specification of biological effects of interest is very important to factor-based approaches like SVA. We demonstrate greatly sharpened global and gene-specific differential expression across treatment groups in stem cell systems. Similarly, we demonstrate how to preserve major non-linear effects of age across the lifespan in the brain dataset. However, the gains in precisely defining known effects of interest come at the cost of much other information in the “cleaned” data, including sex, common copy number effects and sample or cell line-specific molecular behavior.

Conclusions

Our analyses indicate that data “cleaning” can be an important component of high-throughput genomic data analysis when interrogating explicitly defined effects in the context of data affected by robust technical artifacts. However, caution should be exercised to avoid removing biological signal of interest. It is also important to note that open data exploration is not possible after such supervised “cleaning”, because effects beyond those stipulated by the researcher may have been removed. With the goal of making these statistical algorithms more powerful and transparent to researchers in the biological sciences, we provide exploratory plots and accompanying R code for identifying and guiding “cleaning” process (https://github.com/andrewejaffe/StemCellSVA). The impact of these methods is significant enough that we have made newly processed data available for the brain data set at http://braincloud.jhmi.edu/plots/ and GSE30272.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-015-0808-5) contains supplementary material, which is available to authorized users.

Genome-wide association and functional studies identify a role for IGFBP3 in hip osteoarthritis

OBJECTIVES

To identify genetic associations with hip osteoarthritis (HOA), we performed a meta-analysis of genome-wide association studies (GWAS) of HOA.

METHODS

The GWAS meta-analysis included approximately 2.5 million imputed HapMap single nucleotide polymorphisms (SNPs). HOA cases and controls defined radiographically and by total hip replacement were selected from the Osteoporotic Fractures in Men (MrOS) Study and the Study of Osteoporotic Fractures (SOF) (654 cases and 4697 controls, combined). Replication of genome-wide significant SNP associations (p ≤5×10(-8)) was examined in five studies (3243 cases and 6891 controls, combined). Functional studies were performed using in vitro models of chondrogenesis and osteogenesis.

RESULTS

The A allele of rs788748, located 65 kb upstream of the IGFBP3 gene, was associated with lower HOA odds at the genome-wide significance level in the discovery stage (OR 0.71, p=2×10(-8)). The association replicated in five studies (OR 0.92, p=0.020), but the joint analysis of discovery and replication results was not genome-wide significant (p=1×10(-6)). In separate study populations, the rs788748 A allele was also associated with lower circulating IGFBP3 protein levels (p=4×10(-13)), suggesting that this SNP or a variant in linkage disequilibrium could be an IGFBP3 regulatory variant. Results from functional studies were consistent with association results. Chondrocyte hypertrophy, a deleterious event in OA pathogenesis, was largely prevented upon IGFBP3 knockdown in chondrocytes. Furthermore, IGFBP3 overexpression induced cartilage catabolism and osteogenic differentiation.

CONCLUSIONS

Results from GWAS and functional studies provided suggestive links between IGFBP3 and HOA.

RIPOSTE: a framework for improving the design and analysis of laboratory-based research

Lack of reproducibility is an ongoing problem in some areas of the biomedical sciences. Poor experimental design and a failure to engage with experienced statisticians at key stages in the design and analysis of experiments are two factors that contribute to this problem. The RIPOSTE (Reducing IrreProducibility in labOratory STudiEs) framework has been developed to support early and regular discussions between scientists and statisticians in order to improve the design, conduct and analysis of laboratory studies and, therefore, to reduce irreproducibility. This framework is intended for use during the early stages of a research project, when specific questions or hypotheses are proposed. The essential points within the framework are explained and illustrated using three examples (a medical equipment test, a macrophage study and a gene expression study). Sound study design minimises the possibility of bias being introduced into experiments and leads to higher quality research with more reproducible results.

Biological knowledge-driven analysis of epistasis in human GWAS with application to lipid traits

While the importance of epistasis is well established, specific gene-gene interactions have rarely been identified in human genome-wide association studies (GWAS), mainly due to low power associated with such interaction tests. In this chapter, we integrate biological knowledge and human GWAS data to reveal epistatic interactions underlying quantitative lipid traits, which are major risk factors for coronary artery disease. To increase power to detect interactions, we only tested pairs of SNPs filtered by prior biological knowledge, including GWAS results, protein-protein interactions (PPIs), and pathway information. Using published GWAS and 9,713 European Americans (EA) from the Atherosclerosis Risk in Communities (ARIC) study, we identified an interaction between HMGCR and LIPC affecting high-density lipoprotein cholesterol (HDL-C) levels. We then validated this interaction in additional multiethnic cohorts from ARIC, the Framingham Heart Study, and the Multi-Ethnic Study of Atherosclerosis. Both HMGCR and LIPC are involved in the metabolism of lipids and lipoproteins, and LIPC itself has been marginally associated with HDL-C. Furthermore, no significant interaction was detected using PPI and pathway information, mainly due to the stringent significance level required after correcting for the large number of tests conducted. These results suggest the potential of biological knowledge-driven approaches to detect epistatic interactions in human GWAS, which may hold the key to exploring the role gene-gene interactions play in connecting genotypes and complex phenotypes in future GWAS.

svaseq: removing batch effects and other unwanted noise from sequencing data

It is now known that unwanted noise and unmodeled artifacts such as batch effects can dramatically reduce the accuracy of statistical inference in genomic experiments. These sources of noise must be modeled and removed to accurately measure biological variability and to obtain correct statistical inference when performing high-throughput genomic analysis. We introduced surrogate variable analysis (sva) for estimating these artifacts by (i) identifying the part of the genomic data only affected by artifacts and (ii) estimating the artifacts with principal components or singular vectors of the subset of the data matrix. The resulting estimates of artifacts can be used in subsequent analyses as adjustment factors to correct analyses. Here I describe a version of the sva approach specifically created for count data or FPKMs from sequencing experiments based on appropriate data transformation. I also describe the addition of supervised sva (ssva) for using control probes to identify the part of the genomic data only affected by artifacts. I present a comparison between these versions of sva and other methods for batch effect estimation on simulated data, real count-based data and FPKM-based data. These updates are available through the sva Bioconductor package and I have made fully reproducible analysis using these methods available from: https://github.com/jtleek/svaseq.

Genome-wide association studies of suicidal behaviors: a review

Suicidal behaviors represent a fatal dimension of mental ill-health, involving both environmental and heritable (genetic) influences. The putative genetic components of suicidal behaviors have until recent years been mainly investigated by hypothesis-driven research (of "candidate genes"). But technological progress in genotyping has opened the possibilities towards (hypothesis-generating) genomic screens and novel opportunities to explore polygenetic perspectives, now spanning a wide array of possible analyses falling under the term Genome-Wide Association Study (GWAS). Here we introduce and discuss broadly some apparent limitations but also certain developing opportunities of GWAS. We summarize the results from all the eight GWAS conducted up to date focused on suicidality outcomes; treatment emergent suicidal ideation (3 studies), suicide attempts (4 studies) and completed suicides (1 study). Clearly, there are few (if any) genome-wide significant and reproducible findings yet to be demonstrated. We then discuss and pinpoint certain future considerations in relation to sample sizes, the units of genetic associations used, study designs and outcome definitions, psychiatric diagnoses or biological measures, as well as the use of genomic sequencing. We conclude that GWAS should have a lot more potential to show in the case of suicidal outcomes, than what has yet been realized.

Removing batch effects for prediction problems with frozen surrogate variable analysis

Batch effects are responsible for the failure of promising genomic prognostic signatures, major ambiguities in published genomic results, and retractions of widely-publicized findings. Batch effect corrections have been developed to remove these artifacts, but they are designed to be used in population studies. But genomic technologies are beginning to be used in clinical applications where samples are analyzed one at a time for diagnostic, prognostic, and predictive applications. There are currently no batch correction methods that have been developed specifically for prediction. In this paper, we propose an new method called frozen surrogate variable analysis (fSVA) that borrows strength from a training set for individual sample batch correction. We show that fSVA improves prediction accuracy in simulations and in public genomic studies. fSVA is available as part of the sva Bioconductor package.

Rare-variant association analysis: study designs and statistical tests

Despite the extensive discovery of trait- and disease-associated common variants, much of the genetic contribution to complex traits remains unexplained. Rare variants can explain additional disease risk or trait variability. An increasing number of studies are underway to identify trait- and disease-associated rare variants. In this review, we provide an overview of statistical issues in rare-variant association studies with a focus on study designs and statistical tests. We present the design and analysis pipeline of rare-variant studies and review cost-effective sequencing designs and genotyping platforms. We compare various gene- or region-based association tests, including burden tests, variance-component tests, and combined omnibus tests, in terms of their assumptions and performance. Also discussed are the related topics of meta-analysis, population-stratification adjustment, genotype imputation, follow-up studies, and heritability due to rare variants. We provide guidelines for analysis and discuss some of the challenges inherent in these studies and future research directions.

Development of the Biological Experimental Design Concept Inventory (BEDCI)

Interest in student conception of experimentation inspired the development of a fully validated 14-question inventory on experimental design in biology (BEDCI) by following established best practices in concept inventory (CI) design. This CI can be used to diagnose specific examples of non-expert-like thinking in students and to evaluate the success of teaching strategies that target conceptual changes. We used BEDCI to diagnose non-expert-like student thinking in experimental design at the pre- and posttest stage in five courses (total n = 580 students) at a large research university in western Canada. Calculated difficulty and discrimination metrics indicated that BEDCI questions are able to effectively capture learning changes at the undergraduate level. A high correlation (r = 0.84) between responses by students in similar courses and at the same stage of their academic career, also suggests that the test is reliable. Students showed significant positive learning changes by the posttest stage, but some non-expert-like responses were widespread and persistent. BEDCI is a reliable and valid diagnostic tool that can be used in a variety of life sciences disciplines.

A structured approach to predictive modeling of a two-class problem using multidimensional data sets

Biological experiments in the post-genome era can generate a staggering amount of complex data that challenges experimentalists to extract meaningful information. Increasingly, the success of an appropriately controlled experiment relies on a robust data analysis pipeline. In this paper, we present a structured approach to the analysis of multidimensional data that relies on a close, two-way communication between the bioinformatician and experimentalist. A sequential approach employing data exploration (visualization, graphical and analytical study), pre-processing, feature reduction and supervised classification using machine learning is presented. This standardized approach is illustrated by an example from a proteomic data analysis that has been used to predict the risk of infectious disease outcome. Strategies for model selection and post hoc model diagnostics are presented and applied to the case illustration. We discuss some of the practical lessons we have learned applying supervised classification to multidimensional data sets, one of which is the importance of feature reduction in achieving optimal modeling performance.

Genetic variants and their interactions in disease risk prediction - machine learning and network perspectives

A central challenge in systems biology and medical genetics is to understand how interactions among genetic loci contribute to complex phenotypic traits and human diseases. While most studies have so far relied on statistical modeling and association testing procedures, machine learning and predictive modeling approaches are increasingly being applied to mining genotype-phenotype relationships, also among those associations that do not necessarily meet statistical significance at the level of individual variants, yet still contributing to the combined predictive power at the level of variant panels. Network-based analysis of genetic variants and their interaction partners is another emerging trend by which to explore how sub-network level features contribute to complex disease processes and related phenotypes. In this review, we describe the basic concepts and algorithms behind machine learning-based genetic feature selection approaches, their potential benefits and limitations in genome-wide setting, and how physical or genetic interaction networks could be used as a priori information for providing improved predictive power and mechanistic insights into the disease networks. These developments are geared toward explaining a part of the missing heritability, and when combined with individual genomic profiling, such systems medicine approaches may also provide a principled means for tailoring personalized treatment strategies in the future.

The use and misuse of biomedical data: is bigger really better?

Very large biomedical research databases, containing electronic health records (EHR) and genomic data from millions of patients, have been heralded recently for their potential to accelerate scientific discovery and produce dramatic improvements in medical treatments. Research enabled by these databases may also lead to profound changes in law, regulation, social policy, and even litigation strategies. Yet, is "big data" necessarily better data? This paper makes an original contribution to the legal literature by focusing on what can go wrong in the process of biomedical database research and what precautions are necessary to avoid critical mistakes. We address three main reasons for approaching such research with care and being cautious in relying on its outcomes for purposes of public policy or litigation. First, the data contained in biomedical databases is surprisingly likely to be incorrect or incomplete. Second, systematic biases, arising from both the nature of the data and the preconceptions of investigators, are serious threats to the validity of research results, especially in answering causal questions. Third, data mining of biomedical databases makes it easier for individuals with political, social, or economic agendas to generate ostensibly scientific but misleading research findings for the purpose of manipulating public opinion and swaying policymakers. In short, this paper sheds much-needed light on the problems of credulous and uninformed acceptance of research results derived from biomedical databases. An understanding of the pitfalls of big data analysis is of critical importance to anyone who will rely on or dispute its outcomes, including lawyers, policymakers, and the public at large. The Article also recommends technical, methodological, and educational interventions to combat the dangers of database errors and abuses.

OSAT: a tool for sample-to-batch allocations in genomics experiments

Background

Batch effect is one type of variability that is not of primary interest but ubiquitous in sizable genomic experiments. To minimize the impact of batch effects, an ideal experiment design should ensure the even distribution of biological groups and confounding factors across batches. However, due to the practical complications, the availability of the final collection of samples in genomics study might be unbalanced and incomplete, which, without appropriate attention in sample-to-batch allocation, could lead to drastic batch effects. Therefore, it is necessary to develop effective and handy tool to assign collected samples across batches in an appropriate way in order to minimize the impact of batch effects.

Results

We describe OSAT (Optimal Sample Assignment Tool), a bioconductor package designed for automated sample-to-batch allocations in genomics experiments.

Conclusions

OSAT is developed to facilitate the allocation of collected samples to different batches in genomics study. Through optimizing the even distribution of samples in groups of biological interest into different batches, it can reduce the confounding or correlation between batches and the biological variables of interest. It can also optimize the homogeneous distribution of confounding factors across batches. It can handle challenging instances where incomplete and unbalanced sample collections are involved as well as ideally balanced designs.

Clinical review: Genome-wide association studies of skeletal phenotypes: what we have learned and where we are headed

CONTEXT

The primary goals of genome-wide association studies (GWAS) are to discover new molecular and biological pathways involved in the regulation of bone metabolism that can be leveraged for drug development. In addition, the identified genetic determinants may be used to enhance current risk factor profiles.

EVIDENCE ACQUISITION

There have been more than 40 published GWAS on skeletal phenotypes, predominantly focused on dual-energy x-ray absorptiometry-derived bone mineral density (BMD) of the hip and spine.

EVIDENCE SYNTHESIS

Sixty-six BMD loci have been replicated across all the published GWAS, confirming the highly polygenic nature of BMD variation. Only seven of the 66 previously reported genes (LRP5, SOST, ESR1, TNFRSF11B, TNFRSF11A, TNFSF11, PTH) from candidate gene association studies have been confirmed by GWAS. Among 59 novel BMD GWAS loci that have not been reported by previous candidate gene association studies, some have been shown to be involved in key biological pathways involving the skeleton, particularly Wnt signaling (AXIN1, LRP5, CTNNB1, DKK1, FOXC2, HOXC6, LRP4, MEF2C, PTHLH, RSPO3, SFRP4, TGFBR3, WLS, WNT3, WNT4, WNT5B, WNT16), bone development: ossification (CLCN7, CSF1, MEF2C, MEPE, PKDCC, PTHLH, RUNX2, SOX6, SOX9, SPP1, SP7), mesenchymal-stem-cell differentiation (FAM3C, MEF2C, RUNX2, SOX4, SOX9, SP7), osteoclast differentiation (JAG1, RUNX2), and TGF-signaling (FOXL1, SPTBN1, TGFBR3). There are still 30 BMD GWAS loci without prior molecular or biological evidence of their involvement in skeletal phenotypes. Other skeletal phenotypes that either have been or are being studied include hip geometry, bone ultrasound, quantitative computed tomography, high-resolution peripheral quantitative computed tomography, biochemical markers, and fractures such as vertebral, nonvertebral, hip, and forearm.

CONCLUSIONS

Although several challenges lie ahead as GWAS moves into the next generation, there are prospects of new discoveries in skeletal biology. This review integrates findings from previous GWAS and provides a roadmap for future directions building on current GWAS successes.