Ten Simple Rules for Effective Statistical Practice

Ten simple rules for managing laboratory information

Information is the cornerstone of research, from experimental (meta)data and computational processes to complex inventories of reagents and equipment. These 10 simple rules discuss best practices for leveraging laboratory information management systems to transform this large information load into useful scientific findings.

Guidance on protocol development for EFSA generic scientific assessments

EFSA Strategy 2027 outlines the need for fit-for-purpose protocols for EFSA generic scientific assessments to aid in delivering trustworthy scientific advice. This EFSA Scientific Committee guidance document helps address this need by providing a harmonised and flexible framework for developing protocols for EFSA generic assessments. The guidance replaces the 'Draft framework for protocol development for EFSA's scientific assessments' published in 2020. The two main steps in protocol development are described. The first is problem formulation, which illustrates the objectives of the assessment. Here a new approach to translating the mandated Terms of Reference into scientifically answerable assessment questions and sub-questions is proposed: the 'APRIO' paradigm (Agent, Pathway, Receptor, Intervention and Output). Owing to its cross-cutting nature, this paradigm is considered adaptable and broadly applicable within and across the various EFSA domains and, if applied using the definitions given in this guidance, is expected to help harmonise the problem formulation process and outputs and foster consistency in protocol development. APRIO may also overcome the difficulty of implementing some existing frameworks across the multiple EFSA disciplines, e.g. the PICO/PECO approach (Population, Intervention/Exposure, Comparator, Outcome). Therefore, although not mandatory, APRIO is recommended. The second step in protocol development is the specification of the evidence needs and the methods that will be applied for answering the assessment questions and sub-questions, including uncertainty analysis. Five possible approaches to answering individual (sub-)questions are outlined: using evidence from scientific literature and study reports; using data from databases other than bibliographic; using expert judgement informally collected or elicited via semi-formal or formal expert knowledge elicitation processes; using mathematical/statistical models; and - not covered in this guidance - generating empirical evidence ex novo. The guidance is complemented by a standalone 'template' for EFSA protocols that guides the users step by step through the process of planning an EFSA scientific assessment.

Identification of interacting neural populations: methods and statistical considerations

As improved recording technologies have created new opportunities for neurophysiological investigation, emphasis has shifted from individual neurons to multiple populations that form circuits, and it has become important to provide evidence of cross-population coordinated activity. We review various methods for doing so, placing them in six major categories while avoiding technical descriptions and instead focusing on high-level motivations and concerns. Our aim is to indicate what the methods can achieve and the circumstances under which they are likely to succeed. Toward this end, we include a discussion of four cross-cutting issues: the definition of neural populations, trial-to-trial variability and Poisson-like noise, time-varying dynamics, and causality.

Health record hiccups-5,526 real-world time series with change points labelled by crowdsourced visual inspection

BACKGROUND

Large routinely collected data such as electronic health records (EHRs) are increasingly used in research, but the statistical methods and processes used to check such data for temporal data quality issues have not moved beyond manual, ad hoc production and visual inspection of graphs. With the prospect of EHR data being used for disease surveillance via automated pipelines and public-facing dashboards, automation of data quality checks will become increasingly valuable.

FINDINGS

We generated 5,526 time series from 8 different EHR datasets and engaged >2,000 citizen-science volunteers to label the locations of all suspicious-looking change points in the resulting graphs. Consensus labels were produced using density-based clustering with noise, with validation conducted using 956 images containing labels produced by an experienced data scientist. Parameter tuning was done against 670 images and performance calculated against 286 images, resulting in a final sensitivity of 80.4% (95% CI, 77.1%-83.3%), specificity of 99.8% (99.7%-99.8%), positive predictive value of 84.5% (81.4%-87.2%), and negative predictive value of 99.7% (99.6%-99.7%). In total, 12,745 change points were found within 3,687 of the time series.

CONCLUSIONS

This large collection of labelled EHR time series can be used to validate automated methods for change point detection in real-world settings, encouraging the development of methods that can successfully be applied in practice. It is particularly valuable since change point detection methods are typically validated using synthetic data, so their performance in real-world settings cannot be assumed to be comparable. While the dataset focusses on EHRs and data quality, it should also be applicable in other fields.

Using digital interventions to reduce digitalisation-related stress-does it work?

Digitalisation entails positive and negative consequences for employees. In a longitudinal, randomized control group design over 14 days (N = 95 participants), we piloted and expected each of three app-based interventions to positively influence general well-being, well-being related to information and communication technology (ICT), and recovery compared to the control group with no intervention. The meditation intervention significantly increased general well-being (satisfaction) and recovery (detachment) compared to the control group but did not reduce general stress. The cognitive-behavioural intervention significantly increased general well-being (less stress). The informational intervention however increased the general stress level. No intervention changed the level of ICT-specific well-being. Thus, classic stress interventions conveyed via ICTs (app-based) may be effective for addressing classic stress symptoms, but not yet for new forms of stress. Future research should investigate structural differences between classic stressors and new kinds of ICT-related stressors to identify starting points for new types of interventions.

Frontiers in quantifying wildlife behavioural responses to chemical pollution

Animal behaviour is remarkably sensitive to disruption by chemical pollution, with widespread implications for ecological and evolutionary processes in contaminated wildlife populations. However, conventional approaches applied to study the impacts of chemical pollutants on wildlife behaviour seldom address the complexity of natural environments in which contamination occurs. The aim of this review is to guide the rapidly developing field of behavioural ecotoxicology towards increased environmental realism, ecological complexity, and mechanistic understanding. We identify research areas in ecology that to date have been largely overlooked within behavioural ecotoxicology but which promise to yield valuable insights, including within- and among-individual variation, social networks and collective behaviour, and multi-stressor interactions. Further, we feature methodological and technological innovations that enable the collection of data on pollutant-induced behavioural changes at an unprecedented resolution and scale in the laboratory and the field. In an era of rapid environmental change, there is an urgent need to advance our understanding of the real-world impacts of chemical pollution on wildlife behaviour. This review therefore provides a roadmap of the major outstanding questions in behavioural ecotoxicology and highlights the need for increased cross-talk with other disciplines in order to find the answers.

A Brief Guide to Statistical Analysis and Presentation for the Plant Pathology Journal

Statistical analysis of data is an integral part of research projects in all scientific disciplines including the plant pathology. Appropriate design, application and interpretation of statistical analysis are also, therefore, at the center of publishing and properly evaluating studies in plant pathology. A survey of research works published in the Plant Pathology Journal, however, cast doubt on high standard of statistical analysis required for scientific rigor and reproducibility in the journal. Here I first describe, based on the survey of published works, what mistakes are commonly made and what components are often lacking during statistical analysis and interpretation of its results. Next, I provide possible remedies and suggestions to help guide researchers in preparing manuscript and reviewers in evaluating manuscripts submitted to the Plant Pathology Journal. This is not aiming at delineating technical and practical details of particular statistical methods or approaches.

Practical foundations of machine learning for addiction research. Part II. Workflow and use cases

In a continuum with applied statistics, machine learning offers a wide variety of tools to explore, analyze, and understand addiction data. These tools include algorithms that can leverage useful information from data to build models; these models can solve particular tasks to answer addiction scientific questions. In this second part of a two-part review on machine learning, we explain how to apply machine learning methods to addiction research. Like other analytical tools, machine learning methods require a careful implementation to carry out a reproducible and transparent research process with reliable results. This review describes a workflow to guide the application of machine learning in addiction research, detailing study design, data collection, data pre-processing, modeling, and results communication. How to train, validate, and test a model, detect and characterize overfitting, and determine an adequate sample size are some of the key issues when applying machine learning. We also illustrate the process and particular nuances with examples of how researchers in addiction have applied machine learning techniques with different goals, study designs, or data sources as well as explain the main limitations of machine learning approaches and how to best address them. A good use of machine learning enriches the addiction research toolkit.

What to Do If N Is Two?

The field of in vivo neurophysiology currently uses statistical standards that are based on tradition rather than formal analysis. Typically, data from two (or few) animals are pooled for one statistical test, or a significant test in a first animal is replicated in one (or few) further animals. The use of more than one animal is widely believed to allow an inference on the population. Here, we explain that a useful inference on the population would require larger numbers and a different statistical approach. The field should consider to perform studies at that standard, potentially through coordinated multicenter efforts, for selected questions of exceptional importance. Yet, for many questions, this is ethically and/or economically not justifiable. We explain why in those studies with two (or few) animals, any useful inference is limited to the sample of investigated animals, irrespective of whether it is based on few animals, two animals, or a single animal.

Should I use fixed effects or random effects when I have fewer than five levels of a grouping factor in a mixed-effects model?

As linear mixed-effects models (LMMs) have become a widespread tool in ecology, the need to guide the use of such tools is increasingly important. One common guideline is that one needs at least five levels of the grouping variable associated with a random effect. Having so few levels makes the estimation of the variance of random effects terms (such as ecological sites, individuals, or populations) difficult, but it need not muddy one's ability to estimate fixed effects terms-which are often of primary interest in ecology. Here, I simulate datasets and fit simple models to show that having few random effects levels does not strongly influence the parameter estimates or uncertainty around those estimates for fixed effects terms-at least in the case presented here. Instead, the coverage probability of fixed effects estimates is sample size dependent. LMMs including low-level random effects terms may come at the expense of increased singular fits, but this did not appear to influence coverage probability or RMSE, except in low sample size (N = 30) scenarios. Thus, it may be acceptable to use fewer than five levels of random effects if one is not interested in making inferences about the random effects terms (i.e. when they are 'nuisance' parameters used to group non-independent data), but further work is needed to explore alternative scenarios. Given the widespread accessibility of LMMs in ecology and evolution, future simulation studies and further assessments of these statistical methods are necessary to understand the consequences both of violating and of routinely following simple guidelines.

Hypothalamus volume and DNA methylation of stress axis genes in major depressive disorder: A CAN-BIND study report

Dysfunction of the hypothalamic-pituitary-adrenal (HPA) axis is considered one of the mechanisms underlying the development of major depressive disorder (MDD), but the exact nature of this dysfunction is unknown. We investigated the relationship between hypothalamus volume (HV) and blood-derived DNA methylation in MDD. We obtained brain MRI, clinical and molecular data from 181 unmedicated MDD and 90 healthy control (HC) participants. MDD participants received a 16-week standardized antidepressant treatment protocol, as part of the first Canadian Biomarker Integration Network in Depression (CAN-BIND) study. We collected bilateral HV measures via manual segmentation by two independent raters. DNA methylation and RNA sequencing were performed for three key HPA axis-regulating genes coding for the corticotropin-binding protein (CRHBP), glucocorticoid receptor (NR3C1) and FK506 binding protein 5 (FKBP5). We used elastic net regression to perform variable selection and assess predictive ability of methylation variables on HV. Left HV was negatively associated with duration of current episode (ρ = -0.17, p = 0.035). We did not observe significant differences in HV between MDD and HC or any associations between HV and treatment response at weeks 8 or 16, overall depression severity, illness duration or childhood maltreatment. We also did not observe any differentially methylated CpG sites between MDD and HC groups. After assessing functionality by correlating methylation levels with RNA expression of the respective genes, we observed that the number of functionally relevant CpG sites differed between MDD and HC groups in FKBP5 (χ² = 77.25, p < 0.0001) and NR3C1 (χ² = 7.29, p = 0.007). Cross-referencing functionally relevant CpG sites to those that were highly ranked in predicting HV in elastic net modeling identified one site from FKBP5 (cg03591753) and one from NR3C1 (cg20728768) within the MDD group. Stronger associations between DNA methylation, gene expression and HV in MDD suggest a novel putative molecular pathway of stress-related sensitivity in depression. Future studies should consider utilizing the epigenome and ultra-high field MR data which would allow the investigation of HV sub-fields.

Objectivity for the research worker

In the last decade, many problematic cases of scientific conduct have been diagnosed; some of which involve outright fraud (e.g., Stapel, 2012) others are more subtle (e.g., supposed evidence of extrasensory perception; Bem, 2011). These and similar problems can be interpreted as caused by lack of scientific objectivity. The current philosophical theories of objectivity do not provide scientists with conceptualizations that can be effectively put into practice in remedying these issues. We propose a novel way of thinking about objectivity for individual scientists; a negative and dynamic approach.We provide a philosophical conceptualization of objectivity that is informed by empirical research. In particular, it is our intention to take the first steps in providing an empirically and methodologically informed inventory of factors that impair the scientific practice. The inventory will be compiled into a negative conceptualization (i.e., what is not objective), which could in principle be used by individual scientists to assess (deviations from) objectivity of scientific practice. We propose a preliminary outline of a usable and testable instrument for indicating the objectivity of scientific practice.

Virology Laboratory Guidelines

Starting work in a virology research laboratory as a new technician, graduate student, or postdoc can be complex, intimidating, confusing, and stressful. From laboratory logistics to elemental expectations to scientific specifics, there is much to learn. To help new laboratory members adjust and excel, a series of guidelines for working and thriving in a virology laboratory is presented. While guidelines may be most helpful for new laboratory members, everyone, including principal investigators, is encouraged to use a set of published guidelines as a resource to maximize the time and efforts of all laboratory members. The topics covered here are safety, wellness, balance, teamwork, integrity, reading, research, writing, speaking, and timelines.

Optimisation of a gold nanoparticle-based aptasensor integrated with image processing for the colorimetric detection of acephate using response surface methodology

Acephate (Ac) is an organophosphate (OP) compound, which is able to inhibit the activity of acetylcholinesterase. Thus, the aim of this study was to optimize the detection of Ac using a thiolated acephate binding aptamer-citrate capped gold nanoparticle (TABA-Cit-AuNP) sensor that also incorporated an image processing technique. The effects of independent variables, such as the incubation period of TABA-Cit-AuNPs (3-24 h) for binding TABA to Cit-AuNPs, the concentration of phosphate buffer saline (PBS) (0.001-0.01 M), the concentration of thiolated acephate binding aptamer (TABA) (50-200 nM), and the concentration of magnesium sulphate (MgSO4) (1-300 mM) were investigated. A quadratic model was developed using a central composite design (CCD) from response surface methodology (RSM) to predict the sensing response to Ac. The optimum conditions such as the concentration of PBS (0.01 M), the concentration of TABA (200 nM), the incubation period of TABA-Cit-AuNPs (3 h), and the concentration of MgSO4 (1 mM) were used to produce a TABA-Cit-AuNPs sensor for the detection of Ac. Under optimal conditions, this sensor showed a detection ranging from 0.01 to 2.73 μM and a limit of detection (LOD) of 0.06 μM. Real sample analysis demonstrated this aptasensor as a good analytical method to detect Ac.

Teaching Computation in Neuroscience: Notes on the 2019 Society for Neuroscience Professional Development Workshop on Teaching

The 2019 Society for Neuroscience Professional Development Workshop on Teaching reviewed current tools, approaches, and examples for teaching computation in neuroscience. Robert Kass described the statistical foundations that students need to properly analyze data. Pascal Wallisch compared MATLAB and Python as programming languages for teaching students. Adrienne Fairhall discussed computational methods, training opportunities, and curricular considerations. Walt Babiec provided a view from the trenches on practical aspects of teaching computational neuroscience. Mathew Abrams concluded the session with an overview of resources for teaching and learning computational modeling in neuroscience.

Violating the normality assumption may be the lesser of two evils

When data are not normally distributed, researchers are often uncertain whether it is legitimate to use tests that assume Gaussian errors, or whether one has to either model a more specific error structure or use randomization techniques. Here we use Monte Carlo simulations to explore the pros and cons of fitting Gaussian models to non-normal data in terms of risk of type I error, power and utility for parameter estimation. We find that Gaussian models are robust to non-normality over a wide range of conditions, meaning that p values remain fairly reliable except for data with influential outliers judged at strict alpha levels. Gaussian models also performed well in terms of power across all simulated scenarios. Parameter estimates were mostly unbiased and precise except if sample sizes were small or the distribution of the predictor was highly skewed. Transformation of data before analysis is often advisable and visual inspection for outliers and heteroscedasticity is important for assessment. In strong contrast, some non-Gaussian models and randomization techniques bear a range of risks that are often insufficiently known. High rates of false-positive conclusions can arise for instance when overdispersion in count data is not controlled appropriately or when randomization procedures ignore existing non-independencies in the data. Hence, newly developed statistical methods not only bring new opportunities, but they can also pose new threats to reliability. We argue that violating the normality assumption bears risks that are limited and manageable, while several more sophisticated approaches are relatively error prone and particularly difficult to check during peer review. Scientists and reviewers who are not fully aware of the risks might benefit from preferentially trusting Gaussian mixed models in which random effects account for non-independencies in the data.

A Novel Individual Mentored Methodology to Peer Review for Residents/Fellows

Individualized guidance and assistance with constructive criticism as a mentored activity to peer review an article helps instill required rudiments, eliminate bad habits, and is shown to be beneficial to all participants. The Society of Laparoscopic & Robotic Surgeons initiated the R/F article mentoring review opportunity in 2014. The intimacy of actively debated discourse allows exposure to various peer review techniques and debate in tandem with education regarding the merits and faults of an article’s hypothesis and conclusions, and how they are evaluated for publication and responses to authors. The benefits of coaching reassessment of ideas, critical analysis, airing of disparate viewpoints; and the need to update, reinforce, and relearn science is not static and is more robust using this method.

Incorporating professional recommendations into a graduate-level statistical consulting laboratory: A case study

Introduction:

There has been a recent trend in medical research towards a more collaborative relationship between statisticians and clinical investigators. This has led to an increased focus on the most efficient and effective ways to structure, conduct, and measure the impact of organizations that provide statistical services to clinical investigators. Several recent guidelines and recommendations on the conduct of statistical consulting services(SCSs) have been made in response to this need, focusing on larger SCSs consisting primarily of faculty and staff statisticians. However, the application of these recommendations to consulting services primarily staffed by graduate students, which have the dual role of providing a professional service and training, remains unclear.

Methods:

Guidelines and recommendations, primarily from the Clinical and Translational Science (CTSA) consortium, were applied to a SCS staffed primarily by graduate students in an academic health center. A description of the organizational structure and outcomes after 3 years of operation is presented.

Results:

The guidelines recommended by the CTSA consortium and other groups were successfully incorporated into the graduate consulting laboratory. At almost one new project request per week, the consulting laboratory demonstrated a large bandwidth and had an excellent feedback from investigators.

Conclusions:

Guidelines developed for larger statistical consulting organizations are able to be applied in student-led consultation organizations. Outcomes and recommendations from 3.5 years of operation are used to describe the successes and challenges we have encountered.

Statistical inference in ensemble modeling of cellular metabolism

Kinetic models of metabolism can be constructed to predict cellular regulation and devise metabolic engineering strategies, and various promising computational workflows have been developed in recent years for this. Due to the uncertainty in the kinetic parameter values required to build kinetic models, these workflows rely on ensemble modeling (EM) principles for sampling and building populations of models describing observed physiologies. Sensitivity coefficients from metabolic control analysis (MCA) of kinetic models can provide important insight about cellular control around a given physiological steady state. However, despite considering populations of kinetic models and their model outputs, current approaches do not provide adequate tools for statistical inference. To derive conclusions from model outputs, such as MCA sensitivity coefficients, it is necessary to rank/compare populations of variables with each other. Currently existing workflows consider confidence intervals (CIs) that are derived independently for each comparable variable. Hence, it is important to derive simultaneous CIs for the variables that we wish to rank/compare. Herein, we used an existing large-scale kinetic model of Escherichia Coli metabolism to present how univariate CIs can lead to incorrect conclusions, and we present a new workflow that applies three different multivariate statistical approaches. We use the Bonferroni and the exact normal methods to build symmetric CIs using the normality assumptions. We then suggest how bootstrapping can compute asymmetric CIs whilst relaxing this normality assumption. We conclude that the Bonferroni and the exact normal methods can provide simple and efficient ways for constructing reliable CIs, with the exact normal method favored over the Bonferroni when the compared variables present dependencies. Bootstrapping, despite its significantly higher computational cost, is recommended when comparing non-normal distributions of variables. Additionally, we show how the Bonferroni method can readily be used to estimate required sample numbers to attain a certain CI size.

Seven Easy Steps to Open Science

Abstract. The open science movement is rapidly changing the scientific landscape. Because exact definitions are often lacking and reforms are constantly evolving, accessible guides to open science are needed. This paper provides an introduction to open science and related reforms in the form of an annotated reading list of seven peer-reviewed articles, following the format of Etz, Gronau, Dablander, Edelsbrunner, and Baribault (2018) . Written for researchers and students – particularly in psychological science – it highlights and introduces seven topics: understanding open science; open access; open data, materials, and code; reproducible analyses; preregistration and registered reports; replication research; and teaching open science. For each topic, we provide a detailed summary of one particularly informative and actionable article and suggest several further resources. Supporting a broader understanding of open science issues, this overview should enable researchers to engage with, improve, and implement current open, transparent, reproducible, replicable, and cumulative scientific practices.

Corrective Muscle Activity Reveals Subject-Specific Sensorimotor Recalibration

Recent studies suggest that planned and corrective actions are recalibrated during some forms of motor adaptation. However, corrective (also known as reactive) movements in human locomotion are thought to simply reflect sudden environmental changes independently from sensorimotor recalibration. Thus, we asked whether corrective responses can indicate the motor system’s adapted state following prolonged exposure to a novel walking situation inducing sensorimotor adaptation. We recorded electromyographic (EMG) signals bilaterally on 15 leg muscles before, during, and after split-belts walking (i.e., novel walking situation), in which the legs move at different speeds. We exploited the rapid temporal dynamics of corrective responses upon unexpected speed transitions to isolate them from the overall motor output. We found that corrective muscle activity was structurally different following short versus long exposures to split-belts walking. Only after a long exposure, removal of the novel environment elicited corrective muscle patterns that matched those expected in response to a perturbation opposite to the one originally experienced. This indicated that individuals who recalibrated their motor system adopted split-belts environment as their new “normal” and transitioning back to the original walking environment causes subjects to react as if it was novel to them. Interestingly, this learning declined with age, but steady state modulation of muscle activity during split-belts walking did not, suggesting potentially different neural mechanisms underlying these motor patterns. Taken together, our results show that corrective motor commands reflect the adapted state of the motor system, which is less flexible as we age.

A Serious Flaw in Nutrition Epidemiology: A Meta-Analysis Study

Background Many researchers have studied the relationship between diet and health. Specifically, there are papers showing an association between the consumption of sugar sweetened beverages and Type 2 diabetes. Many meta-analyses use individual studies that do not attempt to adjust for multiple testing or multiple modeling. Hence the claims reported in a meta-analysis paper may be unreliable as the base papers do not ensure unbiased statistics. Objective Determine (i) the statistical reliability of 10 papers and (ii) indirectly the reliability of the meta-analysis study. Method We obtained copies of each of the 10 papers used in a metaanalysis paper and counted the numbers of outcomes, predictors, and covariates. We estimate the size of the potential analysis search space available to the authors of these papers; i. e. the number of comparisons and models available. The potential analysis search space is the number of outcomes times the number of predictors times 2 c , where c is the number of covariates. This formula was applied to information found in the abstracts (Space A) as well as the text (Space T) of each base paper. Results The median and range of the number of comparisons possible across the base papers are 6.5 and (2 12,288), respectively for Space A, and 196,608 and (3072-117,117,952), respectively for Space T. It is noted that the median of 6.5 for Space A may be misleading as each study has 60-165 foods that could be predictors. Conclusion Given that testing is at the 5% level and the number of comparisons is very large, nominal statistical significance is very weak support for a claim. The claims in these papers are not statistically supported and hence are unreliable so the meta-analysis paper is also unreliable.

Ten quick tips for getting the most scientific value out of numerical data

Most studies in the life sciences and other disciplines involve generating and analyzing numerical data of some type as the foundation for scientific findings. Working with numerical data involves multiple challenges. These include reproducible data acquisition, appropriate data storage, computationally correct data analysis, appropriate reporting and presentation of the results, and suitable data interpretation. Finding and correcting mistakes when analyzing and interpreting data can be frustrating and time-consuming. Presenting or publishing incorrect results is embarrassing but not uncommon. Particular sources of errors are inappropriate use of statistical methods and incorrect interpretation of data by software. To detect mistakes as early as possible, one should frequently check intermediate and final results for plausibility. Clearly documenting how quantities and results were obtained facilitates correcting mistakes. Properly understanding data is indispensable for reaching well-founded conclusions from experimental results. Units are needed to make sense of numbers, and uncertainty should be estimated to know how meaningful results are. Descriptive statistics and significance testing are useful tools for interpreting numerical results if applied correctly. However, blindly trusting in computed numbers can also be misleading, so it is worth thinking about how data should be summarized quantitatively to properly answer the question at hand. Finally, a suitable form of presentation is needed so that the data can properly support the interpretation and findings. By additionally sharing the relevant data, others can access, understand, and ultimately make use of the results. These quick tips are intended to provide guidelines for correctly interpreting, efficiently analyzing, and presenting numerical data in a useful way.

A brief introduction to mixed effects modelling and multi-model inference in ecology

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues regarding methods of model selection, with particular reference to the use of information theory and multi-model inference in ecology. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.

Reproducibility2020: Progress and priorities

The preclinical research process is a cycle of idea generation, experimentation, and reporting of results. The biomedical research community relies on the reproducibility of published discoveries to create new lines of research and to translate research findings into therapeutic applications. Since 2012, when scientists from Amgen reported that they were able to reproduce only 6 of 53 "landmark" preclinical studies, the biomedical research community began discussing the scale of the reproducibility problem and developing initiatives to address critical challenges. Global Biological Standards Institute (GBSI) released the "Case for Standards" in 2013, one of the first comprehensive reports to address the rising concern of irreproducible biomedical research. Further attention was drawn to issues that limit scientific self-correction, including reporting and publication bias, underpowered studies, lack of open access to methods and data, and lack of clearly defined standards and guidelines in areas such as reagent validation. To evaluate the progress made towards reproducibility since 2013, GBSI identified and examined initiatives designed to advance quality and reproducibility. Through this process, we identified key roles for funders, journals, researchers and other stakeholders and recommended actions for future progress. This paper describes our findings and conclusions.

Big Data Science: Opportunities and Challenges to Address Minority Health and Health Disparities in the 21st Century

Addressing minority health and health disparities has been a missing piece of the puzzle in Big Data science. This article focuses on three priority opportunities that Big Data science may offer to the reduction of health and health care disparities. One opportunity is to incorporate standardized information on demographic and social determinants in electronic health records in order to target ways to improve quality of care for the most disadvantaged populations over time. A second opportunity is to enhance public health surveillance by linking geographical variables and social determinants of health for geographically defined populations to clinical data and health outcomes. Third and most importantly, Big Data science may lead to a better understanding of the etiology of health disparities and understanding of minority health in order to guide intervention development. However, the promise of Big Data needs to be considered in light of significant challenges that threaten to widen health disparities. Care must be taken to incorporate diverse populations to realize the potential benefits. Specific recommendations include investing in data collection on small sample populations, building a diverse workforce pipeline for data science, actively seeking to reduce digital divides, developing novel ways to assure digital data privacy for small populations, and promoting widespread data sharing to benefit under-resourced minority-serving institutions and minority researchers. With deliberate efforts, Big Data presents a dramatic opportunity for reducing health disparities but without active engagement, it risks further widening them.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

A manifesto for reproducible science

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

Heading for new shores! Overcoming pitfalls in BCI design

Research in brain-computer interfaces has achieved impressive progress towards implementing assistive technologies for restoration or substitution of lost motor capabilities, as well as supporting technologies for able-bodied subjects. Notwithstanding this progress, effective translation of these interfaces from proof-of concept prototypes into reliable applications remains elusive. As a matter of fact, most of the current BCI systems cannot be used independently for long periods of time by their intended end-users. Multiple factors that impair achieving this goal have already been identified. However, it is not clear how do they affect the overall BCI performance or how they should be tackled. This is worsened by the publication bias where only positive results are disseminated, preventing the research community from learning from its errors. This paper is the result of a workshop held at the 6th International BCI meeting in Asilomar. We summarize here the discussion on concrete research avenues and guidelines that may help overcoming common pitfalls and make BCIs become a useful alternative communication device.