Mapping strategies towards improved external validity in preclinical translational research

INTRODUCTION

Translation is about successfully bringing findings from preclinical contexts into the clinic. This transfer is challenging as clinical trials frequently fail despite positive preclinical results. Limited robustness of preclinical research has been marked as one of the drivers of such failures. One suggested solution is to improve the external validity of in vitro and in vivo experiments via a suite of complementary strategies.

AREAS COVERED

In this review, the authors summarize the literature available on different strategies to improve external validity in in vivo, in vitro, or ex vivo experiments; systematic heterogenization; generalizability tests; and multi-batch and multicenter experiments. Articles that tested or discussed sources of variability in systematically heterogenized experiments were identified, and the most prevalent sources of variability are reviewed further. Special considerations in sample size planning, analysis options, and practical feasibility associated with each strategy are also reviewed.

EXPERT OPINION

The strategies reviewed differentially influence variation in experiments. Different research projects, with their unique goals, can leverage the strengths and limitations of each strategy. Applying a combination of these approaches in confirmatory stages of preclinical research putatively increases the chances of success in clinical studies.

Statistical review of animal trials-A guideline

Any experiment involving living organisms requires justification of the need and moral defensibleness of the study. Statistical planning, design, and sample size calculation of the experiment are no less important review criteria than general medical and ethical points to consider. Errors made in the statistical planning and data evaluation phase can have severe consequences on both results and conclusions. They might proliferate and thus impact future trials-an unintended outcome of fundamental research with profound ethical consequences. Unified statistical standards are currently missing for animal review boards in Germany. In order to accompany, we developed a biometric form to be filled and handed in with the proposal at the concerned local authority on animal welfare. It addresses relevant points to consider for biostatistical planning of animal experiments and can help both the applicants and the reviewers in overseeing the entire experiment(s) planned. Furthermore, the form might also aid in meeting the current standards set by the 3+3R's principle of animal experimentation: Replacement, Reduction, Refinement as well as Robustness, Registration, and Reporting. The form has already been in use by the concerned local authority of animal welfare in Berlin, Germany. In addition, we provide reference to our user guide giving more detailed explanation and examples for each section of the biometric form. Unifying the set of biostatistical aspects will help both the applicants and the reviewers to equal standards and increase quality of preclinical research projects, also for translational, multicenter, or international studies.

Mixed-species groups in bats: non-random roost associations and roost selection in neotropical understory bats

BACKGROUND

Mixed-species groups in animals have been shown to confer antipredator, foraging and other benefits to their members that may provide selective advantages. In most cases, however, it is unclear whether functional benefits are a principal driver of heterospecific groups, or whether groups simply result from simultaneous exploitation of common resources. Mixed-species groups that form independently of environmental conditions may, however, evidence direct benefits of species associations. Bats are among the most gregarious mammals, with sometimes thousands of individuals of various species roosting communally. Despite numerous potential functional benefits of such mixed-species roosting groups, interspecific attraction has never been shown. To explore alternative explanations for mixed-species roosting, we studied roost selection in a speciose neotropical understory bat community in lowland rainforest in Costa Rica. Long term roost data were recorded over 10 years in a total of 133 roosts comprising both natural roosts and structurally uniform artificial roosts. We modelled bat roost occupancy and abundance in each roost type and in forest and pasture habitats to quantify the effects of roost- and environmental variability.

RESULTS

We found that bat species presence in natural roosts is predictable from habitat and structural roost parameters, but that the presence and abundance of other bat species further modifies roost choice. One third of the 12 study species were found to actively associate with selected other bat species in roosts (e.g. Glossophaga commissarisi with Carollia sowelli). Other species did not engage in communal roosting, which in some cases indicates a role for negative interspecific interactions, such as roost competition.

CONCLUSIONS

Mixed-species roosting may provide thermoregulatory benefits, reduce intraspecific competition and promote interspecific information transfer, and hence some heterospecific associations may be selected for in bats. Overall, our study contributes to an improved understanding of the array of factors that shape diverse tropical bat communities and drive the dynamics of heterospecific grouping in mammals more generally.

Improving preclinical studies through replications

The purpose of preclinical research is to inform the development of novel diagnostics or therapeutics, and the results of experiments on animal models of disease often inform the decision to conduct studies in humans. However, a substantial number of clinical trials fail, even when preclinical studies have apparently demonstrated the efficacy of a given intervention. A number of large-scale replication studies are currently trying to identify the factors that influence the robustness of preclinical research. Here, we discuss replications in the context of preclinical research trajectories, and argue that increasing validity should be a priority when selecting experiments to replicate and when performing the replication. We conclude that systematically improving three domains of validity - internal, external and translational - will result in a more efficient allocation of resources, will be more ethical, and will ultimately increase the chances of successful translation.

Assessing the Organizational Climate for Translational Research with a New Survey Tool

Promoting translational research as a means to overcoming chasms in the translation of knowledge through successive fields of research from basic science to public health impacts and back is a central challenge for research managers and policymakers. Organizational leaders need to assess baseline conditions, identify areas needing improvement, and to judge the impact of specific initiatives to sustain or improve translational research practices at their institutions. Currently, there is a lack of such an assessment tool addressing the specific context of translational biomedical research. To close this gap, we have developed a new survey for assessing the organizational climate for translational research. This self-assessment tool measures employees' perceptions of translational research climate and underlying research practices in organizational environments and builds on the established Survey of Organizational Research Climate, assessing research integrity. Using this tool, we show that scientists at a large university hospital (Charité Berlin) perceive translation as a central and important component of their work. Importantly, local resources and direct support are main contributing factors for the practical implementation of translation into their own research practice. We identify and discuss potential leverage points for an improvement of research climate to foster successful translational research.

Judgments of effort exerted by others are influenced by received rewards

Estimating invested effort is a core dimension for evaluating own and others’ actions, and views on the relationship between effort and rewards are deeply ingrained in various societal attitudes. Internal representations of effort, however, are inherently noisy, e.g. due to the variability of sensorimotor and visceral responses to physical exertion. The uncertainty in effort judgments is further aggravated when there is no direct access to the internal representations of exertion – such as when estimating the effort of another person. Bayesian cue integration suggests that this uncertainty can be resolved by incorporating additional cues that are predictive of effort, e.g. received rewards. We hypothesized that judgments about the effort spent on a task will be influenced by the magnitude of received rewards. Additionally, we surmised that such influence might further depend on individual beliefs regarding the relationship between hard work and prosperity, as exemplified by a conservative work ethic. To test these predictions, participants performed an effortful task interleaved with a partner and were informed about the obtained reward before rating either their own or the partner’s effort. We show that higher rewards led to higher estimations of exerted effort in self-judgments, and this effect was even more pronounced for other-judgments. In both types of judgment, computational modelling revealed that reward information and sensorimotor markers of exertion were combined in a Bayes-optimal manner in order to reduce uncertainty. Remarkably, the extent to which rewards influenced effort judgments was associated with conservative world-views, indicating links between this phenomenon and general beliefs about the relationship between effort and earnings in society.

Digital open science-Teaching digital tools for reproducible and transparent research

An important hallmark of science is the transparency and reproducibility of scientific results. Over the last few years, internet-based technologies have emerged that allow for a representation of the scientific process that goes far beyond traditional methods and analysis descriptions. Using these often freely available tools requires a suite of skills that is not necessarily part of a curriculum in the life sciences. However, funders, journals, and policy makers increasingly require researchers to ensure complete reproducibility of their methods and analyses. To close this gap, we designed an introductory course that guides students towards a reproducible science workflow. Here, we outline the course content and possible extensions, report encountered challenges, and discuss how to integrate such a course in existing curricula.

Norm compliance affects perceptual decisions through modulation of a starting point bias

Adaptive decisions in social contexts depend on both perceptual information and social expectations or norms. These are potentially in conflict when certain choices are beneficial for an individual, but societal rules mandate a different course of action. To resolve such a conflict, the reliability of information has to be balanced against potentially deleterious effects of non-compliance such as ostracism. In this study, we systematically investigated how interactions between perceptual and social influences affect decision-relevant cognitive processes. In a direction-of-motion discrimination task, participants received perceptual information alongside information on other players' choices. In addition, we created conflict scenarios where players' choices affected other participants' monetary rewards dependent on whether their choices were in line or against the opinion of the other players. Importantly, we altered the strength of this manipulation in two separate experiments by contrasting motivations of either preventing harm or providing a benefit to others. Behavioural analyses and computational models of perceptual decisions showed that participants successfully integrated perceptual with social information. Participants' reliance on social information was effectively modulated in conflict situations. Critically, these effects were augmented when the strength of social norms was increased, indexing conditions under which social norms effectively influence decisions. These results inform theories of social influence by providing an account of how higher order goals like social norm compliance affect perceptual decisions.

Neural substrates of norm compliance in perceptual decisions

Societal norms exert a powerful influence on our decisions. Behaviours motivated by norms, however, do not always concur with the responses mandated by decision relevant information potentially generating a conflict. To probe the interplay between normative and informational influences, we examined how prosocial norms impact on perceptual decisions subjects made in the context of a simultaneous presentation of social information. Participants displayed a bias in their perceptual decisions towards that mandated by social information. However, normative prescriptions modulated this bias bi-directionally depending on whether norms mandated a decision in accord or contrary to the contextual social information. At a neural level, the addition of a norms increased activity in prefrontal cortex and modulated functional connectivity between prefrontal and parietal areas. The bi-directional effect of our norms was captured by differential activations when participants decided against the social information. When norms indicated a decision in line with social information, non-compliance modulated lateral prefrontal cortex activity. By contrast, when norms mandated a decision against social information norm compliance increased activity in the anterior cingulate cortex. Hence, social norms changed the balance between a reliance on perceptual and social information by modulating brain activity in regions associated with response inhibition and conflict monitoring.

Individual consistency and flexibility in human social information use

Copying others appears to be a cost-effective way of obtaining adaptive information, particularly when flexibly employed. However, adult humans differ considerably in their propensity to use information from others, even when this 'social information' is beneficial, raising the possibility that stable individual differences constrain flexibility in social information use. We used two dissimilar decision-making computer games to investigate whether individuals flexibly adjusted their use of social information to current conditions or whether they valued social information similarly in both games. Participants also completed established personality questionnaires. We found that participants demonstrated considerable flexibility, adjusting social information use to current conditions. In particular, individuals employed a 'copy-when-uncertain' social learning strategy, supporting a core, but untested, assumption of influential theoretical models of cultural transmission. Moreover, participants adjusted the amount invested in their decision based on the perceived reliability of personally gathered information combined with the available social information. However, despite this strategic flexibility, participants also exhibited consistent individual differences in their propensities to use and value social information. Moreover, individuals who favoured social information self-reported as more collectivist than others. We discuss the implications of our results for social information use and cultural transmission.

The neural underpinnings of an optimal exploitation of social information under uncertainty

Social information influences decision-making through an integration of information derived from individual experience with that derived from observing the actions of others. This raises the question as to which extent one should utilize social information. One strategy is to make use of uncertainty estimates, leading to a copy-when-uncertain strategy that weights information from individual and social sources based on their respective reliabilities. Here, we investigate this integration process by extending models of Bayes optimal integration of sensory information to a social decision context. We then use a key parameter of our behavioral model in conjunction with functional magnetic resonance imaging to identify the neural substrate that is specifically linked to the fidelity of this integration process. We show that individuals behave near Bayes optimal when integrating two distinct sources of social information but systematically deviate from Bayes optimal choice when integrating individual with social information. This systematic behavioral deviation from optimality is linked to activity of left inferior frontal gyrus. Thus, an ability to optimally exploit social information depends on processes that overcome an egocentric bias, and this regulatory role involves the left inferior prefrontal cortex. The findings provide a mechanistic explanation for observations wherein individuals neglect the benefits from exploiting social information.

Interval timing behavior in Pallas's long-tongued bat (Glossophaga soricina)

Timing behavior in animals and its underlying mechanisms have been investigated extensively in the peak procedure, a variant of fixed interval procedures. In such experiments, individuals typically start responding with high frequency after an initial inactive time interval and continue their responses after peak time if rewards are omitted. This begs the so far unexplored question as to how timing behavior is influenced when such continuous responses are suppressed. Here, we present results from a nectar-feeding bat species, Glossophaga soricina, that was tested in a modified version of the peak procedure at three fixed time intervals (5 s, 11 s, 20 s). In contrast to standard peak procedures we imposed metabolic costs on individual responses which effectively suppressed trains of rapid responses during trials. Under this manipulation, bats' aggregated responses showed clear peaks around the peak time in the 5-s and 11-s schedules. Bats' responses in the 20-s schedule, however, did not peak around the fixed interval time. Crucially, an analysis of time intervals between successive revisits in all schedules revealed that bats revisited feeders at accurately timed intervals in all three conditions. The individual within trial behavioral responses showed clear oscillatory patterns throughout nonrewarded trials. These findings follow predictions from mechanistic timing models, like the striatal beat frequency model, and are discussed with regard to these models.

Social performance cues induce behavioral flexibility in humans

Behavioral flexibility allows individuals to react to environmental changes, but changing established behavior carries costs, with unknown benefits. Individuals may thus modify their behavioral flexibility according to the prevailing circumstances. Social information provided by the performance level of others provides one possible cue to assess the potential benefits of changing behavior, since out-performance in similar circumstances indicates that novel behaviors (innovations) are potentially useful. We demonstrate that social performance cues, in the form of previous players' scores in a problem-solving computer game, influence behavioral flexibility. Participants viewed only performance indicators, not the innovative behavior of others. While performance cues (high, low, or no scores) had little effect on innovation discovery rates, participants that viewed high scores increased their utilization of innovations, allowing them to exploit the virtual environment more effectively than players viewing low or no scores. Perceived conspecific performance can thus shape human decisions to adopt novel traits, even when the traits employed cannot be copied. This simple mechanism, social performance feedback, could be a driver of both the facultative adoption of innovations and cumulative cultural evolution, processes critical to human success.

Psychometric function for nectar volume perception of a flower-visiting bat

For foraging pollinators one aspect of floral quality is the volume of nectar available. Thus, nectar-feeding animals should be able to estimate volumes of received nectar. In this study, we determined the psychometric function for nectar volume discrimination of a Neotropical flower-visiting bat Glossophaga soricina. For this, we examined the ability of bats to discriminate between two nectar volumes in a two-alternative forced-choice paradigm. We used a Bayesian inference approach to determine psychometric functions. From the derived psychometric function we assessed the discrimination threshold at a value of 0.69. G. soricina could clearly distinguish between two volumes, when the difference between the two nectar volumes divided by their average exceeded this value. This indicates that bats possess a sense for the perception and discrimination of volumes of nectar that is better developed than in honeybees.