Funding priorities and health outcomes in Danish medical research

External research funding is an essential component of the infrastructure of modern, academic research. Priorities in funding decisions drive what knowledge is generated, and how scientists' careers are shaped. For health research, it can ultimately have implications for health outcomes. The aim of this paper is to illustrate how funding information can be used to track priorities in health research, linking them to disease burdens and research outputs. Furthermore, funding concentrations are analysed from both researcher and disease perspectives, to estimate the influence of personal Matthew-effects on the distribution of health research funding. Denmark is used as the case, including funding information from all major public and private research foundations in the period 2004-2016. Grant information is linked to research outputs and disability-adjusted life-years (DALY rates), for 34,160 publications linked to 2630 grants, receiving DKK 4.8 billion in funding. Data show poor correlation between funding priorities, research activity and disease burdens, with several diseases receiving disproportionate amounts of funding. A research opportunity index is calculated to identify diseases with the highest potential for future investments from a burden-centred point of view. Funding is highly concentrated, both on people and on specific diseases. High funding concentrations on researchers can be a driving factor behind the observed funding-to-burden imbalances, and may risk knowledge stagnation through monopolisation of the market place of ideas. Results indicate that funders of clinical and translational research, as well as some types of biomedical research, need to supplement traditional considerations of scientific excellence with measures of societal challenges and relevance.

Questionable research practices in competitive grant funding: A survey

There has been a surge of interest in research integrity over the last decade, with a wide range of studies investigating the prevalence of questionable research practices (QRPs). However, nearly all these studies focus on research design, data collection and analysis, and hardly any empirical research has been done on the occurrence of QRPs in the context of research funding. To fill this gap, we conducted a cross-sectional pre-registered survey of applicants, reviewers and panel members from the Research Foundation-Flanders (FWO), one of the main funding agencies in Belgium. We developed a bespoke survey and further refined it through feedback from experienced researchers and a pilot study. We asked how often respondents had engaged in a series of QRPs over the last ten years. A total of 1748 emails were sent, inviting recipients to participate in the survey, complemented by featuring the survey in the FWO newsletter. This resulted in 704 complete responses. Our results indicate that such QRPs are remarkably prevalent. Of the 496 participants who answered both the applicant and reviewer track, more than 60% responded that they engaged regularly in at least one of such practices, and around 40% indicated that they engaged at least occasionally in half of the QRPs queried. Only 12% reported not to have engaged in any of the QRPs. Contrary to our hypotheses, male respondents did not self-report to engage in the QRPs more often than female respondents, nor was there an association between the prevalence of QRPs and self-reported success rate in grant funding. Furthermore, half of the respondents indicated that they doubted the reliability of the grant peer review process more often than not. These results suggest that preventive action is needed, and provide new reasons to reconsider the practice of allocating research money through grant peer review.

Peer review: Risk and risk tolerance

Peer review, commonly used in grant funding decisions, relies on scientists’ ability to evaluate research proposals’ quality. Such judgments are sometimes beyond reviewers’ discriminatory power and could lead to a reliance on subjective biases, including preferences for lower risk, incremental projects. However, peer reviewers’ risk tolerance has not been well studied. We conducted a cross-sectional experiment of peer reviewers’ evaluations of mock primary reviewers’ comments in which the level and sources of risks and weaknesses were manipulated. Here we show that proposal risks more strongly predicted reviewers’ scores than proposal strengths based on mock proposal evaluations. Risk tolerance was not predictive of scores but reviewer scoring leniency was predictive of overall and criteria scores. The evaluation of risks dominates reviewers’ evaluation of research proposals and is a source of inter-reviewer variability. These results suggest that reviewer scoring variability may be attributed to the interpretation of proposal risks, and could benefit from intervention to improve the reliability of reviews. Additionally, the valuation of risk drives proposal evaluations and may reduce the chances that risky, but highly impactful science, is supported.