NIH funding and the pursuit of edge science

Why are clinical trials of deep brain stimulation terminated? An analysis of clinicaltrials.gov

Background

Although deep brain stimulation (DBS) has established uses for patients with movement disorders and epilepsy, it is under consideration for a wide range of neurologic and neuropsychiatric conditions.

Objective

To review successful and unsuccessful DBS clinical trials and identify factors associated with early trial termination.

Methods

The ClinicalTrials.gov database was screened for all studies related to DBS. Information regarding condition of interest, study aim, trial design, trial success, and, if applicable, reason for failure was collected. Trials were compared and logistic regression was utilized to identify independent factors associated with trial termination.

Results

Of 325 identified trials, 79.7% were successful and 20.3% unsuccessful. Patient recruitment, sponsor decision, and device issues were the most cited reasons for termination. 242 trials (74.5%) were interventional with 78.1% successful. There was a statistically significant difference between successful and unsuccessful trials in number of funding sources (p = 0.0375). NIH funding was associated with successful trials while utilization of other funding sources (academic institutions and community organizations) was associated with unsuccessful trials. 83 trials (25.5%) were observational with 84.0% successful; there were no statistically significant differences between successful and unsuccessful observational trials.

Conclusion

One in five clinical trials for DBS were found to be unsuccessful, most commonly due to patient recruitment difficulties. The source of funding was the only factor associated with trial success. As DBS research continues to grow, understanding the current state of clinical trials will help design successful future studies, thereby minimizing futile expenditures of time, cost, and patient engagement.

The rise of automated curiosity-driven discoveries in chemistry

The quest for generating novel chemistry knowledge is critical in scientific advancement, and machine learning (ML) has emerged as an asset in this pursuit. Through interpolation among learned patterns, ML can tackle tasks that were previously deemed demanding to machines. This distinctive capacity of ML provides invaluable aid to bench chemists in their daily work. However, current ML tools are typically designed to prioritize experiments with the highest likelihood of success, i.e., higher predictive confidence. In this perspective, we build on current trends that suggest a future in which ML could be just as beneficial in exploring uncharted search spaces through simulated curiosity. We discuss how low and 'negative' data can catalyse one-/few-shot learning, and how the broader use of curious ML and novelty detection algorithms can propel the next wave of chemical discoveries. We anticipate that ML for curiosity-driven research will help the community overcome potentially biased assumptions and uncover unexpected findings in the chemical sciences at an accelerated pace.

SciSciNet: A large-scale open data lake for the science of science research

The science of science has attracted growing research interests, partly due to the increasing availability of large-scale datasets capturing the innerworkings of science. These datasets, and the numerous linkages among them, enable researchers to ask a range of fascinating questions about how science works and where innovation occurs. Yet as datasets grow, it becomes increasingly difficult to track available sources and linkages across datasets. Here we present SciSciNet, a large-scale open data lake for the science of science research, covering over 134M scientific publications and millions of external linkages to funding and public uses. We offer detailed documentation of pre-processing steps and analytical choices in constructing the data lake. We further supplement the data lake by computing frequently used measures in the literature, illustrating how researchers may contribute collectively to enriching the data lake. Overall, this data lake serves as an initial but useful resource for the field, by lowering the barrier to entry, reducing duplication of efforts in data processing and measurements, improving the robustness and replicability of empirical claims, and broadening the diversity and representation of ideas in the field.

Data, measurement and empirical methods in the science of science

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding 'science of science'. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field's diverse methodologies and expand researchers' toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

NSF Fellows' perceptions about incentives, research misconduct, and scientific integrity in STEM academia

There is increased concern about perverse incentives, quantitative performance metrics, and hyper-competition for funding and faculty positions in US academia. Recipients of the prestigious National Science Foundation Graduate Research Fellowships (n = 244) from Civil and Environmental Engineering (45.5%) and Computer Science and Engineering (54.5%) were anonymously surveyed to create a baseline snapshot of their perceptions, behaviors and experiences. NSF Fellows ranked scientific advancement as the top metric for evaluating academics followed by publishing in high-impact journals, social impact of research, and publication/citation counts. The self-reported rate of academic cheating was 16.7% and of research misconduct was 3.7%. Thirty-one percent of fellows reported direct knowledge of graduate peers cheating, and 11.9% had knowledge of research misconduct by colleagues. Only 30.7% said they would report suspected misconduct. A majority of fellows (55.3%) felt that mandatory ethics trainings left them unprepared for dealing with ethical issues. Fellows stated academic freedom, flexible schedules and opportunity to mentor students were the most positive aspects of academia, whereas pressures for funding, publication, and tenure were cited as the most negative aspects. These data may be useful in considering how to better prepare STEM graduate trainees for academic careers.

Resistance to Change

Advances in medical technology do not follow a smooth process and are highly variable. Implementation can occasionally be rapid, but often faces varying degrees of resistance resulting at the very least in delayed implementation. Using qualitative comparative analysis, we have evaluated numerous technological advances from the perspective of how they were introduced, implemented, and opposed. Resistance varies from benign - often happening because of inertia or lack of resources to more active forms, including outright opposition using both appropriate and inappropriate methods to resist/delay changes in care. Today, even public health has become politicized, having nothing to do with the underlying science, but having catastrophic results. Two other corroding influences are marketing pressure from the private sector and vested interests in favor of one outcome or another. This also applies to governmental agencies. There are a number of ways in which papers have been buried including putting the thumb on the scale where reviewers can sabotage new ideas. Unless we learn to harness new technologies earlier in their life course and understand how to maneuver around the pillars of obstruction to their implementation, we will not be able to provide medical care at the forefront of technological capabilities.

Creativity as an antidote to research becoming too predictable

In this commentary, Sonne-Hansen and colleagues argue that research leaders and organizations should encourage more "theory-guessing" by budding young scientists, rather than incentivizing safe mainstream research.

Papers and patents are becoming less disruptive over time

Theories of scientific and technological change view discovery and invention as endogenous processes^1,2, wherein previous accumulated knowledge enables future progress by allowing researchers to, in Newton's words, 'stand on the shoulders of giants'^3-7. Recent decades have witnessed exponential growth in the volume of new scientific and technological knowledge, thereby creating conditions that should be ripe for major advances^8,9. Yet contrary to this view, studies suggest that progress is slowing in several major fields^10,11. Here, we analyse these claims at scale across six decades, using data on 45 million papers and 3.9 million patents from six large-scale datasets, together with a new quantitative metric-the CD index¹²-that characterizes how papers and patents change networks of citations in science and technology. We find that papers and patents are increasingly less likely to break with the past in ways that push science and technology in new directions. This pattern holds universally across fields and is robust across multiple different citation- and text-based metrics^1,13-17. Subsequently, we link this decline in disruptiveness to a narrowing in the use of previous knowledge, allowing us to reconcile the patterns we observe with the 'shoulders of giants' view. We find that the observed declines are unlikely to be driven by changes in the quality of published science, citation practices or field-specific factors. Overall, our results suggest that slowing rates of disruption may reflect a fundamental shift in the nature of science and technology.

Fund behavioral science like the frameworks we endorse: the case for increased funding of preliminary studies by the National Institutes of Health

Innovative, groundbreaking science relies upon preliminary studies (aka pilot, feasibility, proof-of-concept). In the behavioral sciences, almost every large-scale intervention is supported by a series of one or more rigorously conducted preliminary studies. The importance of preliminary studies was established by the National Institutes of Health (NIH) in 2014/2015 in two translational science frameworks (NIH Stage and ORBIT models). These frameworks outline the essential role preliminary studies play in developing the next generation of evidence-based behavioral prevention and treatment interventions. Data produced from preliminary studies are essential to secure funding from the NIH’s most widely used grant mechanism for large-scale clinical trials, namely the R01. Yet, despite their unquestionable importance, the resources available for behavioral scientists to conduct rigorous preliminary studies are limited. In this commentary, we discuss ways the existing funding structure at the NIH, despite its clear reliance upon high-quality preliminary studies, inadvertently discourages and disincentivizes their pursuit by systematically underfunding them. We outline how multiple complementary and pragmatic steps via a small reinvestment of funds from larger trials could result in a large increase in funding for smaller preliminary studies. We make the case such a reinvestment has the potential to increase innovative science, increase the number of investigators currently funded, and would yield lasting benefits for behavioral science and scientists alike.

Why citizen review might beat peer review at identifying pursuitworthy scientific research

In response to increasing fatigue with the failings of the peer review system, granting agencies are beginning to consider lotteries as an alternative. I argue that citizen review, in which non-scientists determine funding allocations, has advantages over both. This is particularly true when it comes to identifying which research is most pursuitworthy.

Trends of sources of clinical research funding from 1990 to 2020: a meta-epidemiological study

Evidence has raised concerns regarding the association between funding sources and doubtful data. Our main outcome was to analyze trends on funding sources in articles published from 1990 to 2020 in the more influential journals of internal and general medicine. In this meta-epidemiological study, we included peer-reviewed studies from the 10 highest impact journals in general and internal medicine published between January 1990 and February 2020 based on published original research according to the 2018 InCites Journal of Citation Reports, these consisted of the following: The New England Journal of Medicine, The Lancet, JAMA, BMJ, JAMA Internal Medicine, Annals of Internal Medicine, PLOS Medicine, Cachexia, BMC Medicine, and Mayo Clinic Proceedings Two reviewers working in duplicate extracted data regarding year of publication, study design, and sources of funding. In total, 496 articles were found; of these, 311 (62.7%) were observational studies, 167 (33.7%) were experimental, and 16 (3.2%) were secondary analyses. Percentages of grant sources through the years were predominantly from government (60%), industry (23.83%), and non-governmental (16.06%) organizations. The percentage of industry subsidies tended to decrease, but this was not significant in a linear regression model (r=0.02, p≥0.05). Government and non-government funding sources showed a trend to decrease in the same univariate analysis with both significant associations (r=0.21, p≤0.001 and r=0.10, p≤0.001, respectively). The main funding source in medical research has consistently been government aid. Despite previous reported data, no association was found between the source of funding and statistically significant results favoring study authors' hypothesis.

A model for cooperative scientific research inspired by the ant colony algorithm

Modern scientific research has become largely a cooperative activity in the Internet age. We build a simulation model to understand the population-level creativity based on the heuristic ant colony algorithm. Each researcher has two heuristic parameters characterizing the goodness of his own judgments and his trust on literature. We study how the distributions of contributor heuristic parameters change with the research problem scale, stage of the research problem, and computing power available. We also identify situations where path dependence and hasty research due to the pressure on productivity can significantly impede the long-term advancement of scientific research. Our work provides some preliminary understanding and guidance for the dynamical process of cooperative scientific research in various disciplines.

The characteristics of early-stage research into human genes are substantially different from subsequent research

Throughout the last 2 decades, several scholars observed that present day research into human genes rarely turns toward genes that had not already been extensively investigated in the past. Guided by hypotheses derived from studies of science and innovation, we present here a literature-wide data-driven meta-analysis to identify the specific scientific and organizational contexts that coincided with early-stage research into human genes throughout the past half century. We demonstrate that early-stage research into human genes differs in team size, citation impact, funding mechanisms, and publication outlet, but that generalized insights derived from studies of science and innovation only partially apply to early-stage research into human genes. Further, we demonstrate that, presently, genome biology accounts for most of the initial early-stage research, while subsequent early-stage research can engage other life sciences fields. We therefore anticipate that the specificity of our findings will enable scientists and policymakers to better promote early-stage research into human genes and increase overall innovation within the life sciences.

Análisis de la producción y redes de colaboración en los programas de doctorado en psicología en Colombia

El objetivo de la formación doctoral es la generación y difusión de nuevo conocimiento, sin embargo, no es claro cómo los programas de doctorado en Colombia se relacionan con este tipo de producción académica. A partir de ello, en el presente estudio se presenta el panorama general de la producción académica a través de las instituciones de educación superior colombianas que ofertan programas de formación doctoral en psicología. La producción académica se obtuvo de la base de datos Scopus en una muestra de 13 universidades. Tras una búsqueda inicial, se encontraron 1345 documentos, con los cuales se realizó un análisis de los indicadores de productividad y de las redes de colaboración al interior de la disciplina en el país. Posteriormente, esta información se contrastó con la información nacional registrada en el sistema de Currículum Vitae Latinoamericano y del Caribe (CvLAC) de 193 profesores asociados a las universidades que ofertan los programas de doctorado. Al comparar la producción académica nacional e internacional, se encontró una mayor correlación entre la dirección de tesis de posgrado y la publicación en revistas nacionales. Por lo tanto, se puede afirmar que el impacto de la formación doctoral en la generación de nuevo conocimiento es relativo y tiene un alcance limitado, siendo principalmente las revistas nacionales el medio de difusión de este conocimiento. Se recomienda evaluar el impacto de los productos asociados a la formación posgradual y la visibilidad internacional, con énfasis en las revistas indexadas en bases de datos internacionales.

Scientific evidence based rare disease research discovery with research funding data in knowledge graph

BACKGROUND

Limited knowledge and unclear underlying biology of many rare diseases pose significant challenges to patients, clinicians, and scientists. To address these challenges, there is an urgent need to inspire and encourage scientists to propose and pursue innovative research studies that aim to uncover the genetic and molecular causes of more rare diseases and ultimately to identify effective therapeutic solutions. A clear understanding of current research efforts, knowledge/research gaps, and funding patterns as scientific evidence is crucial to systematically accelerate the pace of research discovery in rare diseases, which is an overarching goal of this study.

METHODS

To semantically represent NIH funding data for rare diseases and advance its use of effectively promoting rare disease research, we identified NIH funded projects for rare diseases by mapping GARD diseases to the project based on project titles; subsequently we presented and managed those identified projects in a knowledge graph using Neo4j software, hosted at NCATS, based on a pre-defined data model that captures semantics among the data. With this developed knowledge graph, we were able to perform several case studies to demonstrate scientific evidence generation for supporting rare disease research discovery.

RESULTS

Of 5001 rare diseases belonging to 32 distinct disease categories, we identified 1294 diseases that are mapped to 45,647 distinct, NIH-funded projects obtained from the NIH ExPORTER by implementing semantic annotation of project titles. To capture semantic relationships presenting amongst mapped research funding data, we defined a data model comprised of seven primary classes and corresponding object and data properties. A Neo4j knowledge graph based on this predefined data model has been developed, and we performed multiple case studies over this knowledge graph to demonstrate its use in directing and promoting rare disease research.

CONCLUSION

We developed an integrative knowledge graph with rare disease funding data and demonstrated its use as a source from where we can effectively identify and generate scientific evidence to support rare disease research. With the success of this preliminary study, we plan to implement advanced computational approaches for analyzing more funding related data, e.g., project abstracts and PubMed article abstracts, and linking to other types of biomedical data to perform more sophisticated research gap analysis and identify opportunities for future research in rare diseases.

National Institute on Aging seed funding enables Alzheimer's disease startups to reach key value inflection points

INTRODUCTION

The National Institute on Aging (NIA) provides funding to academic researchers and small businesses working in the Alzheimer's Disease (AD) and AD-related dementia (ADRD) fields to help commercialize their products. The NIA uses Small Business Innovation Research (SBIR) awards to bridge the funding gap in the diagnostic, therapeutic, and care interventions areas, enabling startups to reach key value inflection points to achieve scientific milestones.

METHODS

Only publicly available information is reported. The National Institutes of Health Report Portfolio Online Reporting Tool database and the commercial database Global Data, were used to track the progress of companies that received SBIR or Small Business Technology Transfer (STTR) funding from the NIA.

RESULTS

Since 2008, the NIA has awarded $280 million-including $207 million from fiscal year (FY) 2015 to FY 2019-in new small business program awards for AD/ADRD research.

DISCUSSION

NIA seed capital and mentoring programs are critical resources to help small businesses reach key value inflection points and advance their research from concept to commercialization.

The modified lottery: Formalizing the intrinsic randomness of research funding

Competition for research funds has, in the recent decade, become hypercompetitive. Commonly, to determine which proposals receive funding, a system of peer review is used, which is broadly accepted, easily understood, and broadly trusted among researchers. It is often considered the best system in use, but it suffers from important shortcomings, and adaptations to overcome these shortcomings have small and often short-lived effects. Hence, the preference for peer review does not mean it necessarily outperforms all other systems. In fact, it is time for an open discussion about alternative allocation mechanisms. Random allocation of research funding may be a viable alternative to the current peer review system. In particular the "organized randomness" of a modified lottery is interesting, combining the benefits of randomization with some of the most valuable aspects of peer review. Still, many questions remain and this is certainly not a plea to allocate all research funds using lotteries without further research. But we need to be prepared to consider alternatives, even though they are not perfect, and modified lotteries should be part of the solution.

Current Issues on Research Conducted to Improve Women's Health

There are varied lessons to be learned regarding the current methodological approaches to women’s health research. In the present scheme of growing medical literature and inflation of novel results claiming significance, the sheer amount of information can render evidence-based practice confusing. The factors that classically determined the impact of discoveries appear to be losing ground: citation count and publication rates, hierarchy in author lists according to contribution, and a journal’s impact factor. Through a comprehensive literature search on the currently available data from theses, opinion, and original articles and reviews on this topic, we seek to present to clinicians a narrative synthesis of three crucial axes underlying the totality of the research production chain: (a) critical advances in research methodology, (b) the interplay of academy and industry in a trial conduct, and (c) review- and publication-associated developments. We also provide specific recommendations on the study design and conduct, reviewing the processes and dissemination of data and the conclusions and implementation of findings. Overall, clinicians and the public should be aware of the discourse behind the marketing of alleged breakthrough research. Still, multiple initiatives, such as patient review and strict, supervised literature synthesis, have become more widely accepted. The “bottom-up” approach of a wide dissemination of information to clinicians, together with practical incentives for stakeholders with competing interests to collaborate, promise to improve women’s healthcare.

Unlocking the potential of marine biodiscovery

The tremendous diversity of life in the ocean has proven to be a rich source of inspiration for drug discovery, with success rates for marine natural products up to 4 times higher than other naturally derived compounds. Yet the marine biodiscovery pipeline is characterized by chronic underfunding, bottlenecks and, ultimately, untapped potential. For instance, a lack of taxonomic capacity means that, on average, 20 years pass between the discovery of new organisms and the formal publication of scientific names, a prerequisite to proceed with detecting and isolating promising bioactive metabolites. The need for "edge" research that can spur novel lines of discovery and lengthy high-risk drug discovery processes, are poorly matched with research grant cycles. Here we propose five concrete pathways to broaden the biodiscovery pipeline and open the social and economic potential of the ocean genome for global benefit: (1) investing in fundamental research, even when the links to industry are not immediately apparent; (2) cultivating equitable collaborations between academia and industry that share both risks and benefits for these foundational research stages; (3) providing new opportunities for early-career researchers and under-represented groups to engage in high-risk research without risking their careers; (4) sharing data with global networks; and (5) protecting genetic diversity at its source through strong conservation efforts. The treasures of the ocean have provided fundamental breakthroughs in human health and still remain under-utilised for human benefit, yet that potential may be lost if we allow the biodiscovery pipeline to become blocked in a search for quick-fix solutions.

The Government and Pharmaceutical Innovation: Looking Back and Looking Ahead

Current debates about the roles of the public and private sectors in pharmaceutical innovation have a long history. The extent to which, and ways in which, the public sector supports drug innovation has implications for assessments of the returns to public research funding, taxpayer rights in drugs, the argument the high prices are needed to support drug innovation, and the desirability of patenting publicly funded research.

NIH funding and the pursuit of edge science

The National Institutes of Health (NIH) plays a critical role in funding scientific endeavors in biomedicine. Funding innovative science is an essential element of the NIH's mission, but many have questioned the NIH's ability to fulfill this aim. Based on an analysis of a comprehensive corpus of published biomedical research articles, we measure whether the NIH succeeds in funding work with novel ideas, which we term edge science. We find that edge science is more often NIH funded than less novel science, but with a delay. Papers that build on very recent ideas are NIH funded less often than are papers that build on ideas that have had a chance to mature for at least 7 y. We have three further findings. First, the tendency to fund edge science is mostly limited to basic science. Papers that build on novel clinical ideas are not more often NIH funded than are papers that build on well-established clinical knowledge. Second, novel papers tend to be NIH funded more often because there are more NIH-funded papers in innovative areas of investigation, rather than because the NIH funds innovative papers within research areas. Third, the NIH's tendency to have funded papers that build on the most recent advances has declined over time. In this regard, NIH funding has become more conservative despite initiatives to increase funding for innovative projects. Given our focus on published papers, the results reflect both the funding preferences of the NIH and the composition of the applications it receives.