New and atypical combinations: An assessment of novelty and interdisciplinarity

Evaluating interdisciplinary research: Disparate outcomes for topic and knowledge base

Interdisciplinary research is essential for addressing complex global challenges, but there are concerns that scientific institutions like journals select against it. Prior work has focused largely on how interdisciplinarity relates to outcomes for published papers, but which papers get accepted for publication in the first place is unclear. Furthermore, journals may evaluate two key dimensions of interdisciplinarity,-topic and knowledge base,-differently. Topic interdisciplinarity (measured through title and abstract) may incur evaluation penalties by cutting across disciplinary evaluation standards and threatening symbolic boundaries, while knowledge-base interdisciplinarity (measured through references) may incur benefits by combining a large pool of nonredundant information. Evaluations may also depend on how well these dimensions align with each other and the intended audience. We test these arguments using data on 128,950 submissions to 62 journals across STEM disciplines, including both accepted and rejected manuscripts. We find that a 1SD increase in knowledge-base interdisciplinarity is associated with a 0.9 percentage-point higher acceptance probability, while a 1SD increase in topic interdisciplinarity corresponds to a 1.2 percentage-point lower acceptance probability. However, the penalty for high topic-interdisciplinarity diminishes when knowledge-base interdisciplinarity is also high, and when submitted to journals designated as "interdisciplinary." These findings challenge the narrative of a uniform bias against interdisciplinary research and highlight the importance of distinguishing between its dimensions, as well as their alignment with each other and the intended audience.

Quantifying the dynamics of peak disruption in scientific careers

We examine the disruption of researchers with long-lived careers in Computer Science and Physics. Despite the epistemological differences between such disciplines, we consistently find that a researcher's most disruptive publication does not occur at random during their career, as it cannot be explained by a null model. Such publication is accompanied by a peak year in which researchers publish other work that exhibits a higher level of disruption than average. Through a series of linear models, we show that the disruption achieved by a researcher during their peak year is higher when it is preceded by a long period of focus and low productivity. These findings are in stark contrast with the dynamics of academic impact. In these dynamics, researchers are incentivized by the prevalent paradigms of scientific evaluation to pursue high productivity and incremental-less disruptive-work, as evidenced by extensive literature.

The dynamics of higher-order novelties

Studying how we explore the world in search of novelties is key to understand the mechanisms that can lead to new discoveries. Previous studies analyzed novelties in various exploration processes, defining them as the first appearance of an element. However, novelties can also be generated by combining what is already known. We hence define higher-order novelties as the first time two or more elements appear together, and we introduce higher-order Heaps' exponents as a way to characterize their pace of discovery. Through extensive analysis of real-world data, we find that processes with the same pace of discovery, as measured by the standard Heaps' exponent, can instead differ at higher orders. We then propose to model an exploration process as a random walk on a network in which the possible connections between elements evolve in time. The model reproduces the empirical properties of higher-order novelties, revealing how the network we explore changes over time along with the exploration process.

Promotional language and the adoption of innovative ideas in science

How are the merits of innovative ideas communicated in science? Here, we conduct semantic analyses of grant application success with a focus on scientific promotional language, which may help to convey an innovative idea's originality and significance. Our analysis attempts to surmount the limitations of prior grant studies by examining the full text of tens of thousands of both funded and unfunded grants from three leading public and private funding agencies: the NIH, the NSF, and the Novo Nordisk Foundation, one of the world's largest private science funding foundations. We find a robust association between promotional language and the support and adoption of innovative ideas by funders and other scientists. First, a grant proposal's percentage of promotional language is associated with up to a doubling of the grant's probability of being funded. Second, a grant's promotional language reflects its intrinsic innovativeness. Third, the percentage of promotional language is predictive of the expected citation and productivity impact of publications that are supported by funded grants. Finally, a computer-assisted experiment that manipulates the promotional language in our data demonstrates how promotional language can communicate the merit of ideas through cognitive activation. With the incidence of promotional language in science steeply rising, and the pivotal role of grants in converting promising and aspirational ideas into solutions, our analysis provides empirical evidence that promotional language is associated with effectively communicating the merits of innovative scientific ideas.

Identify novel elements of knowledge with word embedding

As novelty is a core value in science, a reliable approach to measuring the novelty of scientific documents is critical. Previous novelty measures however had a few limitations. First, the majority of previous measures are based on recombinant novelty concept, attempting to identify a novel combination of knowledge elements, but insufficient effort has been made to identify a novel element itself (element novelty). Second, most previous measures are not validated, and it is unclear what aspect of newness is measured. Third, some of the previous measures can be computed only in certain scientific fields for technical constraints. This study thus aims to provide a validated and field-universal approach to computing element novelty. We drew on machine learning to develop a word embedding model, which allows us to extract semantic information from text data. Our validation analyses suggest that our word embedding model does convey semantic information. Based on the trained word embedding, we quantified the element novelty of a document by measuring its distance from the rest of the document universe. We then carried out a questionnaire survey to obtain self-reported novelty scores from 800 scientists. We found that our element novelty measure is significantly correlated with self-reported novelty in terms of discovering and identifying new phenomena, substances, molecules, etc. and that this correlation is observed across different scientific fields.

Effect of the topic-combination novelty on the disruption and impact of scientific articles: Evidence from PubMed

Novelty, disruption and impact are essential concepts for understanding the originality and importance of scientific discoveries. By drawing on a large-scale corpus consisting of nearly 0.9 million PubMed papers published between 1970 and 2009 and their citations before 2018 in the Web of Science, we found that the topic-combination novelty has different effects on the impact and disruption of scientific papers, that is, an inverted U-shaped effect on the impact and a positive effect on disruption. One of our contributions is that we have significantly improved the reliability of topic-combination novelty by applying MeSH terms of PubMed to the measurement of novelty. Another contribution is that we have explained how a novel combination of MeSH terms of an article contributes to citations and citation networks, that is, the middle-level novelty is more likely to achieve large citation counts. In contrast, high topic-combination novelty relates to the discontinuity in the focal paper’s citation network.

Interdisciplinary collaboration from diverse science teams can produce significant outcomes

Scientific teams are increasingly diverse in discipline, international scope and demographics. Diversity has been found to be a driver of innovation but also can be a source of interpersonal friction. Drawing on a mixed-method study of 22 scientific working groups, this paper presents evidence that team diversity has a positive impact on scientific output (i.e., the number of journal papers and citations) through the mediation of the interdisciplinarity of the collaborative process, as evidenced by publishing in and citing more diverse sources. Ironically these factors also seem to be related to lower team member satisfaction and perceived effectiveness, countered by the gender balance of the team. Qualitative data suggests additional factors that facilitate collaboration, such as trust and leadership. Our findings have implications for team design and management, as team diversity seems beneficial, but the process of integration can be difficult and needs management to lead to a productive and innovative process.

Is novel research worth doing? Evidence from peer review at 49 journals

There are long-standing concerns that peer review, which is foundational to scientific institutions like journals and funding agencies, favors conservative ideas over novel ones. We investigate the association between novelty and the acceptance of manuscripts submitted to a large sample of scientific journals. The data cover 20,538 manuscripts submitted between 2013 and 2018 to the journals Cell and Cell Reports and 6,785 manuscripts submitted in 2018 to 47 journals published by the Institute of Physics Publishing. Following previous work that found that a balance of novel and conventional ideas predicts citation impact, we measure the novelty and conventionality of manuscripts by the atypicality of combinations of journals in their reference lists, taking the 90th percentile most atypical combination as "novelty" and the 50th percentile as "conventionality." We find that higher novelty is consistently associated with higher acceptance; submissions in the top novelty quintile are 6.5 percentage points more likely than bottom quintile ones to get accepted. Higher conventionality is also associated with acceptance (+16.3% top-bottom quintile difference). Disagreement among peer reviewers was not systematically related to submission novelty or conventionality, and editors select strongly for novelty even conditional on reviewers' recommendations (+7.0% top-bottom quintile difference). Manuscripts exhibiting higher novelty were more highly cited. Overall, the findings suggest that journal peer review favors novel research that is well situated in the existing literature, incentivizing exploration in science and challenging the view that peer review is inherently antinovelty.

Scientific collaboration, research funding, and novelty in scientific knowledge

Disruptive advancements in science and technology often rely on new ideas and findings, which in turn brings us to focus on the value of novelty in scholarly activities. Using Web of Science publication data from European regions for the period between 2008 and 2017, this study examines, first, the impact of scientific collaboration on novelty of research. Here, five levels of collaboration are considered for each article–country, three levels of regions, and institutions, and novelty is measured with keywords information. Second, we investigate both the effect and moderating effect of research funding on novelty. Our findings show that there is a negative and significant relationship between scientific collaboration and novelty. Furthermore, funded papers show lower novelty than the unfunded, but funding does have a significant moderating effect on the relationship between collaboration and novelty. This study contributes by linking diverse levels of collaboration and funding sources to article’s novelty and thus extending the scope of bibliometric research of publications.

Benefits and detriments of interdisciplinarity on early career scientists’ performance. An author-level approach for U.S. physicists and psychologists

Is the pursuit of interdisciplinary or innovative research beneficial or detrimental for the impact of early career researchers? We focus on young scholars as they represent an understudied population who have yet to secure a place within academia. Which effects promise higher scientific recognition (i.e., citations) is therefore crucial for the high-stakes decisions young researchers face. To capture these effects, we introduce measurements for interdisciplinarity and novelty that can be applied to a researcher’s career. In contrast to previous studies investigating research impact on the paper level, hence, our paper focuses on a career perspective (i.e., the level of authors). To consider different disciplinary cultures, we utilize a comprehensive dataset on U.S. physicists (n = 4003) and psychologists (n = 4097), who graduated between 2008 and 2012, and traced their publication records. Our results indicate that conducting interdisciplinary research as an early career researcher in physics is beneficial, while it is negatively associated with research impact in psychology. In both fields, physics and psychology, early career researchers focusing on novel combinations of existing knowledge are associated with higher future impact. Taking some risks by deviating to a certain degree from mainstream paradigms seems therefore like a rewarding strategy for young scholars.

Measuring novelty in science with word embedding

Novelty is a core value in science, and a reliable measurement of novelty is crucial. This study proposes a new approach of measuring the novelty of scientific articles based on both citation data and text data. The proposed approach considers an article to be novel if it cites a combination of semantically distant references. To this end, we first assign a word embedding–a vector representation of each vocabulary–to each cited reference on the basis of text information included in the reference. With these vectors, a distance between every pair of references is computed. Finally, the novelty of a focal document is evaluated by summarizing the distances between all references. The approach draws on limited text information (the titles of references) and publicly shared library for word embeddings, which minimizes the requirement of data access and computational cost. We share the code, with which one can compute the novelty score of a document of interest only by having the focal document’s reference list. We validate the proposed measure through three exercises. First, we confirm that word embeddings can be used to quantify semantic distances between documents by comparing with an established bibliometric distance measure. Second, we confirm the criterion-related validity of the proposed novelty measure with self-reported novelty scores collected from a questionnaire survey. Finally, as novelty is known to be correlated with future citation impact, we confirm that the proposed measure can predict future citation.