Deep forecasting of translational impact in medical research

PTNS: patent citation trajectory prediction based on temporal network snapshots

With the rapid development of science and technology, the pace of development in the knowledge economy is accelerating. Intellectual property, especially patents, is a strategic resource for technological innovation and a crucial support for building an innovative country. Therefore, it is particularly important to predict patents with high value and strong impact from the numerous and uneven-quality patents. However, patent citation behavior involves many uncertainties, and it is difficult to capture its temporal variations effectively. Therefore, this paper proposes a patent citation trajectory prediction model (PTNS) based on temporal network snapshots. It adopts relational graph convolutional networks (R-GCN) to learn the complex relationships among multiple attributes of patents and utilizes bidirectional long short-term memory networks (BiLSTM) to aggregate the temporal evolution differences of patents. Subsequently, principal component analysis (PCA) is used to explore the evolution characteristics of patent citations in depth, thereby capturing the aging effect and the 'sleeping beauty' phenomenon. Compared with other baselines, the PTNS performs well. In predicting new, grown, and random patents, the RMSLE decreases by approximately 0.04, 0.14, and 0.18 respectively, while the MALE decreases by approximately 0.04, 0.12, and 0.16 respectively.

Machine learning to promote translational research: predicting patent and clinical trial inclusion in dementia research

Projected to impact 1.6 million people in the UK by 2040 and costing £25 billion annually, dementia presents a growing challenge to society. This study, a pioneering effort to predict the translational potential of dementia research using machine learning, hopes to address the slow translation of fundamental discoveries into practical applications despite dementia's significant societal and economic impact. We used the Dimensions database to extract data from 43 091 UK dementia research publications between the years 1990 and 2023, specifically metadata (authors, publication year, etc.), concepts mentioned in the paper and the paper abstract. To prepare the data for machine learning, we applied methods such as one-hot encoding and word embeddings. We trained a CatBoost Classifier to predict whether a publication will be cited in a future patent or clinical trial. We trained several model variations. The model combining metadata, concept and abstract embeddings yielded the highest performance: for patent predictions, an area under the receiver operating characteristic curve of 0.84 and 77.17% accuracy; for clinical trial predictions, an area under the receiver operating characteristic curve of 0.81 and 75.11% accuracy. The results demonstrate that integrating machine learning within current research methodologies can uncover overlooked publications, expediting the identification of promising research and potentially transforming dementia research by predicting real-world impact and guiding translational strategies.

The State of Artificial Intelligence in Skin Cancer Publications

BACKGROUND

Artificial intelligence (AI) in skin cancer is a promising research field to assist physicians and to provide support to patients remotely. Physicians' awareness to new developments in AI research is important to define the best practices and scope of integrating AI-enabled technologies within a clinical setting.

OBJECTIVES

To analyze the characteristics and trends of AI skin cancer publications from dermatology journals.

METHODS

AI skin cancer publications were retrieved in June 2022 from the Web of Science. Publications were screened by title, abstract, and keywords to assess eligibility. Publications were fully reviewed. Publications were divided between nonmelanoma skin cancer (NMSC), melanoma, and skin cancer studies. The primary measured outcome was the number of citations. The secondary measured outcomes were articles' general characteristics and features related to AI.

RESULTS

A total of 168 articles were included: 25 on NMSC, 77 on melanoma, and 66 on skin cancer. The most common types of skin cancers were melanoma (134, 79.8%), basal cell carcinoma (61, 36.3%), and squamous cell carcinoma (45, 26.9%). All articles were published between 2000 and 2022, with 49 (29.2%) of them being published in 2021. Original studies that developed or assessed an algorithm predominantly used supervised learning (66, 97.0%) and deep neural networks (42, 67.7%). The most used imaging modalities were standard dermoscopy (76, 45.2%) and clinical images (39, 23.2%).

CONCLUSIONS

Most publications focused on developing or assessing screening technologies with mainly deep neural network algorithms. This indicates the eminent need for dermatologists to label or annotate images used by novel AI systems.

Prioritising and incentivising productivity within indicator-based approaches to Research Impact Assessment: a commentary

Research Impact Assessment (RIA) represents one of a suite of policies intended to improve the impact generated from investment in health and medical research (HMR). Positivist indicator-based approaches to RIA are widely implemented but increasingly criticised as theoretically problematic, unfair, and burdensome. This commentary proposes there are useful outcomes that emerge from the process of applying an indicator-based RIA framework, separate from those encapsulated in the metrics themselves. The aim for this commentary is to demonstrate how the act of conducting an indicator-based approach to RIA can serve to optimise the productive gains from the investment in HMR. Prior research found that the issues regarding RIA are less about the choice of indicators/metrics, and more about the discussions prompted and activities incentivised by the process. This insight provides an opportunity to utilise indicator-based methods to purposely optimise the research impact. An indicator-based RIA framework specifically designed to optimise research impacts should: focus on researchers and the research process, rather than institution-level measures; utilise a project level unit of analysis that provides control to researchers and supports collaboration and accountability; provide for prospective implementation of RIA and the prospective orientation of research; establish a line of sight to the ultimate anticipated beneficiaries and impacts; Include process metrics/indicators to acknowledge interim steps on the pathway to final impacts; integrate 'next' users and prioritise the utilisation of research outputs as a critical measure; Integrate and align the incentives for researchers/research projects arising from RIA, with those existing within the prevailing research system; integrate with existing peer-review processes; and, adopt a system-wide approach where incremental improvements in the probability of translation from individual research projects, yields higher impact across the whole funding portfolio.Optimisation of the impacts from HMR investment represents the primary purpose of Research Impact policy. The process of conducting an indicator-based approach to RIA, which engages the researcher during the inception and planning phase, can directly contribute to this goal through improvements in the probability that an individual project will generate interim impacts. The research project funding process represents a promising forum to integrate this approach within the existing research system.

Exploring the clinical translation intensity of papers published by the world's top scientists in basic medicine

The extent to which basic medical research is translated into clinical practice is a topic of interest to all stakeholders. In this study, we assessed the clinical translation intensity of papers published by scientists who have made outstanding contributions to the field of basic medicine (Lasker Prize winners for Basic Medical Research). Approximate Potential for Translation (APT), Translational science scores (TS), and Citations by clinical research (Cited by Clin.) were analyzed as dependent variables. A traditional citation indicator was used as a reference (relative citation ratio, RCR). In order to examine the correlation between these different indicators and the characteristics of the paper, the author, and the institution. we used nonparametric tests, Spearman correlations, ordinal least squares regressions (OLS), quantile regressions, and zero-inflated negative binomial regression methods. We found that among the basic medical research papers published by Lasker Basic Medicine Award winners, (1) 20% are cited by clinical research; 11.6% of the papers were more valuable for clinical research than basic research; 12.8% have a probability of more than 50% to be cited in future clinical studies; (2) Spearman correlations were conducted among APT, TS, Cited by Clin., RCR, and all of the other continuous variables. There is a significant, positive, low to moderate correlation between APT, TS, and Cited by Clin (APT and TS: r = 0.549, p < 0.01; APT and Cited by Clin: r = 0.530, p < 0.01; TS and Cited by Clin: r = 0.383, p < 0.01). However, the relationship between RCR and the three indicators of clinical translation intensity was not consistent. APT was positively correlated with RCR (r = 0.553, p < 0.01). Cited by Clin. is weakly positively correlated with RCR (r = 0.381, p < 0.01). There is almost no correlation between TS and RCR (r = 0.184, p < 0.01). (3) Publication age, primary research paper, multidisciplinary science, number of disciplines, authors, institutions, funded projects, references, length of the title, length of paper, physical age, gender, nationality, institutional type, Nobel Prize have a significant relationship with 1 to 3 types of clinical translation intensity measures. In a sample of basic medical research papers published by the world's top scientists in basic medicine, we came to the following conclusions: the three indicators, APT, TS and Cited by Clin., measured the clinical translation intensity of the papers from different perspectives. They are both related to each other and have their own characteristics. In a sample of basic medical research papers published by the world's top scientists in basic medicine, characteristics at the paper, winner, and institution level significantly correlated with the measures of clinical translation intensity. Gender effect on the clinical translation intensity of papers was confirmed. Traditional citation-based indicators and translational-focused indicators measure academic impact and clinical impact respectively. There is a certain degree of disconnect between them. Two types of indicators should be used in combination in future assessments of basic medical research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11192-023-04634-4.

Reframing research impact

Amy Nelson, Senior Research Associate at University College London, and her team proposed a suite of deep learning models for scientific research evaluation that goes beyond citation-based features in impact analysis of biomedical research. In this People of Data, she talks about the future of medicine and patient care from the perspective of data science.