A pipeline for the retrieval and extraction of domain-specific information with application to COVID-19 immune signatures

BACKGROUND

The accelerating pace of biomedical publication has made it impractical to manually, systematically identify papers containing specific information and extract this information. This is especially challenging when the information itself resides beyond titles or abstracts. For emerging science, with a limited set of known papers of interest and an incomplete information model, this is of pressing concern. A timely example in retrospect is the identification of immune signatures (coherent sets of biomarkers) driving differential SARS-CoV-2 infection outcomes.

IMPLEMENTATION

We built a classifier to identify papers containing domain-specific information from the document embeddings of the title and abstract. To train this classifier with limited data, we developed an iterative process leveraging pre-trained SPECTER document embeddings, SVM classifiers and web-enabled expert review to iteratively augment the training set. This training set was then used to create a classifier to identify papers containing domain-specific information. Finally, information was extracted from these papers through a semi-automated system that directly solicited the paper authors to respond via a web-based form.

RESULTS

We demonstrate a classifier that retrieves papers with human COVID-19 immune signatures with a positive predictive value of 86%. The type of immune signature (e.g., gene expression vs. other types of profiling) was also identified with a positive predictive value of 74%. Semi-automated queries to the corresponding authors of these publications requesting signature information achieved a 31% response rate.

CONCLUSIONS

Our results demonstrate the efficacy of using a SVM classifier with document embeddings of the title and abstract, to retrieve papers with domain-specific information, even when that information is rarely present in the abstract. Targeted author engagement based on classifier predictions offers a promising pathway to build a semi-structured representation of such information. Through this approach, partially automated literature mining can help rapidly create semi-structured knowledge repositories for automatic analysis of emerging health threats.

KEBLM: Knowledge-Enhanced Biomedical Language Models

Pretrained language models (PLMs) have demonstrated strong performance on many natural language processing (NLP) tasks. Despite their great success, these PLMs are typically pretrained only on unstructured free texts without leveraging existing structured knowledge bases that are readily available for many domains, especially scientific domains. As a result, these PLMs may not achieve satisfactory performance on knowledge-intensive tasks such as biomedical NLP. Comprehending a complex biomedical document without domain-specific knowledge is challenging, even for humans. Inspired by this observation, we propose a general framework for incorporating various types of domain knowledge from multiple sources into biomedical PLMs. We encode domain knowledge using lightweight adapter modules, bottleneck feed-forward networks that are inserted into different locations of a backbone PLM. For each knowledge source of interest, we pretrain an adapter module to capture the knowledge in a self-supervised way. We design a wide range of self-supervised objectives to accommodate diverse types of knowledge, ranging from entity relations to description sentences. Once a set of pretrained adapters is available, we employ fusion layers to combine the knowledge encoded within these adapters for downstream tasks. Each fusion layer is a parameterized mixer of the available trained adapters that can identify and activate the most useful adapters for a given input. Our method diverges from prior work by including a knowledge consolidation phase, during which we teach the fusion layers to effectively combine knowledge from both the original PLM and newly-acquired external knowledge using a large collection of unannotated texts. After the consolidation phase, the complete knowledge-enhanced model can be fine-tuned for any downstream task of interest to achieve optimal performance. Extensive experiments on many biomedical NLP datasets show that our proposed framework consistently improves the performance of the underlying PLMs on various downstream tasks such as natural language inference, question answering, and entity linking. These results demonstrate the benefits of using multiple sources of external knowledge to enhance PLMs and the effectiveness of the framework for incorporating knowledge into PLMs. While primarily focused on the biomedical domain in this work, our framework is highly adaptable and can be easily applied to other domains, such as the bioenergy sector.

Querying semantic catalogues of biomedical databases

BACKGROUND

Secondary use of health data is a valuable source of knowledge that boosts observational studies, leading to important discoveries in the medical and biomedical sciences. The fundamental guiding principle for performing a successful observational study is the research question and the approach in advance of executing a study. However, in multi-centre studies, finding suitable datasets to support the study is challenging, time-consuming, and sometimes impossible without a deep understanding of each dataset.

METHODS

We propose a strategy for retrieving biomedical datasets of interest that were semantically annotated, using an interface built by applying a methodology for transforming natural language questions into formal language queries. The advantages of creating biomedical semantic data are enhanced by using natural language interfaces to issue complex queries without manipulating a logical query language.

RESULTS

Our methodology was validated using Alzheimer's disease datasets published in a European platform for sharing and reusing biomedical data. We converted data to semantic information format using biomedical ontologies in everyday use in the biomedical community and published it as a FAIR endpoint. We have considered natural language questions of three types: single-concept questions, questions with exclusion criteria, and multi-concept questions. Finally, we analysed the performance of the question-answering module we used and its limitations. The source code is publicly available at https://bioinformatics-ua.github.io/BioKBQA/.

CONCLUSION

We propose a strategy for using information extracted from biomedical data and transformed into a semantic format using open biomedical ontologies. Our method uses natural language to formulate questions to be answered by this semantic data without the direct use of formal query languages.

Sharing is caring: a call for a new era of rare disease research and development

Scientific advances in the understanding of the genetics and mechanisms of many rare diseases with previously unknown etiologies are inspiring optimism in the patient, clinical, and research communities and there is hope that disease-specific treatments are on the way. However, the rare disease community has reached a critical point in which its increasingly fragmented structure and operating models are threatening its ability to harness the full potential of advancing genomic and computational technologies. Changes are therefore needed to overcome these issues plaguing many rare diseases while also supporting economically viable therapy development. In "Data silos are undermining drug development and failing rare disease patients (Orphanet Journal of Rare Disease, Apr 2021)," we outlined many of the broad issues underpinning the increasingly fragmented and siloed nature of the rare disease space, as well as how the issues encountered by this community are representative of biomedical research more generally. Here, we propose several initiatives for key stakeholders - including regulators, private and public foundations, and research institutions - to reorient the rare disease ecosystem and its incentives in a way that we believe would cultivate and accelerate innovation. Specifically, we propose supporting non-proprietary patient registries, greater data standardization, global regulatory harmonization, and new business models that encourage data sharing and research collaboration as the default mode. Leadership needs to be integrated across sectors to drive meaningful change between patients, industry, sponsors, and academic medical centers. To transform the research and development landscape and unlock its vast healthcare, economic, and scientific potential for rare disease patients, a new model is ultimately the goal for all.

A data driven learning approach for the assessment of data quality

Background

Data quality assessment is important but complex and task dependent. Identifying suitable measurement methods and reference ranges for assessing their results is challenging. Manually inspecting the measurement results and current data driven approaches for learning which results indicate data quality issues have considerable limitations, e.g. to identify task dependent thresholds for measurement results that indicate data quality issues.

Objectives

To explore the applicability and potential benefits of a data driven approach to learn task dependent knowledge about suitable measurement methods and assessment of their results. Such knowledge could be useful for others to determine whether a local data stock is suitable for a given task.

Methods

We started by creating artificial data with previously defined data quality issues and applied a set of generic measurement methods on this data (e.g. a method to count the number of values in a certain variable or the mean value of the values). We trained decision trees on exported measurement methods’ results and corresponding outcome data (data that indicated the data’s suitability for a use case). For evaluation, we derived rules for potential measurement methods and reference values from the decision trees and compared these regarding their coverage of the true data quality issues artificially created in the dataset. Three researchers independently derived these rules. One with knowledge about present data quality issues and two without.

Results

Our self-trained decision trees were able to indicate rules for 12 of 19 previously defined data quality issues. Learned knowledge about measurement methods and their assessment was complementary to manual interpretation of measurement methods’ results.

Conclusions

Our data driven approach derives sensible knowledge for task dependent data quality assessment and complements other current approaches. Based on labeled measurement methods’ results as training data, our approach successfully suggested applicable rules for checking data quality characteristics that determine whether a dataset is suitable for a given task.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01656-x.

OnTARi: an ontology for factors influencing therapy adherence to rehabilitation

BACKGROUND

Adherence and motivation are key factors for successful treatment of patients with chronic diseases, especially in long-term care processes like rehabilitation. However, only a few patients achieve good treatment adherence. The causes are manifold. Adherence-influencing factors vary depending on indications, therapies, and individuals. Positive and negative effects are rarely confirmed or even contradictory. An ontology seems to be convenient to represent existing knowledge in this domain and to make it available for information retrieval.

METHODS

First, a manual data extraction of current knowledge in the domain of treatment adherence in rehabilitation was conducted. Data was retrieved from various sources, including basic literature, scientific publications, and health behavior models. Second, all adherence and motivation factors identified were formalized according to the ontology development methodology METHONTOLOGY. This comprises the specification, conceptualization, formalization, and implementation of the ontology "Ontology for factors influencing therapy adherence to rehabilitation" (OnTARi) in Protégé. A taxonomy-oriented evaluation was conducted by two domain experts.

RESULTS

OnTARi includes 281 classes implemented in ontology web language, ten object properties, 22 data properties, 1440 logical axioms, 244 individuals, and 1023 annotations. Six higher-level classes are differentiated: (1) Adherence, (2) AdherenceFactors, (3) AdherenceFactorCategory, (4) Rehabilitation, (5) RehabilitationForm, and (6) RehabilitationType. By means of the class AdherenceFactors 227 adherence factors, thereof 49 hard factors, are represented. Each factor involves a proper description, synonyms, possibly existing acronyms, and a German translation. OnTARi illustrates links between adherence factors through 160 influences-relations. Description logic queries implemented in Protégé allow multiple targeted requests, e.g., for the extraction of adherence factors in a specific rehabilitation area.

CONCLUSIONS

With OnTARi, a generic reference model was built to represent potential adherence and motivation factors and their interrelations in rehabilitation of patients with chronic diseases. In terms of information retrieval, this formalization can serve as a basis for implementation and adaptation of conventional rehabilitative measures, taking into account (patient-specific) adherence factors. OnTARi also enables the development of medical assistance systems to increase motivation and adherence in rehabilitation processes.

Data silos are undermining drug development and failing rare disease patients

Data silos are proliferating while research and development activity explode following genetic and immunological advances for many clinically described disorders with previously unknown etiologies. The latter event has inspired optimism in the patient, clinical, and research communities that disease-specific treatments are on the way. However, we fear the tendency of various stakeholders to balkanize databases in proprietary formats, driven by current economic and academic incentives, will inevitably fragment the expanding knowledge base and undermine current and future research efforts to develop much-needed treatments. The proliferation of proprietary databases, compounded by a paucity of meaningful outcome measures and/or good natural history data, slows our ability to generate scalable solutions to benefit chronically underserved patient populations in ways that would translate to more common diseases. The current research and development landscape sets too many projects up for unnecessary failure, particularly in the rare disease sphere, and does a grave disservice to highly vulnerable patients. This system also encourages the collection of redundant data in uncoordinated parallel studies and registries to ultimately delay or deny potential treatments for ostensibly tractable diseases; it also promotes the waste of precious time, energy, and resources. Groups at the National Institutes of Health and Food and Drug Administration have started programs to address these issues. However, we and many others feel there should be significantly more discussion of how to coordinate and scale registry efforts. Such discourse aims to reduce needless complexity and duplication of efforts, as well as promote a pre-competitive knowledge ecosystem for rare disease drug development that cultivates and accelerates innovation.

Data silos are undermining drug development and failing rare disease patients

Data silos are proliferating while research and development activity explode following genetic and immunological advances for many clinically described disorders with previously unknown etiologies. The latter event has inspired optimism in the patient, clinical, and research communities that disease-specific treatments are on the way. However, we fear the tendency of various stakeholders to balkanize databases in proprietary formats, driven by current economic and academic incentives, will inevitably fragment the expanding knowledge base and undermine current and future research efforts to develop much-needed treatments. The proliferation of proprietary databases, compounded by a paucity of meaningful outcome measures and/or good natural history data, slows our ability to generate scalable solutions to benefit chronically underserved patient populations in ways that would translate to more common diseases. The current research and development landscape sets too many projects up for unnecessary failure, particularly in the rare disease sphere, and does a grave disservice to highly vulnerable patients. This system also encourages the collection of redundant data in uncoordinated parallel studies and registries to ultimately delay or deny potential treatments for ostensibly tractable diseases; it also promotes the waste of precious time, energy, and resources. Groups at the National Institutes of Health and Food and Drug Administration have started programs to address these issues. However, we and many others feel there should be significantly more discussion of how to coordinate and scale registry efforts. Such discourse aims to reduce needless complexity and duplication of efforts, as well as promote a pre-competitive knowledge ecosystem for rare disease drug development that cultivates and accelerates innovation.

A method for interoperable knowledge-based data quality assessment

BACKGROUND

Assessing the quality of healthcare data is a complex task including the selection of suitable measurement methods (MM) and adequately assessing their results.

OBJECTIVES

To present an interoperable data quality (DQ) assessment method that formalizes MMs based on standardized data definitions and intends to support collaborative governance of DQ-assessment knowledge, e.g. which MMs to apply and how to assess their results in different situations.

METHODS

We describe and explain central concepts of our method using the example of its first real world application in a study on predictive biomarkers for rejection and other injuries of kidney transplants. We applied our open source tool-openCQA-that implements our method utilizing the openEHR specifications. Means to support collaborative governance of DQ-assessment knowledge are the version-control system git and openEHR clinical information models.

RESULTS

Applying the method on the study's dataset showed satisfactory practicability of the described concepts and produced useful results for DQ-assessment.

CONCLUSIONS

The main contribution of our work is to provide applicable concepts and a tested exemplary open source implementation for interoperable and knowledge-based DQ-assessment in healthcare that considers the need for flexible task and domain specific requirements.

Use of online knowledge base in primary health care and correlation to health care quality: an observational study

Background

Evidence-based information available at the point of care improves patient care outcomes. Online knowledge bases can increase the application of evidence-based medicine and influence patient outcome data which may be captured in quality registries. The aim of this study was to explore the effect of use of an online knowledge base on patient experiences and health care quality.

Methods

The study was conducted as a retrospective, observational study of 24 primary health care centers in Sweden exploring their use of an online knowledge base. Frequency of use was compared to patient outcomes in two national quality registries. A socio-economic Care Need Index was applied to assess whether the burden of care influenced the results from those quality registries. Non-parametric statistical methods and linear regression were used.

Results

Frequency of knowledge base use showed two groups: frequent and non-frequent users, with a significant use difference between the groups (p < 0.001). Outcome data showed significant higher values for all seven National Primary Care Patient Survey dimensions in the frequent compared to the non-frequent knowledge base users (p < 0.001), whereas 10 out of 11 parameters in the National Diabetes Register showed no differences between the groups (p > 0.05). Adjusting for Care Need Index had almost no effect on the outcomes for the groups.

Conclusions

Frequent users of a national online knowledge base received higher ratings on patient experiences, but figures on health care quality in diabetes showed near to no correlation. The findings indicate that some effects may be attributed to the use of knowledge bases and requires a controlled evaluation.

Silencing trust: confidence and familiarity in re-engineering knowledge infrastructures

In this paper, we tell the story of efforts currently underway, on diverse fronts, to build digital knowledge repositories ('knowledge-bases') to support research in the life sciences. If successful, knowledge bases will be part of a new knowledge infrastructure-capable of facilitating ever-more comprehensive, computational models of biological systems. Such an infrastructure would, however, represent a sea-change in the technological management and manipulation of complex data, inducing a generational shift in how questions are asked and answered and results published and circulated. Integrating such knowledge bases into the daily workflow of the lab thus destabilizes a number of well-established habits which biologists rely on to ensure the quality of the knowledge they produce, evaluate, communicate and exploit. As the story we tell here shows, such destabilization introduces a situation of unfamiliarity, one that carries with it epistemic risks. It should elicit-to use Niklas Luhmann's terms-the question of trust: a shared recognition that the reliability of research practices is being risked, but that such a risk is worth taking in view of what may be gained. And yet, the problem of trust is being unexpectedly silenced. How that silencing has come about, why it matters, and what might yet be done forms the heart of this paper.

Preconditions for Enabling Advanced Patient-Centered Decision Support on a National Knowledge Information Infrastructure

In Western healthcare, an important goal is to provide clinical decision support "for the right healthcare personnel, in the right situation, at the right time". In this poster, we use a qualitative approach to outline the preconditions for enabling such advanced patient-centered decision support. This study indicates that establishing a national knowledge information infrastructure demands well-defined national standards, codes, and terminologies, as well as structured clinical data. An extensive governance structure is also required.

The application of big data and the development of nursing science: A discussion paper

Based on the concept and research status of big data, we analyze and examine the importance of constructing the knowledge system of nursing science for the development of the nursing discipline in the context of big data and propose that it is necessary to establish big data centers for nursing science to share resources, unify language standards, improve professional nursing databases, and establish a knowledge system structure.

Discovering hidden knowledge through auditing clinical diagnostic knowledge bases

OBJECTIVE

Evaluate potential for data mining auditing techniques to identify hidden concepts in diagnostic knowledge bases (KB). Improving completeness enhances KB applications such as differential diagnosis and patient case simulation.

MATERIALS AND METHODS

Authors used unsupervised (Pearson's correlation - PC, Kendall's correlation - KC, and a heuristic algorithm - HA) methods to identify existing and discover new finding-finding interrelationships ("properties") in the INTERNIST-1/QMR KB. Authors estimated KB maintenance efficiency gains (effort reduction) of the approaches.

RESULTS

The methods discovered new properties at 95% CI rates of [0.1%, 5.4%] (PC), [2.8%, 12.5%] (KC), and [5.6%, 18.8%] (HA). Estimated manual effort reduction for HA-assisted determination of new properties was approximately 50-fold.

CONCLUSION

Data mining can provide an efficient supplement to ensuring the completeness of finding-finding interdependencies in diagnostic knowledge bases. Authors' findings should be applicable to other diagnostic systems that record finding frequencies within diseases (e.g., DXplain, ISABEL).

Harmonising phenomics information for a better interoperability in the rare disease field

HIPBI-RD (Harmonising phenomics information for a better interoperability in the rare disease field) is a three-year project which started in 2016 funded via the E-Rare 3 ERA-NET program. This project builds on three resources largely adopted by the rare disease (RD) community: Orphanet, its ontology ORDO (the Orphanet Rare Disease Ontology), HPO (the Human Phenotype Ontology) as well as PhenoTips software for the capture and sharing of structured phenotypic data for RD patients. Our project is further supported by resources developed by the European Bioinformatics Institute and the Garvan Institute. HIPBI-RD aims to provide the community with an integrated, RD-specific bioinformatics ecosystem that will harmonise the way phenomics information is stored in databases and patient files worldwide, and thereby contribute to interoperability. This ecosystem will consist of a suite of tools and ontologies, optimized to work together, and made available through commonly used software repositories. The project workplan follows three main objectives: The HIPBI-RD ecosystem will contribute to the interpretation of variants identified through exome and full genome sequencing by harmonising the way phenotypic information is collected, thus improving diagnostics and delineation of RD. The ultimate goal of HIPBI-RD is to provide a resource that will contribute to bridging genome-scale biology and a disease-centered view on human pathobiology. Achievements in Year 1.

The Mining Minds digital health and wellness framework

Background

The provision of health and wellness care is undergoing an enormous transformation. A key element of this revolution consists in prioritizing prevention and proactivity based on the analysis of people’s conducts and the empowerment of individuals in their self-management. Digital technologies are unquestionably destined to be the main engine of this change, with an increasing number of domain-specific applications and devices commercialized every year; however, there is an apparent lack of frameworks capable of orchestrating and intelligently leveraging, all the data, information and knowledge generated through these systems.

Methods

This work presents Mining Minds, a novel framework that builds on the core ideas of the digital health and wellness paradigms to enable the provision of personalized support. Mining Minds embraces some of the most prominent digital technologies, ranging from Big Data and Cloud Computing to Wearables and Internet of Things, as well as modern concepts and methods, such as context-awareness, knowledge bases or analytics, to holistically and continuously investigate on people’s lifestyles and provide a variety of smart coaching and support services.

Results

This paper comprehensively describes the efficient and rational combination and interoperation of these technologies and methods through Mining Minds, while meeting the essential requirements posed by a framework for personalized health and wellness support. Moreover, this work presents a realization of the key architectural components of Mining Minds, as well as various exemplary user applications and expert tools to illustrate some of the potential services supported by the proposed framework.

Conclusions

Mining Minds constitutes an innovative holistic means to inspect human behavior and provide personalized health and wellness support. The principles behind this framework uncover new research ideas and may serve as a reference for similar initiatives.

Automated Fictional Ideation via Knowledge Base Manipulation

The invention of fictional ideas (ideation) is often a central process in the creative production of artefacts such as poems, music and paintings, but has barely been studied in the computational creativity community. We present here a general approach to automated fictional ideation that works by manipulating facts specified in knowledge bases. More specifically, we specify a number of constructions which, by altering and combining facts from a knowledge base, result in the generation of fictions. Moreover, we present an instantiation of these constructions through the use of ConceptNet, a database of common sense knowledge. In order to evaluate the success of these constructions, we present a curation analysis that calculates the proportion of ideas which pass a typicality judgement. We further evaluate the output of this approach through a crowd-sourcing experiment in which participants were asked to rank ideas. We found a positive correlation between the participant’s rankings and a chaining inference technique that automatically assesses the value of the fictions generated through our approach. We believe that these results show that this approach constitutes a firm basis for automated fictional ideation with evaluative capacity.

Development of a clinician reputation metric to identify appropriate problem-medication pairs in a crowdsourced knowledge base

BACKGROUND

Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements.

OBJECTIVE

We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review.

APPROACH

We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs.

RESULTS

Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases.

CONCLUSION

A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.