Evaluation of Discrepancies Among National Library of Medicine (NLM) Value Set Authority Center (VSAC) ICD-10-CM Value Sets: Case Study for Diagnoses of Common Chronic Conditions, Implications, and Potential Solutions

The National Library of Medicine (NLM)'s Value Set Authority Center (VSAC) is a crowd-sourced repository with a potential for substantial discrepancy among value sets for the same clinical concepts. To characterize this potential problem, we identified the most common chronic conditions affecting US adults and assessed for discrepancy among VSAC ICD-10-CM value sets for these conditions. An analysis of 32 value sets for 12 conditions identified that a median of 45% of codes for a given condition were potentially problematic (included in at least one, but not all, theoretically equivalent value sets). These problematic codes were used to document clinical care for potentially over 20 million patients in a data warehouse of approximately 150 million US adults. Users of VSAC diagnosis value sets should be cognizant of the prevalence of these discrepancies and take proactive steps to mitigate their impact. Further research is warranted to characterize and address this issue.

Automatic Mapping of Terminology Items with Transformers

Biomedical ontologies are a key component in many systems for the analysis of textual clinical data. They are employed to organize information about a certain domain relying on a hierarchy of different classes. Each class maps a concept to items in a terminology developed by domain experts. These mappings are then leveraged to organize the information extracted by Natural Language Processing (NLP) models to build knowledge graphs for inferences. The creation of these associations, however, requires extensive manual review. In this paper, we present an automated approach and repeatable framework to learn a mapping between ontology classes and terminology terms derived from vocabularies in the Unified Medical Language System (UMLS) metathesaurus. According to our evaluation, the proposed system achieves a performance close to humans and provides a substantial improvement over existing systems developed by the National Library of Medicine to assist researchers through this process.

Effects of Porting Essie Tokenization and Normalization to Solr

Search for information is now an integral part of healthcare. Searches are enabled by search engines whose objective is to efficiently retrieve the relevant information for the user query. When it comes to retrieving biomedical text and literature, Essie search engine developed at the National Library of Medicine (NLM) performs exceptionally well. However, Essie is a software system developed for NLM that has ceased development and support. On the other hand, Solr is a popular opensource enterprise search engine used by many of the world's largest internet sites, offering continuous developments and improvements along with the state-of-the-art features. In this paper, we present our approach to porting the key features of Essie and developing custom components to be used in Solr. We demonstrate the effectiveness of the added components on three benchmark biomedical datasets. The custom components may aid the community in improving search methods for biomedical text retrieval.

Database resources of the National Center for Biotechnology Information

The National Center for Biotechnology Information (NCBI) provides online information resources for biology, including the GenBank® nucleic acid sequence database and the PubMed® database of citations and abstracts published in life science journals. NCBI provides search and retrieval operations for most of these data from 35 distinct databases. The E-utilities serve as the programming interface for most of these databases. Resources receiving significant updates in the past year include PubMed, PMC, Bookshelf, SciENcv, the NIH Comparative Genomics Resource (CGR), NCBI Virus, SRA, RefSeq, foreign contamination screening tools, Taxonomy, iCn3D, ClinVar, GTR, MedGen, dbSNP, ALFA, ClinicalTrials.gov, Pathogen Detection, antimicrobial resistance resources, and PubChem. These resources can be accessed through the NCBI home page at https://www.ncbi.nlm.nih.gov.

Interactive Images in Library Instruction: A Case Study

This paper discusses the design and implementation of the H5P image hotspots and describes additional use cases for library instruction in order to teach learners from across the country on how different spaces can be used to create private, accessible areas where patrons can conduct telehealth appointments. The Network of the National Library of Medicine created an online class using H5P image hotspots to allow learners to virtually explore a physical space. The class used Moodle, an open-source learning platform, and H5P image hotspots to demonstrate how libraries implemented telehealth programs.

Notes on the data quality of bibliographic records from the MEDLINE database

The US National Library of Medicine has created and maintained the PubMed® database, a collection of over 33.8 million records that contain citations and abstracts from the biomedical and life sciences literature. This database is an important resource for researchers and information service providers alike. As part of our work related to the creation of an author graph for coronaviruses, we encountered several data quality issues with records from a curated subset of the PubMed database called MEDLINE. We provide a data quality assessment for records selected from the MEDLINE database and report on several issues ranging from parsing issues (e.g. character encodings and schema definition weaknesses) to low scores for identifiers against several data quality metrics (e.g. completeness, validity and uniqueness). Database URL  https://pubmed.ncbi.nlm.nih.gov.

Future advancement of health care through standardized nursing terminologies: reflections from a Friends of the National Library of Medicine workshop honoring Virginia K. Saba

OBJECTIVE

To honor the legacy of nursing informatics pioneer and visionary, Dr. Virginia Saba, the Friends of the National Library of Medicine convened a group of international experts to reflect on Dr. Saba's contributions to nursing standardized nursing terminologies.

PROCESS

Experts led a day-and-a-half virtual update on nursing's sustained and rigorous efforts to develop and use valid, reliable, and computable standardized nursing terminologies over the past 5 decades. Over the course of the workshop, policymakers, industry leaders, and scholars discussed the successful use of standardized nursing terminologies, the potential for expanded use of these vetted tools to advance healthcare, and future needs and opportunities. In this article, we elaborate on this vision and key recommendations for continued and expanded adoption and use of standardized nursing terminologies across settings and systems with the goal of generating new knowledge that improves health.

CONCLUSION

Much of the promise that the original creators of standardized nursing terminologies envisioned has been achieved. Secondary analysis of clinical data using these terminologies has repeatedly demonstrated the value of nursing and nursing's data. With increased and widespread adoption, these achievements can be replicated across settings and systems.

Collaborative collection development: a MedPrint case report

Background

In response to several of Texas' largest medical libraries being forced to discard all serial print holdings, the Texas A&M University System and University of Texas System's Joint Library Facility (JLF) staff worked to help provide a solution to save and store these resources. This process fire-started a comprehensive effort by JLF staff to contact the National Library of Medicine (NLM) and devise a blueprint that would be used to help save and preserve all serial medical resources listed in NLM's medical retention program.

Case Presentation

In an unprecedented approach, the Texas A&M JLF staff launched efforts to collect and preserve the complete holdings range of all NLM MedPrint periodical runs. This case report details the planning and steps JLF staff took to accomplish this feat; highlights important matters of consideration for the medical community which heavily relies upon continuous access to MedPrint materials; and provides insight on the apparent preservation vulnerabilities these materials increasingly face in an environment where digitization may create a false sense of security.

Discussion

By May 2021, JLF had collected complete title runs up to year 2000 for 202 of the 254 MedPrint titles, which consists of more than twelve thousand volumes. These efforts proved particularly beneficial in the wake of the COVID-19 pandemic, which forced NLM to halt ILL processing from their print collection. During this time, JLF was uniquely positioned to meet and respond to the historic high number of medical literature ILL requests it received during this time.

Examination of reporting status of pediatric oncology trials within the national library of medicine's trial database

Finding safer and more effective treatment options are critical in progressing the field of pediatric oncology. These treatment options are discovered through completion and publication of clinical trials. The primary objective of this study was to assess the overall study characteristics of pediatric oncology clinical trials initiated between 2008 and 2021. The secondary objective of our study was to assess rates of discontinuation and reporting of results as required by the Food and Drug Administration (FDA). After acquiring pediatric oncology clinical trials from ClinicalTrials.gov, a cross-sectional study was performed. Included trials have an intervention exclusive to pediatrics and were conducted between 2008 and 2021. The results measured were characteristics of the clinical trials and their rate of discontinuation. Of the 7,930 trials originally returned from the search, 349 trials met inclusion criteria. The majority of the trials were phase 1 and 2 pharmaceutical interventions studying brain and blood cancer. Our study found that 14.9% (52) of the pediatric oncology trials were discontinued. Given the breadth of study within pediatric oncology, our overarching assessment shows that drug trials geared toward treating cancers of the brain and blood were dominant in the field. It is crucial for the advancement of science that results of trials are known. This avoids duplication of studies and waste of funds. Of the trials that were completed, 40.3% (58) did not report results to ClinicalTrials.gov. The nonreporting of this data limits the information available delaying the advancement of treatment options.

Chemical identification and indexing in full-text articles: an overview of the NLM-Chem track at BioCreative VII

The BioCreative National Library of Medicine (NLM)-Chem track calls for a community effort to fine-tune automated recognition of chemical names in the biomedical literature. Chemicals are one of the most searched biomedical entities in PubMed, and-as highlighted during the coronavirus disease 2019 pandemic-their identification may significantly advance research in multiple biomedical subfields. While previous community challenges focused on identifying chemical names mentioned in titles and abstracts, the full text contains valuable additional detail. We, therefore, organized the BioCreative NLM-Chem track as a community effort to address automated chemical entity recognition in full-text articles. The track consisted of two tasks: (i) chemical identification and (ii) chemical indexing. The chemical identification task required predicting all chemicals mentioned in recently published full-text articles, both span [i.e. named entity recognition (NER)] and normalization (i.e. entity linking), using Medical Subject Headings (MeSH). The chemical indexing task required identifying which chemicals reflect topics for each article and should therefore appear in the listing of MeSH terms for the document in the MEDLINE article indexing. This manuscript summarizes the BioCreative NLM-Chem track and post-challenge experiments. We received a total of 85 submissions from 17 teams worldwide. The highest performance achieved for the chemical identification task was 0.8672 F-score (0.8759 precision and 0.8587 recall) for strict NER performance and 0.8136 F-score (0.8621 precision and 0.7702 recall) for strict normalization performance. The highest performance achieved for the chemical indexing task was 0.6073 F-score (0.7417 precision and 0.5141 recall). This community challenge demonstrated that (i) the current substantial achievements in deep learning technologies can be utilized to improve automated prediction accuracy further and (ii) the chemical indexing task is substantially more challenging. We look forward to further developing biomedical text-mining methods to respond to the rapid growth of biomedical literature. The NLM-Chem track dataset and other challenge materials are publicly available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BC7-NLM-Chem-track/. Database URL https://ftp.ncbi.nlm.nih.gov/pub/lu/BC7-NLM-Chem-track/.

Database resources of the National Center for Biotechnology Information in 2023

The National Center for Biotechnology Information (NCBI) provides online information resources for biology, including the GenBank® nucleic acid sequence database and the PubMed® database of citations and abstracts published in life science journals. NCBI provides search and retrieval operations for most of these data from 35 distinct databases. The E-utilities serve as the programming interface for most of these databases. New resources include the Comparative Genome Resource (CGR) and the BLAST ClusteredNR database. Resources receiving significant updates in the past year include PubMed, PMC, Bookshelf, IgBLAST, GDV, RefSeq, NCBI Virus, GenBank type assemblies, iCn3D, ClinVar, GTR, dbGaP, ALFA, ClinicalTrials.gov, Pathogen Detection, antimicrobial resistance resources, and PubChem. These resources can be accessed through the NCBI home page at https://www.ncbi.nlm.nih.gov.

Alignment of two standard terminologies for dosage form: RxNorm from the National Library of Medicine for the United States and EDQM from the European Directorate for the Quality in Medicines and Healthcare for Europe

BACKGROUND

There is currently no system that aligns pharmaceutically equivalent medicinal products across nations, creating obstacles to transnational medication prescribing and medical research. EDQM has been internationally recognized as the leading system in systematic pharmaceutical product descriptions. RxNorm is a critical terminology based in the US and used widely in applications internationally that would benefit from alignment with EDQM-based dosage form descriptions.

GOAL

Demonstrate a method for alignment of RxNorm dosage forms with EDQM terminologies and with EDQM dosage forms. Describe obstacles and advantages of such an alignment for ultimate application in calculating universal Pharmaceutical Product Identifiers.

METHODS

A pharmaceutical sciences student and a clinical pharmacology expert in dosage forms used definitions supplied by RxNorm and EDQM technical documentation to align the 120 RxNorm dose forms to EDQM-based dosage form description terms. The alignment of RxNorm to EDQM was then used to fit the RxNorm dose forms into an ontology based on EDQM.

RESULTS AND CONCLUSIONS

The alignment of RxNorm and EDQM requires further validation but provides a potential method of establishing interoperability between the two terminologies without cumbersome manual reclassification. There remain ambiguities within each dosage form nomenclature that create obstacles to integrating medication databases rooted in EDQM and RxNorm into a single worldwide database.

A joint NCBI and EMBL-EBI transcript set for clinical genomics and research

Comprehensive genome annotation is essential to understand the impact of clinically relevant variants. However, the absence of a standard for clinical reporting and browser display complicates the process of consistent interpretation and reporting. To address these challenges, Ensembl/GENCODE1 and RefSeq2 launched a joint initiative, the Matched Annotation from NCBI and EMBL-EBI (MANE) collaboration, to converge on human gene and transcript annotation and to jointly define a high-value set of transcripts and corresponding proteins. Here, we describe the MANE transcript sets for use as universal standards for variant reporting and browser display. The MANE Select set identifies a representative transcript for each human protein-coding gene, whereas the MANE Plus Clinical set provides additional transcripts at loci where the Select transcripts alone are not sufficient to report all currently known clinical variants. Each MANE transcript represents an exact match between the exonic sequences of an Ensembl/GENCODE transcript and its counterpart in RefSeq such that the identifiers can be used synonymously. We have now released MANE Select transcripts for 97% of human protein-coding genes, including all American College of Medical Genetics and Genomics Secondary Findings list v3.0 (ref. 3) genes. MANE transcripts are accessible from major genome browsers and key resources. Widespread adoption of these transcript sets will increase the consistency of reporting, facilitate the exchange of data regardless of the annotation source and help to streamline clinical interpretation.

Preliminary comparison of the performance of the National Library of Medicine’s systematic review publication type and the sensitive clinical queries filter for systematic reviews in PubMed

Objective: The National Library of Medicine (NLM) inaugurated a “publication type” concept to facilitate searches for systematic reviews (SRs). On the other hand, clinical queries (CQs) are validated search strategies designed to retrieve scientifically sound, clinically relevant original and review articles from biomedical literature databases. We compared the retrieval performance of the SR publication type (SR[pt]) against the most sensitive CQ for systematic review articles (CQrs) in PubMed. Methods: We ran date-limited searches of SR[pt] and CQrs to compare the relative yield of articles and SRs, focusing on the differences in retrieval of SRs by SR[pt] but not CQrs (SR[pt] NOT CQrs) and CQrs NOT SR[pt]. Random samples of articles retrieved in each of these comparisons were examined for SRs until a consistent pattern became evident. Results: For SR[pt] NOT CQrs, the yield was relatively low in quantity but rich in quality, with 79% of the articles being SRs. For CQrs NOT SR[pt], the yield was high in quantity but low in quality, with only 8% being SRs. For CQrs AND SR[pt], the quality was highest, with 92% being SRs. Conclusions: We found that SR[pt] had high precision and specificity for SRs but low recall (sensitivity), whereas CQrs had much higher recall. SR[pt] OR CQrs added valid SRs to the CQrs yield at low cost (i.e., added few non-SRs). For searches that are intended to be exhaustive for SRs, SR[pt] can be added to existing sensitive search filters.

Dr. Lindberg and Scholarly Publishing

Donald A.B. Lindberg M.D., Director of the U.S. National Library of Medicine (NLM) from August 1984-March 2015, had a remarkable vision for NLM's scope, goals, and function. This vision resulted in many external partnerships and initiatives with the publishing industry, commercial and non-profit, journal editors, and professional organizations. These partnerships ranged from ongoing collaboration and dialogue, such as the NLM Publisher's Committee and the International Committee of Medical Journal Editors (ICMJE). to the more practical, such as the creation of HINARI and the Emergency Access Initiative (EAI). Dr. Lindberg fostered partnerships outside the NLM to expand the use and reach of Library resources, including MEDLINE and ClinicalTrials.gov to support innovations in the processes that build them, and improve the quality of biomedical journals. Dr. Lindberg also encouraged the use of technology to enhance medical information and supported the early development of fully interactive publications. Attitudes that contained a measure of skepticism and distrust faded as collaborators came to have a better understanding of both NLM and their partners. This chapter discusses these relationships and accomplishments that NLM achieved working with publishers and other creators and disseminators of medical information under Dr. Lindberg's leadership.

Grateful Med: Direct Access to MEDLINE for Health Professionals with Personal Computers

Donald A.B. Lindberg M.D. arrived as Director, U.S. National Library of Medicine (NLM) in late 1984 with the intention of implementing a physician-friendly interface to MEDLINE, a prime example of his interest in making NLM information services more directly useful in medical care. By early 1986, NLM's Grateful Med, an inexpensive PC search interface to MEDLINE useful for health professionals, had joined the group of end-user systems for searching MEDLINE that emerged in the 1980s. This chapter recounts Grateful Med's rapid iterative development and the subsequent campaign to bring it to attention of health professionals. It emphasizes Lindberg's role, the challenges faced by those introducing and using the interface in a pre-Internet world, and some longer-term effects of the effort to expand health professionals' use of MEDLINE during the decade from 1986 to 1996.

Promoting New and Expanded Roles for Librarians and Information Specialists

This chapter describes how the U.S. National Library of Medicine (NLM), under the leadership of Donald A.B. Lindberg M.D., promoted new and expanded roles for librarians and information specialists in response to advances in technology and public policy. These advances brought information services directly to all potential users, including health professionals and the public and stimulated NLM to expand its programs, policies, and services to serve all. Dr. Lindberg included librarians and information specialists in all of NLM's new endeavors, helping both to recognize and establish new or expanded roles. The involvement of librarians and information specialists in multidisciplinary healthcare research teams, in underserved communities, and in research data management and compliance has helped to redefine the health sciences information profession for the 21st century.

HIV/AIDS Community Information Outreach Program (ACIOP): A Landmark NIH Conference and an Enduring NLM Role in Meeting the Affected Community's Need for Information Access

In June 1993, the National Library of Medicine (NLM) joined with the National Institutes of Health's (NIH) Office of AIDS Research (OAR), and the National Institute of Allergy and Infectious Diseases (NIAID) to host a conference at a pivotal time in the HIV/AIDS epidemic to understand better the information needs of five major constituency groups: clinical researchers; clinical providers; news media and the public; patients; and the affected community. NLM's director, Donald A.B. Lindberg M.D., and staff sought to identify new program possibilities benefitting from the input of current and potential users of the Library's information services. Conference recommendations led to a key NLM policy change providing cost-free access to all AIDS data, and the establishment of the HIV/AIDS Community Information Outreach Program (ACIOP), which enabled new partnerships with local community-based organizations serving the affected community. Uniquely funded and long running, more than 300 ACIOP projects have been supported to-date. These projects have improved awareness and use of national HIV/AIDS information resources; enhanced information seeking skills; developed locally generated information resources; and enhanced the capacity of community-based organizations to use new information and computer technologies providing access to essential information resources and services.

When Spider Webs Unite, They Can Tie Up a Lion: NLM's Work in Information Technology and Health in Africa, 1997-20111

In 1997, Donald A.B. Lindberg M.D., Director, U.S. National Library of Medicine (NLM) agreed to address the request of African malaria researchers for access to the Internet and medical journals as part of the U.S. National Institutes of Health's (NIH) contribution to the Multilateral Initiative on Malaria (MIM). This challenge matched my interests and previous experience in Africa. I joined NLM in 1997 to help establish the MIM Communications Network (MIMCom) in partnership with several NIH components and more than 30 other partners in Africa, the U.S., the United Kingdom (U.K.), and Europe. After a successful launch of MIMCom, NLM worked with African partners to create a series of innovative programs to build capacity in Africa and enhance global access to indigenous African research.

Native Voices Exhibition: Stories of Health, Wellness, and Illness from American Indians, Alaska Natives, and Native Hawaiians

The U.S. National Library of Medicine (NLM) exhibition known as Native Voices reflected Donald A.B. Lindberg M.D.'s keen and long-held desire to help improve public understanding of Native American health challenges and honor the culture, tradition, and healing ways of Native Peoples. A centerpiece of the exhibition was a large set of video interviews that Dr. Lindberg conducted with Native health and community leaders. Dr. Lindberg and his team engaged Native advisors in the exhibition development; sought Native input through Listening Circles, Tribal Consultations; and site visits, and made the video interviews accessible via interactive kiosks and iPads. For its time, this was state-of-the-art exhibition technology. The exhibition also included Native artifacts and art works to complement the videos, including a scale model of the iconic Hokule'a Native Hawaiian voyaging canoe, and a full-size Lummi Indian Healing Totem Pole. The totem journeyed across the U.S. prior to its installation next to the NLM herbal garden in Bethesda, MD. A traveling version of the exhibition visited more than 130 venues in 40 States across the U.S. The interview clips and other content are accessible on the exhibition website, and the full-length interviews are retained in the NLM permanent video collection.

GenBank

GenBank^® (https://www.ncbi.nlm.nih.gov/genbank/) is a comprehensive, public database that contains 15.3 trillion base pairs from over 2.5 billion nucleotide sequences for 504 000 formally described species. Recent updates include resources for data from the SARS-CoV-2 virus, including a SARS-CoV-2 landing page, NCBI Datasets, NCBI Virus and the Submission Portal. We also discuss upcoming changes to GI identifiers, a new data management interface for BioProject, and advice for providing contextual metadata in submissions.

A survey of biomedical journals to detect editorial bias and nepotistic behavior

Alongside the growing concerns regarding predatory journal growth, other questionable editorial practices have gained visibility recently. Among them, we explored the usefulness of the Percentage of Papers by the Most Prolific author (PPMP) and the Gini index (level of inequality in the distribution of authorship among authors) as tools to identify journals that may show favoritism in accepting articles by specific authors. We examined whether the PPMP, complemented by the Gini index, could be useful for identifying cases of potential editorial bias, using all articles in a sample of 5,468 biomedical journals indexed in the National Library of Medicine. For articles published between 2015 and 2019, the median PPMP was 2.9%, and 5% of journal exhibited a PPMP of 10.6% or more. Among the journals with the highest PPMP or Gini index values, where a few authors were responsible for a disproportionate number of publications, a random sample was manually examined, revealing that the most prolific author was part of the editorial board in 60 cases (61%). The papers by the most prolific authors were more likely to be accepted for publication within 3 weeks of their submission. Results of analysis on a subset of articles, excluding nonresearch articles, were consistent with those of the principal analysis. In most journals, publications are distributed across a large number of authors. Our results reveal a subset of journals where a few authors, often members of the editorial board, were responsible for a disproportionate number of publications. To enhance trust in their practices, journals need to be transparent about their editorial and peer review practices.

NLM-Gene, a richly annotated gold standard dataset for gene entities that addresses ambiguity and multi-species gene recognition

The automatic recognition of gene names and their corresponding database identifiers in biomedical text is an important first step for many downstream text-mining applications. While current methods for tagging gene entities have been developed for biomedical literature, their performance on species other than human is substantially lower due to the lack of annotation data. We therefore present the NLM-Gene corpus, a high-quality manually annotated corpus for genes developed at the US National Library of Medicine (NLM), covering ambiguous gene names, with an average of 29 gene mentions (10 unique identifiers) per document, and a broader representation of different species (including Homo sapiens, Mus musculus, Rattus norvegicus, Drosophila melanogaster, Arabidopsis thaliana, Danio rerio, etc.) when compared to previous gene annotation corpora. NLM-Gene consists of 550 PubMed abstracts from 156 biomedical journals, doubly annotated by six experienced NLM indexers, randomly paired for each document to control for bias. The annotators worked in three annotation rounds until they reached complete agreement. This gold-standard corpus can serve as a benchmark to develop & test new gene text mining algorithms. Using this new resource, we have developed a new gene finding algorithm based on deep learning which improved both on precision and recall from existing tools. The NLM-Gene annotated corpus is freely available at ftp://ftp.ncbi.nlm.nih.gov/pub/lu/NLMGene. We have also applied this tool to the entire PubMed/PMC with their results freely accessible through our web-based tool PubTator (www.ncbi.nlm.nih.gov/research/pubtator).

Impact of a consumer health information specialization (CHIS) sponsorship program on the ability of public library staff to provide consumer health information

OBJECTIVE

In 2018, the Network of the National Libraries of Medicine (NNLM) launched a sponsorship program to support public library staff in completing the Medical Library Association's Consumer Health Information Specialization (CHIS). The objectives of our study were to: (1) determine whether completion of the sponsored specialization improved ability to provide consumer health information; (2) identify new health information services, programming, and outreach activities at public libraries; (3) investigate benefits of the specialization; and (4) determine the impact of sponsorship on obtaining and continuing the specialization.

METHODS

We used REDCap to administer a 16-question survey in August 2019 to 224 public library staff who were sponsored during the first year of the program. We measured competence in providing consumer health information aligned with the eight Core Competencies for Providing Consumer Health Information Services [1] as well as new activities at public libraries, benefits of the specialization to public library staff, career gains, and the likelihood of continuing the specialization based on funding.

RESULTS

More than 80% of 136 participants reported an increase in core consumer health competencies, with a statistically significant improvement in mean competency scores after completing the specialization. Ninety percent of participants have continued their engagement with NNLM, and more than half offered new health information programs and services. While more than half planned to renew the specialization or obtain the Level II specialization, 72% indicated they would not continue without NNLM sponsorship.

CONCLUSIONS

Findings indicate that NNLM sponsorship of the CHIS specialization was successful in increasing the capacity of public library staff to provide health information to their communities.

Automatic MeSH Indexing: Revisiting the Subheading Attachment Problem

This year less than 200 National Library of Medicine indexers expect to index 1 million articles, and this would not be possible without the assistance of the Medical Text Indexer (MTI) system. MTI is an automated indexing system that provides MeSH main heading/subheading pair recommendations to assist indexers with their heavy workload. Over the years, a lot of research effort has focused on improving main heading prediction performance, but automated fine-grained indexing with main heading/subheading pairs has received much less attention. This work revisits the subheading attachment problem, and demonstrates very significant performance improvements using modern Convolutional Neural Network classifiers. The best performing method is shown to outperform the current MTI implementation with a 3.7% absolute improvement in precision, and a 27.6% absolute improvement in recall. We also conducted a manual review of false positive predictions, and 70% were found to be acceptable indexing.

Database resources of the National Center for Biotechnology Information

The National Center for Biotechnology Information (NCBI) provides a large suite of online resources for biological information and data, including the GenBank® nucleic acid sequence database and the PubMed® database of citations and abstracts published in life science journals. The Entrez system provides search and retrieval operations for most of these data from 34 distinct databases. The E-utilities serve as the programming interface for the Entrez system. Custom implementations of the BLAST program provide sequence-based searching of many specialized datasets. New resources released in the past year include a new PubMed interface and NCBI datasets. Additional resources that were updated in the past year include PMC, Bookshelf, Genome Data Viewer, SRA, ClinVar, dbSNP, dbVar, Pathogen Detection, BLAST, Primer-BLAST, IgBLAST, iCn3D and PubChem. All of these resources can be accessed through the NCBI home page at https://www.ncbi.nlm.nih.gov.

GrantExtractor: Accurate Grant Support Information Extraction from Biomedical Fulltext Based on Bi-LSTM-CRF

Grant support (GS) in the MEDLINE database refers to funding agencies and contract numbers. It is important for funding organizations to track their funding outcomes from the GS information. As such, how to accurately and automatically extract funding information from biomedical literature is challenging. In this paper, we present a pipeline system called GrantExtractor that is able to accurately extract GS information from fulltext biomedical literature. GrantExtractor effectively integrates several advanced machine learning techniques. In particular, we use a sentence classifier to identify funding sentences from articles first. A bi-directional LSTM and the CRF layer (BiLSTM-CRF), and pattern matching are then used to extract entities of grant numbers and agencies from these identified funding sentences. After removing noisy numbers by a multi-class model, we finally match each grant number with its corresponding agency. Experimental results on benchmark datasets have demonstrated that GrantExtractor clearly outperforms all baseline methods. It is further evident that GrantExtractor won the first place in Task 5C of 2017 BioASQ challenge, with achieving the Micro-recall of 0.9526 for 22,610 articles. Moreover, GrantExtractor has achieved the Micro F-measure score as high as 0.90 in extracting grant pairs.

MedlinePlus at 21: A Website Devoted to Consumer Health Information

This report is about the consumer health website MedlinePlus.gov. The latter website was created by the U.S. National Library of Medicine and features content produced by the U.S. National Institutes of Health. The report provides an overview, origin, content, and possible future evolution of the website. The report also spotlights the specific features of the MedlinePlus health topic on health literacy and discusses the PubMed Topic Specific Query on health literacy.

A High Recall Classifier for Selecting Articles for MEDLINE Indexing

MEDLINE is the National Library of Medicine's premier bibliographic database for biomedical literature. A highly valuable feature of the database is that each record is manually indexed with a controlled vocabulary called MeSH. Most MEDLINE journals are indexed cover-to-cover, but there are about 200 selectively indexed journals for which only articles related to biomedicine and life sciences are indexed. In recent years, the selection process has become an increasing burden for indexing staff, and this paper presents a machine learning based system that offers very significant time savings by semi-automating the task. At the core of the system is a high recall classifier for the identification of journal articles that are in-scope for MEDLINE. The system is shown to reduce the number of articles requiring manual review by 54%, equivalent to approximately 40,000 articles per year.

Common Data Element for Unruptured Intracranial Aneurysm and Subarachnoid Hemorrhage: Recommendations from Assessments and Clinical Examination Workgroup/Subcommittee

BACKGROUND

Clinical studies of subarachnoid hemorrhage (SAH) and unruptured cerebral aneurysms lack uniformity in terms of variables used for assessments and clinical examination of patients which has led to difficulty in comparing studies and performing meta-analyses. The overall goal of the National Institute of Health/National Institute of Neurological Disorders and Stroke Unruptured Intracranial Aneurysms (UIA) and subarachnoid hemorrhage (SAH) Common Data Elements (CDE) Project was to provide common definitions and terminology for future unruptured intracranial aneurysm and SAH research.

METHODS

This paper summarizes the recommendations of the subcommittee on SAH Assessments and Clinical Examination. The subcommittee consisted of an international and multidisciplinary panel of experts in UIA and SAH. Consensus recommendations were developed by reviewing previously published CDEs for other neurological diseases including traumatic brain injury, epilepsy and stroke, and the SAH literature. Recommendations for CDEs were classified by priority into "core," "supplemental-highly recommended," "supplemental" and "exploratory."

RESULTS

We identified 248 variables for Assessments and Clinical Examination. Only the World Federation of Neurological Societies grading scale was classified as "Core." The Glasgow Coma Scale was classified as "Supplemental-Highly Recommended." All other Assessments and Clinical Examination variables were categorized as "Supplemental."

CONCLUSION

The recommended Assessments and Clinical Examination variables have been collated from a large number of potentially useful scales, history, clinical presentation, laboratory, and other tests. We hope that adherence to these recommendations will facilitate the comparison of results across studies and meta-analyses of individual patient data.

Common Data Elements for Radiological Imaging of Patients with Subarachnoid Hemorrhage: Proposal of a Multidisciplinary Research Group

INTRODUCTION

Lack of homogeneous definitions for imaging data and consensus on their relevance in the setting of subarachnoid hemorrhage and unruptured intracranial aneurysms lead to a difficulty of data pooling and lack of robust data. The aim of the National Institute of Health/National Institute of Neurological Disorders and Stroke, Unruptured Intracranial Aneurysm (UIA) and Subarachnoid Hemorrhage (SAH) Common Data Elements (CDE) Project was to standardize data elements to ultimately facilitate data pooling and establish a more robust data quality in future neurovascular research on UIA and SAH.

METHODS

For the subcommittee 'Radiological imaging of SAH,' international cerebrovascular specialists with imaging expertise in the setting of SAH were selected by the steering committee. CDEs were developed after reviewing the literature on neuroradiology and already existing CDEs for other neurological diseases. For prioritization, the CDEs were classified into 'Core,' 'Supplemental-Highly Recommended,' 'Supplemental' and 'Exploratory.'

RESULTS

The subcommittee compiled 136 CDEs, 100 out of which were derived from previously established CDEs on ischemic stroke and 36 were newly created. The CDEs were assigned to four main categories (several CDEs were assigned to more than one category): 'Parenchymal imaging' with 42 CDEs, 'Angiography' with 49 CDEs, 'Perfusion imaging' with 20 CDEs, and 'Transcranial doppler' with 55 CDEs. The CDEs were classified into core, supplemental highly recommended, supplemental and exploratory elements. The core CDEs were imaging modality, imaging modality type, imaging modality vessel, angiography type, vessel angiography arterial anatomic site and imaging vessel angiography arterial result.

CONCLUSIONS

The CDEs were established based on the current literature and consensus across cerebrovascular specialists. The use of these CDEs will facilitate standardization and aggregation of imaging data in the setting of SAH. However, the CDEs may require reevaluation and periodic adjustment based on current research and improved imaging quality and novel modalities.

NCBI Taxonomy: a comprehensive update on curation, resources and tools

The National Center for Biotechnology Information (NCBI) Taxonomy includes organism names and classifications for every sequence in the nucleotide and protein sequence databases of the International Nucleotide Sequence Database Collaboration. Since the last review of this resource in 2012, it has undergone several improvements. Most notable is the shift from a single SQL database to a series of linked databases tied to a framework of data called NameBank. This means that relations among data elements can be adjusted in more detail, resulting in expanded annotation of synonyms, the ability to flag names with specific nomenclatural properties, enhanced tracking of publications tied to names and improved annotation of scientific authorities and types. Additionally, practices utilized by NCBI Taxonomy curators specific to major taxonomic groups are described, terms peculiar to NCBI Taxonomy are explained, external resources are acknowledged and updates to tools and other resources are documented. Database URL: https://www.ncbi.nlm.nih.gov/taxonomy.

Tea Compounds and the Gut Microbiome: Findings from Trials and Mechanistic Studies

In recent years, the gut microbiome has become a focal point of interest with growing recognition that a well-balanced gut microbiota composition is highly relevant to an individual’s health status and well-being. Its profile can be modulated by a number of dietary factors, although few publications have focused on the effects of what we drink. The present review performed a systematic review of trials and mechanistic studies examining the effects of tea consumption, its associated compounds and their effects on the gut microbiome. Registered articles were searched up to 10th September 2019, in the PubMed and Cochrane library databases along with references of original articles. Human trials were graded using the Jadad scale to assess quality. Altogether 24 publications were included in the main review—six were human trials and 18 mechanistic studies. Of these, the largest body of evidence related to green tea with up to 1000 mL daily (4–5 cups) reported to increase proportions of Bifidobacterium. Mechanistic studies also show promise suggesting that black, oolong, Pu-erh and Fuzhuan teas (microbially fermented ‘dark tea’) can modulate microbial diversity and the ratio of Firmicutes to Bacteroidetes. These findings appear to support the hypothesis that tea ingestion could favourably regulate the profile of the gut microbiome and help to offset dysbiosis triggered by obesity or high-fat diets. Further well-designed human trials are now required to build on provisional findings.

Database resources of the National Center for Biotechnology Information

The National Center for Biotechnology Information (NCBI) provides a large suite of online resources for biological information and data, including the GenBank^® nucleic acid sequence database and the PubMed database of citations and abstracts for published life science journals. The Entrez system provides search and retrieval operations for most of these data from 39 distinct databases. The E-utilities serve as the programming interface for the Entrez system. Augmenting many of the Web applications are custom implementations of the BLAST program optimized to search specialized data sets. New resources released in the past year include PubMed Data Management, RefSeq Functional Elements, genome data download, variation services API, Magic-BLAST, QuickBLASTp, and Identical Protein Groups. Resources that were updated in the past year include the genome data viewer, a human genome resources page, Gene, virus variation, OSIRIS, and PubChem. All of these resources can be accessed through the NCBI home page at www.ncbi.nlm.nih.gov.

"Plutchik": artificial intelligence chatbot for searching NCBI databases

As genetic testing gains ground in medicine, the ability to search across the suite of biomedical and clinical care databases offered through the National Library of Medicine/National Center for Biotechnology Information (NCBI)-such as PubMed, GENE, Structure, the Genetic Testing Registry, and others-holds the potential to enhance quality of clinical care best practices. "Plutchik" is a voice-enabled, embodied artificial intelligence (AI) chatbot that can perform highly technical medical searches in and across the NCBI suite of databases.

Challenges in Developing Analytically Validated Laboratory-Derived Dietary Supplement Databases

The Dietary Supplement Label Database (DSLD) is sponsored by the Office of Dietary Supplements (ODS) and the National Library of Medicine (NLM). It provides a searchable, free database of the contents of ∼65,000 supplement labels. A companion database of analytically verified product labels [the Dietary Supplement Ingredient Database (DSID)] was created by ODS, NLM, and the USDA. There are considerable challenges to populating both databases, but the DSID faces unique analytic chemistry challenges. This article describes the challenges to creating analytically verified marketplace surveys of dietary supplement (DS) product content claims for inclusion in public databases. Nutritionists and public health scientists require information on actual exposures to DS constituents because labeled content may not match labeled product content. Analytic verification of composition of DSs provides a link to actual exposure. A public database of analytically derived DS content was developed to provide more accurate estimates of dietary intake in population-based epidemiologic studies. The DSID has conducted surveys of several types of vitamin- and mineral-containing DSs. Results showing label content claims as analytically derived values are available in the current DSID. A recent pilot project explored the feasibility of adding botanical DS products to the DSID. Candidates for future botanical DSID studies will be based on sales volume, potential public health impacts, and the availability of validated analytic methods and reference materials. Databases like DSID and the DSLD are essential for researchers and clinicians to evaluate dietary ingredient intakes in population-based epidemiologic studies. Together, these databases provide a picture of the DS marketplace. The DSID provides an analytic survey of marketed DSs. However, selection of future botanical supplements for DSID evaluation involves analytic challenges. Even when appropriate resources are available, method selection and data evaluation are resource- and time-consuming.

Genetics Home Reference: A Review

Genetics Home Reference is a free, online resource created and maintained by the National Library of Medicine. It is designed to provide genetic information to a wide variety of audiences, particularly the general public. The site consists of original information and links to other curated resources.

Health sciences library outreach to family caregivers: a call to service

This commentary discusses the information needs of family caregivers and care recipients in the United States. Health sciences library services and outreach activities that support family caregivers include: (1) advocacy, (2) resource building, and (3) programming and education. Ethical issues related to the privacy and confidentiality of clients are outlined in the commentary for information service providers. Also, continuing professional education resources are identified to assist librarians in providing high-quality information services for this special family caregiver population, such as those designed by the National Library of Medicine (NLM) through the NLM 4 Caregivers program.

The NCBI BioCollections Database

The rapidly growing set of GenBank submissions includes sequences that are derived from vouchered specimens. These are associated with culture collections, museums, herbaria and other natural history collections, both living and preserved. Correct identification of the specimens studied, along with a method to associate the sample with its institution, is critical to the outcome of related studies and analyses. The National Center for Biotechnology Information BioCollections Database was established to allow the association of specimen vouchers and related sequence records to their home institutions. This process also allows cross-linking from the home institution for quick identification of all records originating from each collection. Database URL: https://www.ncbi.nlm.nih.gov/biocollections

Native Voices: Native Peoples’ Concepts of Health and Illness in New Mexico: opening a local conversation by hosting a national traveling exhibit

Background: The University of New Mexico Health Sciences Library and Informatics Center hosted the National Library of Medicine’s Native Voices: Native Peoples’ Concepts of Health and Illness traveling exhibit. The authors’ goal was to promote local interest in the Native Voices exhibit, with an emphasis on making the exhibit content and materials available to American Indian communities throughout rural New Mexico.Case Presentation: We convened a daylong summit to highlight the exhibit and encourage discussion among 30 American Indian community health educators. The summit prompted the compilation and distribution of descriptions of 23 community projects that promote health and wellness. We also took a scaled-down version of the exhibit to 4 rural college campuses around the state that serve significant Native American student populations. Approximately 140 students and faculty interacted with the exhibit materials, and all 4 sites incorporated the exhibit into curriculum activities.Conclusions: A hosted national exhibit developed into a multifaceted, funded project that engaged with Native American communities. We demonstrated successful field deployment of a downsized, portable version of the full traveling exhibit to make meaningful connections with members of our outreach population.

Wide-Open: Accelerating public data release by automating detection of overdue datasets

Open data is a vital pillar of open science and a key enabler for reproducibility, data reuse, and novel discoveries. Enforcement of open-data policies, however, largely relies on manual efforts, which invariably lag behind the increasingly automated generation of biological data. To address this problem, we developed a general approach to automatically identify datasets overdue for public release by applying text mining to identify dataset references in published articles and parse query results from repositories to determine if the datasets remain private. We demonstrate the effectiveness of this approach on 2 popular National Center for Biotechnology Information (NCBI) repositories: Gene Expression Omnibus (GEO) and Sequence Read Archive (SRA). Our Wide-Open system identified a large number of overdue datasets, which spurred administrators to respond directly by releasing 400 datasets in one week.

Determining the accuracy of open-access databases for identifying commonly prescribed oral medications

OBJECTIVES

To compare the accuracy of open-access medication identification databases in identifying commonly prescribed oral tablets and capsules.

METHODS

We compiled a dataset from the most frequently prescribed medications in 2012 and 2013 by total prescription volume. Imprints for the medications were obtained from the Food and Drug Administration label information accessed via the National Library of Medicine's (NLM) DailyMed. The imprints were used to determine if the correct medication could be identified in the open-access medication identification databases Drugs.com, Healthline, NLM's Pillbox, RxList, and WebMD by means of the imprints alone. A general web search using the imprints as key words via Google search engine was also conducted to mimic the lay public's approach to identifying an unknown medication.

RESULTS

Database accuracy of search results ranged from 26.5% to 89.27% with the NLM Pillbox having the highest accuracy when using tablet/capsule imprint information alone. The secondary web search with the use of the Google search engine yielded 75.7% accuracy in using imprints as search terms to identify websites leading to the appropriate drugs. Drugs.com was listed as the first site for the majority of Google searches and was 86.44% accurate.

CONCLUSION

Open-access databases are available to help identify oral tablets and capsules and they have a relatively high success rate of identifying commonly prescribed medications. However no database is 100% accurate, which means that the risk involved with patients identifying their own medications is substantial. Patients should be highly encouraged by all health care practitioners to consult their pharmacist or primary health care provider to prevent misidentification and subsequent misadministration.

Anticipating the third century of the National Library of Medicine: a call for engagement

What is the foundation of the NLM of 2036 that will ensure that it continues to serve medicine, health care, discovery, and citizens?

Improving data collection, documentation, and workflow in a dementia screening study

BACKGROUND

A clinical study team performing three multicultural dementia screening studies identified the need to improve data management practices and facilitate data sharing. A collaboration was initiated with librarians as part of the National Library of Medicine (NLM) informationist supplement program. The librarians identified areas for improvement in the studies' data collection, entry, and processing workflows.

CASE PRESENTATION

The librarians' role in this project was to meet needs expressed by the study team around improving data collection and processing workflows to increase study efficiency and ensure data quality. The librarians addressed the data collection, entry, and processing weaknesses through standardizing and renaming variables, creating an electronic data capture system using REDCap, and developing well-documented, reproducible data processing workflows.

CONCLUSIONS

NLM informationist supplements provide librarians with valuable experience in collaborating with study teams to address their data needs. For this project, the librarians gained skills in project management, REDCap, and understanding of the challenges and specifics of a clinical research study. However, the time and effort required to provide targeted and intensive support for one study team was not scalable to the library's broader user community.