Systematic Evaluation of Research Progress on Natural Language Processing in Medicine Over the Past 20 Years: Bibliometric Study on PubMed

Concept Recognition and Characterization of Patients Undergoing Resection of Vestibular Schwannoma Using Natural Language Processing

Background  Natural language processing (NLP), a subset of artificial intelligence (AI), aims to decipher unstructured human language. This study showcases NLP's application in surgical health care, focusing on vestibular schwannoma (VS). By employing an NLP platform, we identify prevalent text concepts in VS patients' electronic health care records (EHRs), creating concept panels covering symptomatology, comorbidities, and management. Through a case study, we illustrate NLP's potential in predicting postoperative cerebrospinal fluid (CSF) leaks. Methods  An NLP model analyzed EHRs of surgically managed VS patients from 2008 to 2018 in a single center. The model underwent unsupervised (trained on one million documents from EHR) and supervised (300 documents annotated in duplicate) learning phases, extracting text concepts and generating concept panels related to symptoms, comorbidities, and management. Statistical analysis correlated concept occurrences with postoperative complications, notably CSF leaks. Results  Analysis included 292 patients' records, yielding 6,901 unique concepts and 360,929 occurrences. Concept panels highlighted key associations with postoperative CSF leaks, including "antibiotics," "sepsis," and "intensive care unit admission." The NLP model demonstrated high accuracy (precision 0.92, recall 0.96, macro F1 0.93). Conclusion  Our NLP model effectively extracted concepts from VS patients' EHRs, facilitating personalized concept panels with diverse applications. NLP shows promise in surgical settings, aiding in early diagnosis, complication prediction, and patient care. Further validation of NLP's predictive capabilities is warranted.

Perceived Impact of COVID-19 in an Underserved Community: A Natural Language Processing Approach

AIM

To utilise natural language processing (NLP) to analyse interviews about the impact of COVID-19 in underserved communities and to compare it to traditional thematic analysis in a small subset of interviews.

DESIGN

NLP and thematic analysis were used together to comprehensively examine the interview data.

METHODS

Fifty transcribed interviews with purposively sampled adults living in underserved communities in the United States, conducted from June 2021 to May 2022, were analysed to explore the impact of the COVID-19 pandemic on social activities, mental and emotional stress and physical and spiritual well-being. NLP includes several stages: data extraction, preprocessing, processing using word embeddings and topic modelling and visualisation. This was compared to thematic analysis in a random sample of 10 interviews.

RESULTS

Six themes emerged from thematic analysis: The New Normal, Juxtaposition of Emotions, Ripple Effects on Health, Brutal yet Elusive Reality, Evolving Connections and Journey of Spirituality and Self-Realisation. With NLP, four clusters of similar context words for each approach were analysed visually and numerically. The frequency-based word embedding approach was most interpretable and well aligned with the thematic analysis.

CONCLUSION

The NLP results complemented the thematic analysis and offered new insights regarding the passage of time, the interconnectedness of impacts and the semantic connections among words. This research highlights the interdependence of pandemic impacts, simultaneously positive and negative effects and deeply individual COVID-19 experiences in underserved communities.

IMPLICATIONS

The iterative integration of NLP and thematic analysis was efficient and effective, facilitating the analysis of many transcripts and expanding nursing research methodology.

IMPACT

While thematic analysis provided richer, more detailed themes, NLP captured new elements and combinations of words, making it a promising tool in qualitative analysis.

REPORTING METHOD

Not applicable.

PATIENT OR PUBLIC CONTRIBUTION

No patient or public contribution.

Accuracy of ChatGPT 3.5, 4.0, 4o and Gemini in diagnosing oral potentially malignant lesions based on clinical case reports and image recognition

BACKGROUND

The accurate and timely diagnosis of oral potentially malignant lesions (OPMLs) is crucial for effective management and prevention of oral cancer. Recent advancements in artificial intelligence technologies indicates its potential to assist in clinical decision-making. Hence, this study was carried out with the aim to evaluate and compare the diagnostic accuracy of ChatGPT 3.5, 4.0, 4o and Gemini in identifying OPMLs.

MATERIAL AND METHODS

The analysis was carried out using 42 case reports from PubMed, Scopus and Google Scholar and images from two datasets, corresponding to different OPMLs. The reports were inputted separately for text description-based diagnosis in GPT 3.5, 4.0, 4o and Gemini, and for image recognition-based diagnosis in GPT 4o and Gemini. Two subject-matter experts independently reviewed the reports and offered their evaluations.

RESULTS

For text-based diagnosis, among LLMs, GPT 4o got the maximum number of correct responses (27/42), followed by GPT 4.0 (20/42), GPT 3.5 (18/42) and Gemini (15/42). In identifying OPMLs based on image, GPT 4o demonstrated better performance than Gemini. There was fair to moderate agreement found between Large Language Models (LLMs) and subject experts. None of the LLMs matched the accuracy of the subject experts in identifying the correct number of lesions.

CONCLUSIONS

The results point towards cautious optimism with respect to commonly used LLMs in diagnosing OPMLs. While their potential in diagnostic applications is undeniable, their integration should be approached judiciously.

Emerging applications of NLP and large language models in gastroenterology and hepatology: a systematic review

Background and aim

In the last years, natural language processing (NLP) has transformed significantly with the introduction of large language models (LLM). This review updates on NLP and LLM applications and challenges in gastroenterology and hepatology.

Methods

Registered with PROSPERO (CRD42024542275) and adhering to PRISMA guidelines, we searched six databases for relevant studies published from 2003 to 2024, ultimately including 57 studies.

Results

Our review of 57 studies notes an increase in relevant publications in 2023-2024 compared to previous years, reflecting growing interest in newer models such as GPT-3 and GPT-4. The results demonstrate that NLP models have enhanced data extraction from electronic health records and other unstructured medical data sources. Key findings include high precision in identifying disease characteristics from unstructured reports and ongoing improvement in clinical decision-making. Risk of bias assessments using ROBINS-I, QUADAS-2, and PROBAST tools confirmed the methodological robustness of the included studies.

Conclusion

NLP and LLMs can enhance diagnosis and treatment in gastroenterology and hepatology. They enable extraction of data from unstructured medical records, such as endoscopy reports and patient notes, and for enhancing clinical decision-making. Despite these advancements, integrating these tools into routine practice is still challenging. Future work should prospectively demonstrate real-world value.

Bibliometric analysis of teen pregnancy research in Asia-Africa: Explore the future scope

BACKGROUND

Adolescent pregnancy is prevalent in Asian-African countries. Hence, it is critical to track the progress of research and development trends related to the topic. The study aimed to characterize published articles on teen pregnancy by measuring the authors' country and affiliation, most relevant and cited journals, thematic research, and growth trends.

MATERIALS AND METHODS

Descriptive statistics and retrospective bibliometric analysis were used. Using the Scopus database, we collected published articles from 2010 to 2023. Titles and abstracts were screened. Eligible papers were reviewed based on co-occurrence analysis. Classification and visualization of results were conducted using VOSviewer software version 1.6.17.

RESULTS

A total of 369 articles were relevant and included in the review. The countries that produced the most publications were South Africa and the United States. The most relevant affiliation was with the University of Cape Town, the University of the Witwatersrand, and the University of Kwazulu-Natal. The most relevant source journals were PLOS One, Reproductive Health, and BMC Public Health. The most cited sources were Lancet, PLOS One, and AIDS Journal. Four clusters were obtained that reflect the main topics, including human immunodeficiency virus (HIV) infection, child, health services, and pregnancy. The current research terms include HIV infection and maternal health services.

CONCLUSIONS

This study suggests more research on the term "vertical transmission, breastfeeding, and partner violence" related to teen pregnancy. In addition, this research will inspire researchers and adolescent health policymakers to expand the scope of research to solve teenage pregnancy problems based on interdisciplinary theories and methods.

Contrasting rule and machine learning based digital self triage systems in the USA

Patient smart access and self-triage systems have been in development for decades. As of now, no LLM for processing self-reported patient data has been published by health systems. Many expert systems and computational models have been released to millions. This review is the first to summarize progress in the field including an analysis of the exact self-triage solutions available on the websites of 647 health systems in the USA.

Chatbots as Patient Education Resources for Aesthetic Facial Plastic Surgery: Evaluation of ChatGPT and Google Bard Responses

Background: ChatGPT and Google Bard™ are popular artificial intelligence chatbots with utility for patients, including those undergoing aesthetic facial plastic surgery. Objective: To compare the accuracy and readability of chatbot-generated responses to patient education questions regarding aesthetic facial plastic surgery using a response accuracy scale and readability testing. Method: ChatGPT and Google Bard™ were asked 28 identical questions using four prompts: none, patient friendly, eighth-grade level, and references. Accuracy was assessed using Global Quality Scale (range: 1-5). Flesch-Kincaid grade level was calculated, and chatbot-provided references were analyzed for veracity. Results: Although 59.8% of responses were good quality (Global Quality Scale ≥4), ChatGPT generated more accurate responses than Google Bard™ on patient-friendly prompting (p < 0.001). Google Bard™ responses were of a significantly lower grade level than ChatGPT for all prompts (p < 0.05). Despite eighth-grade prompting, response grade level for both chatbots was high: ChatGPT (10.5 ± 1.8) and Google Bard™ (9.6 ± 1.3). Prompting for references yielded 108/108 of chatbot-generated references. Forty-one (38.0%) citations were legitimate. Twenty (18.5%) provided accurately reported information from the reference. Conclusion: Although ChatGPT produced more accurate responses and at a higher education level than Google Bard™, both chatbots provided responses above recommended grade levels for patients and failed to provide accurate references.

Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluation

OBJECTIVE

The paper introduces a framework for the evaluation of the encoding of factual scientific knowledge, designed to streamline the manual evaluation process typically conducted by domain experts. Inferring over and extracting information from Large Language Models (LLMs) trained on a large corpus of scientific literature can potentially define a step change in biomedical discovery, reducing the barriers for accessing and integrating existing medical evidence. This work explores the potential of LLMs for dialoguing with biomedical background knowledge, using the context of antibiotic discovery.

METHODS

The framework involves three evaluation steps, each assessing different aspects sequentially: fluency, prompt alignment, semantic coherence, factual knowledge, and specificity of the generated responses. By splitting these tasks between non-experts and experts, the framework reduces the effort required from the latter. The work provides a systematic assessment on the ability of eleven state-of-the-art LLMs, including ChatGPT, GPT-4 and Llama 2, in two prompting-based tasks: chemical compound definition generation and chemical compound-fungus relation determination.

RESULTS

Although recent models have improved in fluency, factual accuracy is still low and models are biased towards over-represented entities. The ability of LLMs to serve as biomedical knowledge bases is questioned, and the need for additional systematic evaluation frameworks is highlighted.

CONCLUSION

While LLMs are currently not fit for purpose to be used as biomedical factual knowledge bases in a zero-shot setting, there is a promising emerging property in the direction of factuality as the models become domain specialised, scale up in size and level of human feedback.

Advancing rheumatology with natural language processing: insights and prospects from a systematic review

Objectives

Natural language processing (NLP) and large language models (LLMs) have emerged as powerful tools in healthcare, offering advanced methods for analysing unstructured clinical texts. This systematic review aims to evaluate the current applications of NLP and LLMs in rheumatology, focusing on their potential to improve disease detection, diagnosis and patient management.

Methods

We screened seven databases. We included original research articles that evaluated the performance of NLP models in rheumatology. Data extraction and risk of bias assessment were performed independently by two reviewers, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies was used to evaluate the risk of bias.

Results

Of 1491 articles initially identified, 35 studies met the inclusion criteria. These studies utilized various data types, including electronic medical records and clinical notes, and employed models like Bidirectional Encoder Representations from Transformers and Generative Pre-trained Transformers. High accuracy was observed in detecting conditions such as RA, SpAs and gout. The use of NLP also showed promise in managing diseases and predicting flares.

Conclusion

NLP showed significant potential in enhancing rheumatology by improving diagnostic accuracy and personalizing patient care. While applications in detecting diseases like RA and gout are well developed, further research is needed to extend these technologies to rarer and more complex clinical conditions. Overcoming current limitations through targeted research is essential for fully realizing NLP's potential in clinical practice.

Gender Differences in Letters of Recommendations and Personal Statements for Neurotology Fellowship over 10 Years: A Deep Learning Linguistic Analysis

OBJECTIVE

Personal statements (PSs) and letters of recommendation (LORs) are critical components of the neurotology fellowship application process but can be subject to implicit biases. This study evaluated general and deep learning linguistic differences between the applicant genders over a 10-year span.

STUDY DESIGN

Retrospective cohort.

SETTING

Two institutions.

MAIN OUTCOME MEASURES

PSs and LORs were collected from 2014 to 2023 from two institutions. The Valence Aware Dictionary and Sentiment Reasoner (VADER) natural language processing (NLP) package was used to compare the positive or negative sentiment in LORs and PSs. Next, the deep learning tool, Empath, categorized the text into scores, and Wilcoxon rank sum tests were performed for comparisons between applicant gender.

RESULTS

Among 177 applicants over 10 years, 120 were males and 57 were females. There were no differences in word count or VADER sentiment scores between genders for both LORs and PSs. However, among Empath sentiment categories, male applicants had more words of trust ( p = 0.03) and leadership ( p = 0.002) in LORs. Temporally, the trends show a consistently higher VADER sentiment and Empath "trust" and "leader" in male LORs from 2014 to 2019, after which there was no statistical significance in sentiment scores between genders, and females even have higher scores of trust and leadership in 2023.

CONCLUSIONS

Linguistic content overall favored male applicants because they were more frequently described as trustworthy and leaders. However, the temporal analysis of linguistic differences between male and female applicants found an encouraging trend suggesting a reduction of gender bias in recent years, mirroring an increased composition of women in neurotology over time.

Automated information extraction model enhancing traditional Chinese medicine RCT evidence extraction (Evi-BERT): algorithm development and validation

Background

In the field of evidence-based medicine, randomized controlled trials (RCTs) are of critical importance for writing clinical guidelines and providing guidance to practicing physicians. Currently, RCTs rely heavily on manual extraction, but this method has data breadth limitations and is less efficient.

Objectives

To expand the breadth of data and improve the efficiency of obtaining clinical evidence, here, we introduce an automated information extraction model for traditional Chinese medicine (TCM) RCT evidence extraction.

Methods

We adopt the Evidence-Bidirectional Encoder Representation from Transformers (Evi-BERT) for automated information extraction, which is combined with rule extraction. Eleven disease types and 48,523 research articles from the China National Knowledge Infrastructure (CNKI), WanFang Data, and VIP databases were selected as the data source for extraction. We then constructed a manually annotated dataset of TCM clinical literature to train the model, including ten evidence elements and 24,244 datapoints. We chose two models, BERT-CRF and BiLSTM-CRF, as the baseline, and compared the training effects with Evi-BERT and Evi-BERT combined with rule expression (RE).

Results

We found that Evi-BERT combined with RE achieved the best performance (precision score = 0.926, Recall = 0.952, F1 score = 0.938) and had the best robustness. We totally summarized 113 pieces of rule datasets in the regulation extraction procedure. Our model dramatically expands the amount of data that can be searched and greatly improves efficiency without losing accuracy.

Conclusion

Our work provided an intelligent approach to extracting clinical evidence for TCM RCT data. Our model can help physicians reduce the time spent reading journals and rapidly speed up the screening of clinical trial evidence to help generate accurate clinical reference guidelines. Additionally, we hope the structured clinical evidence and structured knowledge extracted from this study will help other researchers build large language models in TCM.

Biosimilars in the Era of Artificial Intelligence-International Regulations and the Use in Oncological Treatments

This research is based on three fundamental aspects of successful biosimilar development in the challenging biopharmaceutical market. First, biosimilar regulations in eight selected countries: Japan, South Korea, the United States, Canada, Brazil, Argentina, Australia, and South Africa, represent the four continents. The regulatory aspects of the countries studied are analyzed, highlighting the challenges facing biosimilars, including their complex approval processes and the need for standardized regulatory guidelines. There is an inconsistency depending on whether the biosimilar is used in a developed or developing country. In the countries observed, biosimilars are considered excellent alternatives to patent-protected biological products for the treatment of chronic diseases. In the second aspect addressed, various analytical AI modeling methods (such as machine learning tools, reinforcement learning, supervised, unsupervised, and deep learning tools) were analyzed to observe patterns that lead to the prevalence of biosimilars used in cancer to model the behaviors of the most prominent active compounds with spectroscopy. Finally, an analysis of the use of active compounds of biosimilars used in cancer and approved by the FDA and EMA was proposed.

Undercounting diagnoses in Australian general practice: a data quality study with implications for population health reporting

BACKGROUND

Diagnosis can often be recorded in electronic medical records (EMRs) as free-text or using a term with a diagnosis code. Researchers, governments, and agencies, including organisations that deliver incentivised primary care quality improvement programs, frequently utilise coded data only and often ignore free-text entries. Diagnosis data are reported for population healthcare planning including resource allocation for patient care. This study sought to determine if diagnosis counts based on coded diagnosis data only, led to under-reporting of disease prevalence and if so, to what extent for six common or important chronic diseases.

METHODS

This cross-sectional data quality study used de-identified EMR data from 84 general practices in Victoria, Australia. Data represented 456,125 patients who attended one of the general practices three or more times in two years between January 2021 and December 2022. We reviewed the percentage and proportional difference between patient counts of coded diagnosis entries alone and patient counts of clinically validated free-text entries for asthma, chronic kidney disease, chronic obstructive pulmonary disease, dementia, type 1 diabetes and type 2 diabetes.

RESULTS

Undercounts were evident in all six diagnoses when using coded diagnoses alone (2.57-36.72% undercount), of these, five were statistically significant. Overall, 26.4% of all patient diagnoses had not been coded. There was high variation between practices in recording of coded diagnoses, but coding for type 2 diabetes was well captured by most practices.

CONCLUSION

In Australia clinical decision support and the reporting of aggregated patient diagnosis data to government that relies on coded diagnoses can lead to significant underreporting of diagnoses compared to counts that also incorporate clinically validated free-text diagnoses. Diagnosis underreporting can impact on population health, healthcare planning, resource allocation, and patient care. We propose the use of phenotypes derived from clinically validated text entries to enhance the accuracy of diagnosis and disease reporting. There are existing technologies and collaborations from which to build trusted mechanisms to provide greater reliability of general practice EMR data used for secondary purposes.

Examining the role of artificial intelligence to advance knowledge and address barriers to research in eating disorders

OBJECTIVE

To provide a brief overview of artificial intelligence (AI) application within the field of eating disorders (EDs) and propose focused solutions for research.

METHOD

An overview and summary of AI application pertinent to EDs with focus on AI's ability to address issues relating to data sharing and pooling (and associated privacy concerns), data augmentation, as well as bias within datasets is provided.

RESULTS

In addition to clinical applications, AI can utilize useful tools to help combat commonly encountered challenges in ED research, including issues relating to low prevalence of specific subpopulations of patients, small overall sample sizes, and bias within datasets.

DISCUSSION

There is tremendous potential to embed and utilize various facets of artificial intelligence (AI) to help improve our understanding of EDs and further evaluate and investigate questions that ultimately seek to improve outcomes. Beyond the technology, issues relating to regulation of AI, establishing ethical guidelines for its application, and the trust of providers and patients are all needed for ultimate adoption and acceptance into ED practice.

PUBLIC SIGNIFICANCE

Artificial intelligence (AI) offers a promise of significant potential within the realm of eating disorders (EDs) and encompasses a broad set of techniques that offer utility in various facets of ED research and by extension delivery of clinical care. Beyond the technology, issues relating to regulation, establishing ethical guidelines for application, and the trust of providers and patients are needed for the ultimate adoption and acceptance of AI into ED practice.

Clinical named entity recognition for percutaneous coronary intervention surgical information with hybrid neural network

Percutaneous coronary intervention (PCI) has become a vital treatment approach for coronary artery disease, but the clinical data of PCI cannot be directly utilized due to its unstructured characteristics. The existing clinical named entity recognition (CNER) has been used to identify specific entities such as body parts, drugs, and diseases, but its specific potential in PCI clinical texts remains largely unexplored. How to effectively use CNER to deeply mine the information in the existing PCI clinical records is worth studying. In this paper, a total of 24 267 corpora are collected from the Cardiovascular Disease Treatment Center of the People's Hospital of Liaoning Province in China. We select three types of clinical record texts of fine-grained PCI surgical information, from which 5.8% of representative surgical records of PCI patients are selected as datasets for labeling. To fully utilize global information and multi-level semantic features, we design a novel character-level vector embedding method and further propose a new hybrid model based on it. Based on the classic Bidirectional Long Short-Term Memory Network (BiLSTM), the model further integrates Convolutional Neural Networks (CNNs) and Bidirectional Encoder Representations from Transformers (BERTs) for feature extraction and representation, and finally uses Conditional Random Field (CRF) for decoding and predicting label sequences. This hybrid model is referred to as BCC-BiLSTM in this paper. In order to verify the performance of the proposed hybrid model for extracting PCI surgical information, we simultaneously compare both representative traditional and intelligent methods. Under the same circumstances, compared with other intelligent methods, the BCC-BiLSTM proposed in this paper reduces the word vector dimension by 15%, and the F1 score reaches 86.2% in named entity recognition of PCI clinical texts, which is 26.4% higher than that of HMM. The improvement is 1.2% higher than BiLSTM + CRF and 0.7% higher than the most popular BERT + BiLSTM + CRF. Compared with the representative models, the hybrid model has better performance and can achieve optimal results faster in the model training process, so it has good clinical application prospects.

Natural Language Processing for Work-Related Stress Detection Among Health Professionals: Protocol for a Scoping Review

BACKGROUND

There is an urgent need worldwide for qualified health professionals. High attrition rates among health professionals, combined with a predicted rise in life expectancy, further emphasize the need for additional health professionals. Work-related stress is a major concern among health professionals, affecting both the well-being of health professionals and the quality of patient care.

OBJECTIVE

This scoping review aims to identify processes and methods for the automatic detection of work-related stress among health professionals using natural language processing (NLP) and text mining techniques.

METHODS

This review follows Joanna Briggs Institute Methodology and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. The inclusion criteria for this scoping review encompass studies involving health professionals using NLP for work-related stress detection while excluding studies involving other professions or children. The review focuses on various aspects, including NLP applications for stress detection, criteria for stress identification, technical aspects of NLP, and implications of stress detection through NLP. Studies within health care settings using diverse NLP techniques are considered, including experimental and observational designs, aiming to provide a comprehensive understanding of NLP's role in detecting stress among health professionals. Studies published in English, German, or French from 2013 to present will be considered. The databases to be searched include MEDLINE (via PubMed), CINAHL, PubMed, Cochrane, ACM Digital Library, and IEEE Xplore. Sources of unpublished studies and gray literature to be searched will include ProQuest Dissertations & Theses and OpenGrey. Two reviewers will independently retrieve full-text studies and extract data. The collected data will be organized in tables, graphs, and a qualitative narrative summary. This review will use tables and graphs to present data on studies' distribution by year, country, activity field, and research methods. Results synthesis involves identifying, grouping, and categorizing. The final scoping review will include a narrative written report detailing the search and study selection process, a visual representation using a PRISMA-ScR flow diagram, and a discussion of implications for practice and research.

RESULTS

We anticipate the outcomes will be presented in a systematic scoping review by June 2024.

CONCLUSIONS

This review fills a literature gap by identifying automated work-related stress detection among health professionals using NLP and text mining, providing insights on an innovative approach, and identifying research needs for further systematic reviews. Despite promising outcomes, acknowledging limitations in the reviewed studies, including methodological constraints, sample biases, and potential oversight, is crucial to refining methodologies and advancing automatic stress detection among health professionals.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/56267.

Natural language processing (NLP) to facilitate abstract review in medical research: the application of BioBERT to exploring the 20-year use of NLP in medical research

BACKGROUND

Abstract review is a time and labor-consuming step in the systematic and scoping literature review in medicine. Text mining methods, typically natural language processing (NLP), may efficiently replace manual abstract screening. This study applies NLP to a deliberately selected literature review problem, the trend of using NLP in medical research, to demonstrate the performance of this automated abstract review model.

METHODS

Scanning PubMed, Embase, PsycINFO, and CINAHL databases, we identified 22,294 with a final selection of 12,817 English abstracts published between 2000 and 2021. We invented a manual classification of medical fields, three variables, i.e., the context of use (COU), text source (TS), and primary research field (PRF). A training dataset was developed after reviewing 485 abstracts. We used a language model called Bidirectional Encoder Representations from Transformers to classify the abstracts. To evaluate the performance of the trained models, we report a micro f1-score and accuracy.

RESULTS

The trained models' micro f1-score for classifying abstracts, into three variables were 77.35% for COU, 76.24% for TS, and 85.64% for PRF. The average annual growth rate (AAGR) of the publications was 20.99% between 2000 and 2020 (72.01 articles (95% CI: 56.80-78.30) yearly increase), with 81.76% of the abstracts published between 2010 and 2020. Studies on neoplasms constituted 27.66% of the entire corpus with an AAGR of 42.41%, followed by studies on mental conditions (AAGR = 39.28%). While electronic health or medical records comprised the highest proportion of text sources (57.12%), omics databases had the highest growth among all text sources with an AAGR of 65.08%. The most common NLP application was clinical decision support (25.45%).

CONCLUSIONS

BioBERT showed an acceptable performance in the abstract review. If future research shows the high performance of this language model, it can reliably replace manual abstract reviews.

Biomedical text readability after hypernym substitution with fine-tuned large language models

The advent of patient access to complex medical information online has highlighted the need for simplification of biomedical text to improve patient understanding and engagement in taking ownership of their health. However, comprehension of biomedical text remains a difficult task due to the need for domain-specific expertise. We aimed to study the simplification of biomedical text via large language models (LLMs) commonly used for general natural language processing tasks involve text comprehension, summarization, generation, and prediction of new text from prompts. Specifically, we finetuned three variants of large language models to perform substitutions of complex words and word phrases in biomedical text with a related hypernym. The output of the text substitution process using LLMs was evaluated by comparing the pre- and post-substitution texts using four readability metrics and two measures of sentence complexity. A sample of 1,000 biomedical definitions in the National Library of Medicine's Unified Medical Language System (UMLS) was processed with three LLM approaches, and each showed an improvement in readability and sentence complexity after hypernym substitution. Readability scores were translated from a pre-processed collegiate reading level to a post-processed US high-school level. Comparison between the three LLMs showed that the GPT-J-6b approach had the best improvement in measures of sentence complexity. This study demonstrates the merit of hypernym substitution to improve readability of complex biomedical text for the public and highlights the use case for fine-tuning open-access large language models for biomedical natural language processing.

Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain-adaptive pre-trained large language model

BACKGROUND

Stroke is a prevalent disease with a significant global impact. Effective assessment of stroke severity is vital for an accurate diagnosis, appropriate treatment, and optimal clinical outcomes. The National Institutes of Health Stroke Scale (NIHSS) is a widely used scale for quantitatively assessing stroke severity. However, the current manual scoring of NIHSS is labor-intensive, time-consuming, and sometimes unreliable. Applying artificial intelligence (AI) techniques to automate the quantitative assessment of stroke on vast amounts of electronic health records (EHRs) has attracted much interest.

OBJECTIVE

This study aims to develop an automatic, quantitative stroke severity assessment framework through automating the entire NIHSS scoring process on Chinese clinical EHRs.

METHODS

Our approach consists of two major parts: Chinese clinical named entity recognition (CNER) with a domain-adaptive pre-trained large language model (LLM) and automated NIHSS scoring. To build a high-performing CNER model, we first construct a stroke-specific, densely annotated dataset "Chinese Stroke Clinical Records" (CSCR) from EHRs provided by our partner hospital, based on a stroke ontology that defines semantically related entities for stroke assessment. We then pre-train a Chinese clinical LLM coined "CliRoberta" through domain-adaptive transfer learning and construct a deep learning-based CNER model that can accurately extract entities directly from Chinese EHRs. Finally, an automated, end-to-end NIHSS scoring pipeline is proposed by mapping the extracted entities to relevant NIHSS items and values, to quantitatively assess the stroke severity.

RESULTS

Results obtained on a benchmark dataset CCKS2019 and our newly created CSCR dataset demonstrate the superior performance of our domain-adaptive pre-trained LLM and the CNER model, compared with the existing benchmark LLMs and CNER models. The high F1 score of 0.990 ensures the reliability of our model in accurately extracting the entities for the subsequent automatic NIHSS scoring. Subsequently, our automated, end-to-end NIHSS scoring approach achieved excellent inter-rater agreement (0.823) and intraclass consistency (0.986) with the ground truth and significantly reduced the processing time from minutes to a few seconds.

CONCLUSION

Our proposed automatic and quantitative framework for assessing stroke severity demonstrates exceptional performance and reliability through directly scoring the NIHSS from diagnostic notes in Chinese clinical EHRs. Moreover, this study also contributes a new clinical dataset, a pre-trained clinical LLM, and an effective deep learning-based CNER model. The deployment of these advanced algorithms can improve the accuracy and efficiency of clinical assessment, and help improve the quality, affordability and productivity of healthcare services.

ChatGPT in healthcare: A taxonomy and systematic review

The recent release of ChatGPT, a chat bot research project/product of natural language processing (NLP) by OpenAI, stirs up a sensation among both the general public and medical professionals, amassing a phenomenally large user base in a short time. This is a typical example of the 'productization' of cutting-edge technologies, which allows the general public without a technical background to gain firsthand experience in artificial intelligence (AI), similar to the AI hype created by AlphaGo (DeepMind Technologies, UK) and self-driving cars (Google, Tesla, etc.). However, it is crucial, especially for healthcare researchers, to remain prudent amidst the hype. This work provides a systematic review of existing publications on the use of ChatGPT in healthcare, elucidating the 'status quo' of ChatGPT in medical applications, for general readers, healthcare professionals as well as NLP scientists. The large biomedical literature database PubMed is used to retrieve published works on this topic using the keyword 'ChatGPT'. An inclusion criterion and a taxonomy are further proposed to filter the search results and categorize the selected publications, respectively. It is found through the review that the current release of ChatGPT has achieved only moderate or 'passing' performance in a variety of tests, and is unreliable for actual clinical deployment, since it is not intended for clinical applications by design. We conclude that specialized NLP models trained on (bio)medical datasets still represent the right direction to pursue for critical clinical applications.

Artificial intelligence: revolutionizing cardiology with large language models

Natural language processing techniques are having an increasing impact on clinical care from patient, clinician, administrator, and research perspective. Among others are automated generation of clinical notes and discharge letters, medical term coding for billing, medical chatbots both for patients and clinicians, data enrichment in the identification of disease symptoms or diagnosis, cohort selection for clinical trial, and auditing purposes. In the review, an overview of the history in natural language processing techniques developed with brief technical background is presented. Subsequently, the review will discuss implementation strategies of natural language processing tools, thereby specifically focusing on large language models, and conclude with future opportunities in the application of such techniques in the field of cardiology.

Transformation and Articulation of Clinical Data to Understand Students' and Health Professionals' Clinical Reasoning: Protocol for a Scoping Review

BACKGROUND

There are still unanswered questions regarding effective educational strategies to promote the transformation and articulation of clinical data while teaching and learning clinical reasoning. Additionally, understanding how this process can be analyzed and assessed is crucial, particularly considering the rapid growth of natural language processing in artificial intelligence.

OBJECTIVE

The aim of this study is to map educational strategies to promote the transformation and articulation of clinical data among students and health care professionals and to explore the methods used to assess these individuals' transformation and articulation of clinical data.

METHODS

This scoping review follows the Joanna Briggs Institute framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist for the analysis. A literature search was performed in November 2022 using 5 databases: CINAHL (EBSCOhost), MEDLINE (Ovid), Embase (Ovid), PsycINFO (Ovid), and Web of Science (Clarivate). The protocol was registered on the Open Science Framework in November 2023. The scoping review will follow the 9-step framework proposed by Peters and colleagues of the Joanna Briggs Institute. A data extraction form has been developed using key themes from the research questions.

RESULTS

After removing duplicates, the initial search yielded 6656 results, and study selection is underway. The extracted data will be qualitatively analyzed and presented in a diagrammatic or tabular form alongside a narrative summary. The review will be completed by February 2024.

CONCLUSIONS

By synthesizing the evidence on semantic transformation and articulation of clinical data during clinical reasoning education, this review aims to contribute to the refinement of educational strategies and assessment methods used in academic and continuing education programs. The insights gained from this review will help educators develop more effective semantic approaches for teaching or learning clinical reasoning, as opposed to fragmented, purely symptom-based or probabilistic approaches. Besides, the results may suggest some ways to address challenges related to the assessment of clinical reasoning and ensure that the assessment tasks accurately reflect learners' developing competencies and educational progress.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50797.

Top 100 cited classical articles in sentinel lymph nodes biopsy for breast cancer

Background

The sentinel lymph node biopsy (SLNB) takes on a critical significance in breast cancer surgery since it is the gold standard for assessing axillary lymph node (ALN) metastasis and determining whether to perform axillary lymph node dissection (ALND). A bibliometric analysis is beneficial to visualize characteristics and hotspots in the field of sentinel lymph nodes (SLNs), and it is conducive to summarizing the important themes in the field to provide more insights into SLNs and facilitate the management of SLNs.

Materials and methods

Search terms relating to SLNs were aggregated and searched in the Web of Science core collection database to identify the top 100 most cited articles. Bibliometric tools were employed to identify and analyze publications for annual article volume, authors, countries, institutions, keywords, as well as hotspot topics.

Results

The period was from 1998 to 2018. The total number of citations ranged from 160 to 1925. LANCET ONCOLOGY and JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION were the top two journals in which the above articles were published. Giuliano, AE was the author with the highest number of articles in this field with 15. EUROPEAN INST ONCOL is the institution with the highest number of publications, with 35 articles. Hotspots include the following 4 topics, false-negative SLNs after neoadjuvant chemotherapy; prediction of metastatic SLNs; quality of life and postoperative complications; and lymphography of SLNs.

Conclusion

This study applies bibliometric tools to analyze the most influential literature, the top 100 cited articles in the field of SLNB, to provide researchers and physicians with research priorities and hotspots.

The Significance of Artificial Intelligence Platforms in Anatomy Education: An Experience With ChatGPT and Google Bard

This study evaluated the use of two large language models (LLMs), ChatGPT and Google Bard, in anatomy education. The models were asked to answer questions, generate multiple-choice questions, and write articles on anatomy topics. The results showed that the models were able to perform these tasks with varying degrees of accuracy. ChatGPT and Google Bard did not differ significantly in terms of answering questions. Both models were able to generate multiple-choice questions with a high degree of accuracy. However, the performance of the models in article writing was not yet at a sufficient level. The study also found that the use of LLMs in medical education requires caution. This is because LLMs are still under development and they can sometimes generate inaccurate or misleading information. It is important to carefully evaluate the output of LLMs before using them in educational settings. Overall, the study found that LLMs have the potential to be valuable tools for anatomy education. However, more research is needed to improve the accuracy of the models and to better understand how they can be used effectively in educational settings.

Large Language Models in Hematology Case Solving: A Comparative Study of ChatGPT-3.5, Google Bard, and Microsoft Bing

Background Large language models (LLMs), such as ChatGPT-3.5, Google Bard, and Microsoft Bing, have shown promising capabilities in various natural language processing (NLP) tasks. However, their performance and accuracy in solving domain-specific questions, particularly in the field of hematology, have not been extensively investigated. Objective This study aimed to explore the capability of LLMs, namely, ChatGPT-3.5, Google Bard, and Microsoft Bing (Precise), in solving hematology-related cases and comparing their performance. Methods This was a cross-sectional study conducted in the Department of Physiology and Pathology, All India Institute of Medical Sciences, Deoghar, Jharkhand, India. We curated a set of 50 cases on hematology covering a range of topics and complexities. The dataset included queries related to blood disorders, hematologic malignancies, laboratory test parameters, calculations, and treatment options. Each case and related question was prepared with a set of correct answers to compare with. We utilized ChatGPT-3.5, Google Bard Experiment, and Microsoft Bing (Precise) for question-answering tasks. The answers were checked by two physiologists and one pathologist. They rated the answers on a rating scale from one to five. The average score of the three models was compared by Friedman's test with Dunn's post-hoc test. The performance of the LLMs was compared with a median of 2.5 by a one-sample median test as the curriculum from which the questions were curated has a 50% pass grade. Results The scores among the three LLMs were significantly different (p-value < 0.0001) with the highest score by ChatGPT (3.15±1.19), followed by Bard (2.23±1.17) and Bing (1.98±1.01). The score of ChatGPT was significantly higher than 50% (p-value = 0.0004), Bard's score was close to 50% (p-value = 0.38), and Bing's score was significantly lower than the pass score (p-value = 0.0015). Conclusion The LLMs reveal significant differences in solving case vignettes in hematology. ChatGPT exhibited the highest score, followed by Google Bard and Microsoft Bing. The observed performance trends suggest that ChatGPT holds promising potential in the medical domain. However, none of the models was capable of answering all questions accurately. Further research and optimization of language models can offer valuable contributions to healthcare and medical education applications.

Towards Medical Billing Automation: NLP for Outpatient Clinician Note Classification

Objectives

Our primary objective was to develop a natural language processing approach that accurately predicts outpatient Evaluation and Management (E/M) level of service (LoS) codes using clinicians' notes from a health system electronic health record. A secondary objective was to investigate the impact of clinic note de-identification on document classification performance.

Methods

We used retrospective outpatient office clinic notes from four medical and surgical specialties. Classification models were fine-tuned on the clinic notes datasets and stratified by subspecialty. The success criteria for the classification tasks were the classification accuracy and F1-scores on internal test data. For the secondary objective, the dataset was de-identified using Named Entity Recognition (NER) to remove protected health information (PHI), and models were retrained.

Results

The models demonstrated similar predictive performance across different specialties, except for internal medicine, which had the lowest classification accuracy across all model architectures. The models trained on the entire note corpus achieved an E/M LoS CPT code classification accuracy of 74.8% (CI 95: 74.1-75.6). However, the de-identified note corpus showed a markedly lower classification accuracy of 48.2% (CI 95: 47.7-48.6) compared to the model trained on the identified notes.

Conclusion

The study demonstrates the potential of NLP-based document classifiers to accurately predict E/M LoS CPT codes using clinical notes from various medical and procedural specialties. The models' performance suggests that the classification task's complexity merits further investigation. The de-identification experiment demonstrated that de-identification may negatively impact classifier performance. Further research is needed to validate the performance of our NLP classifiers in different healthcare settings and patient populations and to investigate the potential implications of de-identification on model performance.

Natural language processing in radiology: Clinical applications and future directions

Natural language processing (NLP) is a wide range of techniques that allows computers to interact with human text. Applications of NLP in everyday life include language translation aids, chat bots, and text prediction. It has been increasingly utilized in the medical field with increased reliance on electronic health records. As findings in radiology are primarily communicated via text, the field is particularly suited to benefit from NLP based applications. Furthermore, rapidly increasing imaging volume will continue to increase burden on clinicians, emphasizing the need for improvements in workflow. In this article, we highlight the numerous non-clinical, provider focused, and patient focused applications of NLP in radiology. We also comment on challenges associated with development and incorporation of NLP based applications in radiology as well as potential future directions.

Clinical research staff perceptions on a natural language processing-driven tool for eligibility prescreening: An iterative usability assessment

BACKGROUND

Participant recruitment is a barrier to successful clinical research. One strategy to improve recruitment is to conduct eligibility prescreening, a resource-intensive process where clinical research staff manually reviews electronic health records data to identify potentially eligible patients. Criteria2Query (C2Q) was developed to address this problem by capitalizing on natural language processing to generate queries to identify eligible participants from clinical databases semi-autonomously.

OBJECTIVE

We examined the clinical research staff's perceived usability of C2Q for clinical research eligibility prescreening.

METHODS

Twenty clinical research staff evaluated the usability of C2Q using a cognitive walkthrough with a think-aloud protocol and a Post-Study System Usability Questionnaire. On-screen activity and audio were recorded and transcribed. After every-five evaluators completed an evaluation, usability problems were rated by informatics experts and prioritized for system refinement. There were four iterations of system refinement based on the evaluation feedback. Guided by the Organizational Framework for Intuitive Human-computer Interaction, we performed a directed deductive content analysis of the verbatim transcriptions.

RESULTS

Evaluators aged from 24 to 46 years old (33.8; SD: 7.32) demonstrated high computer literacy (6.36; SD:0.17); female (75 %), White (35 %), and clinical research coordinators (45 %). C2Q demonstrated high usability during the final cycle (2.26 out of 7 [lower scores are better], SD: 0.74). The number of unique usability issues decreased after each refinement. Fourteen subthemes emerged from three themes: seeking user goals, performing well-learned tasks, and determining what to do next.

CONCLUSIONS

The cognitive walkthrough with a think-aloud protocol informed iterative system refinement and demonstrated the usability of C2Q by clinical research staff. Key recommendations for system development and implementation include improving system intuitiveness and overall user experience through comprehensive consideration of user needs and requirements for task completion.

A Systematic Review of Application Progress on Machine Learning-Based Natural Language Processing in Breast Cancer over the Past 5 Years

Artificial intelligence (AI) has been steadily developing in the medical field in the past few years, and AI-based applications have advanced cancer diagnosis. Breast cancer has a massive amount of data in oncology. There has been a high level of research enthusiasm to apply AI techniques to assist in breast cancer diagnosis and improve doctors' efficiency. However, the wise utilization of tedious breast cancer-related medical care is still challenging. Over the past few years, AI-based NLP applications have been increasingly proposed in breast cancer. In this systematic review, we conduct the review using preferred reporting items for systematic reviews and meta-analyses (PRISMA) and investigate the recent five years of literature in natural language processing (NLP)-based AI applications. This systematic review aims to uncover the recent trends in this area, close the research gap, and help doctors better understand the NLP application pipeline. We first conduct an initial literature search of 202 publications from Scopus, Web of Science, PubMed, Google Scholar, and the Association for Computational Linguistics (ACL) Anthology. Then, we screen the literature based on inclusion and exclusion criteria. Next, we categorize and analyze the advantages and disadvantages of the different machine learning models. We also discuss the current challenges, such as the lack of a public dataset. Furthermore, we suggest some promising future directions, including semi-supervised learning, active learning, and transfer learning.

Exploring the research landscape of the past, present, and future of thyroid nodules

Introduction

The purpose of this study was to assess the landscape of thyroid nodules research during the last 22 years using machine learning and text analysis.

Methods

In November 2021, we obtained from PubMed all works indexed under the Medical Subject Headings (MeSH) subject line "thyroid nodules." The entire set of search results was retrieved in XML format, and metadata such as title, abstract, keywords, MeSH words, and year of publication were extracted for bibliometric evaluation from the original XML files. To increase the specificity of the investigation, the Latent Dirichlet allocation (LDA) topic modeling method was applied.

Results

Our study included 5,770 research papers. By using frequency analysis of MeSH terms, research on thyroid nodules was divided into two categories: clinical and basic. The proportion of clinical research is nearing 89% and is dominated by the differential diagnosis of thyroid nodules. In contrast, the proportion of MeSH terms relating to basic research was just 11%, with DNA mutation analysis being the most common topic. Following this, LDA analysis revealed the thyroid nodule study had three clusters: Imaging Studies, Biopsy and Diagnosis, and Epidemiology and Screening of Thyroid Cancer. The result suggests that current thyroid nodule research appears to have focused on ultrasonography and histological diagnosis, which are tightly correlated. Molecular biomarker research has increased, therefore enhancing the diagnostic precision of thyroid nodules. However, inflammation, anxiety, and mental health disorders related to thyroid nodules have received little attention.

Conclusion

Basic research on thyroid nodules has unmet research requirements. Future research could focus on developing strategies to more efficiently identify malignant nodules, exploring the mechanism of thyroid nodule development, and enhancing the quality of life of thyroid patients.

Practical Considerations for Developing Clinical Natural Language Processing Systems for Population Health Management and Measurement

Experts have noted a concerning gap between clinical natural language processing (NLP) research and real-world applications, such as clinical decision support. To help address this gap, in this viewpoint, we enumerate a set of practical considerations for developing an NLP system to support real-world clinical needs and improve health outcomes. They include determining (1) the readiness of the data and compute resources for NLP, (2) the organizational incentives to use and maintain the NLP systems, and (3) the feasibility of implementation and continued monitoring. These considerations are intended to benefit the design of future clinical NLP projects and can be applied across a variety of settings, including large health systems or smaller clinical practices that have adopted electronic medical records in the United States and globally.

A scoping review on the use of natural language processing in research on political polarization: trends and research prospects

As part of the "text-as-data" movement, Natural Language Processing (NLP) provides a computational way to examine political polarization. We conducted a methodological scoping review of studies published since 2010 (n = 154) to clarify how NLP research has conceptualized and measured political polarization, and to characterize the degree of integration of the two different research paradigms that meet in this research area. We identified biases toward US context (59%), Twitter data (43%) and machine learning approach (33%). Research covers different layers of the political public sphere (politicians, experts, media, or the lay public), however, very few studies involved more than one layer. Results indicate that only a few studies made use of domain knowledge and a high proportion of the studies were not interdisciplinary. Those studies that made efforts to interpret the results demonstrated that the characteristics of political texts depend not only on the political position of their authors, but also on other often-overlooked factors. Ignoring these factors may lead to overly optimistic performance measures. Also, spurious results may be obtained when causal relations are inferred from textual data. Our paper provides arguments for the integration of explanatory and predictive modeling paradigms, and for a more interdisciplinary approach to polarization research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42001-022-00196-2.

An automated method for developing search strategies for systematic review using Natural Language Processing (NLP)

The design and implementation of systematic reviews and meta-analyses are often hampered by high financial costs, significant time commitment, and biases due to researchers' familiarity with studies. We proposed and implemented a fast and standardized method for search term selection using Natural Language Processing (NLP) and co-occurrence networks to identify relevant search terms to reduce biases in conducting systematic reviews and meta-analyses.•The method was implemented using Python packaged dubbed Ananse, which is benchmarked on the search terms strategy for naïve search proposed by Grames et al. (2019) written in "R". Ananse was applied to a case example towards finding search terms to implement a systematic literature review on cumulative effect studies on forest ecosystems.•The software automatically corrected and classified 100% of the duplicate articles identified by manual deduplication. Ananse was applied to the cumulative effects assessment case study, but it can serve as a general-purpose, open-source software system that can support extensive systematic reviews within a relatively short period with reduced biases.•Besides generating keywords, Ananse can act as middleware or a data converter for integrating multiple datasets into a database.

A novel semiautomatic Chinese keywords instrument screening delirium based on electronic medical records

BACKGROUND

Delirium is frequently unrecognized due to the absence of regular screening. In addition to validated bedside tools, the computer-assisted instrument based on clinical notes from electronic medical records may be useful.

AIMS

To assess the psychometric properties of a Chinese-chart-based keyword instrument for semiautomatically screening delirium using Natural language processing (NLP) based on clinical notes from electronic medical records.

METHODS

The patients were admitted to West China Hospital from January 2015 to December 2017. Grouping patients based on the medical notes, those with accessible physician documents but no nurse documents were classified as the physician & no-nurse (PNN) group, while those with accessible physician and nurse documents were classified as the physician & nurse (PN) group. The psychometric properties, test-retest reliability, internal consistency reliability (Cronbach's α), and criterion validity were calculated. Using receiver operating characteristic (ROC) analysis, the criterion validity of delirium was evaluated in comparison to the results of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.

RESULTS

A total of 779 patients were enrolled in the study. Their ages ranged from 65 to 103 years (82.5 ± 6.5), with men accounting for 71.9% of the total. A total of 312 patients had access to only physician documents in the physician & no-nurse (PNN) group, whereas 467 patients had access to both physician and nurse documents in the physician & nurse (PN) group. All 779 patients had a Cronbach's alpha of 0.728 in terms of reliability, with 100% test-retest reliability. The area under the ROC curve (AUC) values of the delirium screening instrument for criterion validity were 0.76 (all patients, n = 779), 0.72 (PNN, n = 312), and 0.79 (PN, n = 467), respectively.

CONCLUSION

A delirium screening instrument composed of Chinese keywords that can be easily and quickly obtained from electronic medical records was developed, which improved delirium detection in older people.

TRIAL REGISTRATION

Not applicable.

Sexual Modesty in Sexual Expression and Experience: A Scoping Review, 2000 - 2021

Sexual modesty is the social, cultural, interpersonal, and psychological systems - defined by the tenets of Script Theory - that regulate individuals' sexual expression and experience at the social, legal, and interpersonal boundaries of acceptable/not-acceptable, private/public, and personal/social. Almost all aspects of sexual expression and experience are touched by the pervasive modesty standards for sexual communication, sexual display, sexual relations, and sexual behaviors. Sexual modesty influences an array of sexual and reproductive health outcomes. Many aspects of sexual modesty are enforced by legal as well as social, cultural, and religious proscriptions, including social shaming and ostracism as well as corporal and capital punishments. The purpose of this paper is to summarize a diverse literature related to sexual modesty from the years 2000 to 2021 in order to clarify its role in sexual health and sexual wellbeing and to identify directions for new research.

Identifying Datasets for Cross-Study Analysis in dbGaP using PhenX

Identifying relevant studies and harmonizing datasets are major hurdles for data reuse. Common Data Elements (CDEs) can help identify comparable study datasets and reduce the burden of retrospective data harmonization, but they have not been required, historically. The collaborative team at PhenX and dbGaP developed an approach to use PhenX variables as a set of CDEs to link phenotypic data and identify comparable studies in dbGaP. Variables were identified as either comparable or related, based on the data collection mode used to harmonize data across mapped datasets. We further added a CDE data field in the dbGaP data submission packet to indicate use of PhenX and annotate linkages in the future. Some 13,653 dbGaP variables from 521 studies were linked through PhenX variable mapping. These variable linkages have been made accessible for browsing and searching in the repository through dbGaP CDE-faceted search filter and the PhenX variable search tool. New features in dbGaP and PhenX enable investigators to identify variable linkages among dbGaP studies and reveal opportunities for cross-study analysis.

Using natural language processing in facilitating pre-hospital telephone triage of emergency calls

Introduction

Natural language processing (NLP) is an area of computer science that involves the use of computers to understand human language and semantics (meaning) and to offer consistent and reliable responses. There is good evidence of significant advancement in the use of NLP technology in dealing with acutely ill patients in hospital (such as differential diagnosis assistance, clinical decision-making and treatment options). Further technical development and research into the use of NLP could enable further improvements in the quality of pre-hospital emergency care. The aim of this literature review was to explore the opportunities and potential obstacles in implementing NLP during this phase of emergency care and to question if NLP could contribute towards improving the process of nature of call screening (NoCS) to enable earlier recognition of life-threatening situations during telephone triage of emergency calls.

Methods

A systematic search strategy using two electronic databases (CINAHL and MEDLINE) was conducted in December 2021. The PRISMA systematic approach was used to conduct a review of the literature, and selected studies were identified and used to support a critical review of the actual and potential use of NLP for the call-taking phase of emergency care.

Results

An initial search offered 204 records: 23 remained after eliminating duplicates and a consideration of title and abstracts. A further 16 full-text articles were deemed ineligible (not related to the subject under investigation), leaving seven included studies. Following a thematic review of these studies two themes emerged, that are considered individually and together: (i) use of NLP for dealing with out-of-hospital cardiac arrest and (ii) responding to increased accuracy of NLP.

Conclusions

NLP has the potential to reduce or eliminate human bias during the emergency triage assessment process and contribute towards improving triage accuracy in pre-hospital decision-making and an early identification and categorisation of life-threatening conditions. Evidence to date is mostly linked to cardiac arrest identification; this review proposes that during the call-taking phase NLP should be extended to include further medical emergencies (including fracture/trauma, stroke and ketoacidosis). Further research is indicated to test the reliability of these findings and a proportionate introduction of NLP simultaneous with increased quality and reliability.

Identify Caregiver Availability Using Medical Notes: Rule-Based Natural Language Processing

Background

Identifying caregiver availability, particularly for patients with dementia or those with a disability, is critical to informing the appropriate care planning by the health systems, hospitals, and providers. This information is not readily available, and there is a paucity of pragmatic approaches to automatically identifying caregiver availability and type.

Objective

Our main objective was to use medical notes to assess caregiver availability and type for hospitalized patients with dementia. Our second objective was to identify whether the patient lived at home or resided at an institution.

Methods

In this retrospective cohort study, we used 2016-2019 telephone-encounter medical notes from a single institution to develop a rule-based natural language processing (NLP) algorithm to identify the patient’s caregiver availability and place of residence. Using note-level data, we compared the results of the NLP algorithm with human-conducted chart abstraction for both training (749/976, 77%) and test sets (227/976, 23%) for a total of 223 adults aged 65 years and older diagnosed with dementia. Our outcomes included determining whether the patients (1) reside at home or in an institution, (2) have a formal caregiver, and (3) have an informal caregiver.

Results

Test set results indicated that our NLP algorithm had high level of accuracy and reliability for identifying whether patients had an informal caregiver (F₁=0.94, accuracy=0.95, sensitivity=0.97, and specificity=0.93), but was relatively less able to identify whether the patient lived at an institution (F₁=0.64, accuracy=0.90, sensitivity=0.51, and specificity=0.98). The most common explanations for NLP misclassifications across all categories were (1) incomplete or misspelled facility names; (2) past, uncertain, or undecided status; (3) uncommon abbreviations; and (4) irregular use of templates.

Conclusions

This innovative work was the first to use medical notes to pragmatically determine caregiver availability. Our NLP algorithm identified whether hospitalized patients with dementia have a formal or informal caregiver and, to a lesser extent, whether they lived at home or in an institutional setting. There is merit in using NLP to identify caregivers. This study serves as a proof of concept. Future work can use other approaches and further identify caregivers and the extent of their availability.

Trends in Worldwide Research in Inflammatory Bowel Disease Over the Period 2012–2021: A Bibliometric Study

Background

Inflammatory bowel disease (IBD) is a continuously increasing and worldwide disease, and the number of publications of IBD has been expanding in the past 10 years. The purpose of this study is to analyze the published articles of IBD in the past decade via machine learning and text analysis and get a more comprehensive understanding of the research trends and changes in IBD in the past 10 years.

Method

In November 2021, we downloaded the published articles related to IBD in PubMed for the past 10 years (2012–2021). We utilized Python to extract the title, publication date, MeSH terms, and abstract from the metadata of each publication for bibliometric assessment. Latent Dirichlet allocation (LDA) was used to the abstracts to identify publications' research topics with greater specificity.

Result

We finally identified and analyzed 34,458 publications in total. We found that publications in the last 10 years were mainly focused on treatment and mechanism. Among them, publications on biological agents and Gastrointestinal Microbiome have a significant advantage in terms of volume and rate of publications. In addition, publications related to IBD and coronavirus disease 2019 (COVID-19) have increased sharply since the outbreak of the worldwide pandemic caused by novel β-coronavirus in 2019. However, researchers seem to pay less attention to the nutritional and psychological status of patients with IBD.

Conclusion

IBD is still a worldwide disease of concern with the publication of IBD-related research has expanded continuously over the past decade. More research related nutritional and psychological status of patients with IBD is needed in the future. Besides, it is worth noting that the management of chronic diseases such as IBD required additional attention during an infectious disease epidemic.

A Mental Health Chatbot with Cognitive Skills for Personalised Behavioural Activation and Remote Health Monitoring

Mental health issues are at the forefront of healthcare challenges facing contemporary human society. These issues are most prevalent among working-age people, impacting negatively on the individual, his/her family, workplace, community, and the economy. Conventional mental healthcare services, although highly effective, cannot be scaled up to address the increasing demand from affected individuals, as evidenced in the first two years of the COVID-19 pandemic. Conversational agents, or chatbots, are a recent technological innovation that has been successfully adapted for mental healthcare as a scalable platform of cross-platform smartphone applications that provides first-level support for such individuals. Despite this disposition, mental health chatbots in the extant literature and practice are limited in terms of the therapy provided and the level of personalisation. For instance, most chatbots extend Cognitive Behavioural Therapy (CBT) into predefined conversational pathways that are generic and ineffective in recurrent use. In this paper, we postulate that Behavioural Activation (BA) therapy and Artificial Intelligence (AI) are more effectively materialised in a chatbot setting to provide recurrent emotional support, personalised assistance, and remote mental health monitoring. We present the design and development of our BA-based AI chatbot, followed by its participatory evaluation in a pilot study setting that confirmed its effectiveness in providing support for individuals with mental health issues.

Global research output in 'pharmacovigilance' during 2010-2020

The rapid spread of Covid-19 pandemic globally has thrust drugs safety into the spotlight and the public is now more aware of the role of healthcare professionals and health regulators. The present study aimed to measure the global research landscape on pharmacovigilance (PV) indexed in Scopus database for eleven years period spanning from 2010–2020. The study has sought to use quantitative and visualization technologies for data analysis and interpretation. The search strategy accumulated a total of 2052 global publications data on PV. The findings disclose that the global research productivity on PV registered 8.74% average growth rate (AGR) and 7.38% compound average growth rate (CAGR). The mean relative growth rate (RGR) and doubling time (DT) of PV global publications for the 11 years is 0.27 and 3.03, respectively. The average number of authors per paper (AAPP) is 1.52 and average productivity per author (PPA) is 0.68. The authorship patterns in PV research shows collaborative trend as most of the publications have been published by multiple authors (80.75%). The mean values of degree of collaboration (DC), collaboration index (CI), collaboration coefficient (CC) and modified collaboration coefficient (MCC) during the selected period of study are 0.79, 2.74, 0.72, and 0.73, respectively which highly significant and indicates the better authorship collaborations. France is the bellwether in PV related scientific research as produced the highest number of publications.

Extracting clinical named entity for pituitary adenomas from Chinese electronic medical records

OBJECTIVE

Pituitary adenomas are the most common type of pituitary disorders, which usually occur in young adults and often affect the patient's physical development, labor capacity and fertility. Clinical free texts noted in electronic medical records (EMRs) of pituitary adenomas patients contain abundant diagnosis and treatment information. However, this information has not been well utilized because of the challenge to extract information from unstructured clinical texts. This study aims to enable machines to intelligently process clinical information, and automatically extract clinical named entity for pituitary adenomas from Chinese EMRs.

METHODS

The clinical corpus used in this study was from one pituitary adenomas neurosurgery treatment center of a 3A hospital in China. Four types of fine-grained texts of clinical records were selected, which included notes from present illness, past medical history, case characteristics and family history of 500 pituitary adenoma inpatients. The dictionary-based matching, conditional random fields (CRF), bidirectional long short-term memory with CRF (BiLSTM-CRF), and bidirectional encoder representations from transformers with BiLSTM-CRF (BERT-BiLSTM-CRF) were used to extract clinical entities from a Chinese EMRs corpus. A comprehensive dictionary was constructed based on open source vocabularies and a domain dictionary for pituitary adenomas to conduct the dictionary-based matching method. We selected features such as part of speech, radical, document type, and the position of characters to train the CRF-based model. Random character embeddings and the character embeddings pretrained by BERT were used respectively as the input features for the BiLSTM-CRF model and the BERT-BiLSTM-CRF model. Both strict metric and relaxed metric were used to evaluate the performance of these methods.

RESULTS

Experimental results demonstrated that the deep learning and other machine learning methods were able to automatically extract clinical named entities, including symptoms, body regions, diseases, family histories, surgeries, medications, and disease courses of pituitary adenomas from Chinese EMRs. With regard to overall performance, BERT-BiLSTM-CRF has the highest strict F1 value of 91.27% and the highest relaxed F1 value of 95.57% respectively. Additional evaluations showed that BERT-BiLSTM-CRF performed best in almost all entity recognition except surgery and disease course. BiLSTM-CRF performed best in disease course entity recognition, and performed as well as the CRF model for part of speech, radical and document type features, with both strict and relaxed F1 value reaching 96.48%. The CRF model with part of speech, radical and document type features performed best in surgery entity recognition with relaxed F1 value of 95.29%.

CONCLUSIONS

In this study, we conducted four entity recognition methods for pituitary adenomas based on Chinese EMRs. It demonstrates that the deep learning methods can effectively extract various types of clinical entities with satisfying performance. This study contributed to the clinical named entity extraction from Chinese neurosurgical EMRs. The findings could also assist in information extraction in other Chinese medical texts.

Extracting Multiple Worries from Breast Cancer Patient Blogs Using Multi-Label Classification with a Natural Language-Processing Model BERT (Bidirectional Encoder Representations from Transformers): Infodemiology Study of Blogs (Preprint)

Background

Patients with breast cancer have a variety of worries and need multifaceted information support. Their accumulated posts on social media contain rich descriptions of their daily worries concerning issues such as treatment, family, and finances. It is important to identify these issues to help patients with breast cancer to resolve their worries and obtain reliable information.

Objective

This study aimed to extract and classify multiple worries from text generated by patients with breast cancer using Bidirectional Encoder Representations From Transformers (BERT), a context-aware natural language processing model.

Methods

A total of 2272 blog posts by patients with breast cancer in Japan were collected. Five worry labels, “treatment,” “physical,” “psychological,” “work/financial,” and “family/friends,” were defined and assigned to each post. Multiple labels were allowed. To assess the label criteria, 50 blog posts were randomly selected and annotated by two researchers with medical knowledge. After the interannotator agreement had been assessed by means of Cohen kappa, one researcher annotated all the blogs. A multilabel classifier that simultaneously predicts five worries in a text was developed using BERT. This classifier was fine-tuned by using the posts as input and adding a classification layer to the pretrained BERT. The performance was evaluated for precision using the average of 5-fold cross-validation results.

Results

Among the blog posts, 477 included “treatment,” 1138 included “physical,” 673 included “psychological,” 312 included “work/financial,” and 283 included “family/friends.” The interannotator agreement values were 0.67 for “treatment,” 0.76 for “physical,” 0.56 for “psychological,” 0.73 for “work/financial,” and 0.73 for “family/friends,” indicating a high degree of agreement. Among all blog posts, 544 contained no label, 892 contained one label, and 836 contained multiple labels. It was found that the worries varied from user to user, and the worries posted by the same user changed over time. The model performed well, though prediction performance differed for each label. The values of precision were 0.59 for “treatment,” 0.82 for “physical,” 0.64 for “psychological,” 0.67 for “work/financial,” and 0.58 for “family/friends.” The higher the interannotator agreement and the greater the number of posts, the higher the precision tended to be.

Conclusions

This study showed that the BERT model can extract multiple worries from text generated from patients with breast cancer. This is the first application of a multilabel classifier using the BERT model to extract multiple worries from patient-generated text. The results will be helpful to identify breast cancer patients’ worries and give them timely social support.

Automatic Identification of Addresses: A Systematic Literature Review

Address matching continues to play a central role at various levels, through geocoding and data integration from different sources, with a view to promote activities such as urban planning, location-based services, and the construction of databases like those used in census operations. However, the task of address matching continues to face several challenges, such as non-standard or incomplete address records or addresses written in more complex languages. In order to better understand how current limitations can be overcome, this paper conducted a systematic literature review focused on automated approaches to address matching and their evolution across time. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed, resulting in a final set of 41 papers published between 2002 and 2021, the great majority of which are after 2017, with Chinese authors leading the way. The main findings revealed a consistent move from more traditional approaches to deep learning methods based on semantics, encoder-decoder architectures, and attention mechanisms, as well as the very recent adoption of hybrid approaches making an increased use of spatial constraints and entities. The adoption of evolutionary-based approaches and privacy preserving methods stand as some of the research gaps to address in future studies.

Biologically-oriented mud volcano database: muddy_db

Mud volcanoes (MVs) are naturally occurring hydrocarbon hotbeds with continuous methane discharge, contributing to global warming. They host microbial communities adapted to hydrocarbon oxidation. Given their research value, MVs still represent a niche topic in microbiology and are neglected by hydrocarbon-oriented research. All the data regarding MVs is sporadic and decentralized. To mitigate this problem, we built a custom Natural Language Processing pipeline (muddy_mine), and collected all the available MV data from open-access articles. Based on this data, we built the muddy_db database. The muddy_db represents the first biologically oriented database rendered as a user-friendly web app. This database includes all the relevant MV data, ranging from microbial taxonomy to hydrocarbon occurrence and geology. The muddy_mine and muddy_db tools are licensed under the GPLv3. muddy_db R Shiny web app: https://muddy-db.shinyapps.io/muddy_db/ muddy_db R package: https://github.com/TracyRage/muddy_db muddy_mine Conda package: https://github.com/TracyRage/muddy_mine.

Evolutionary Overview of Consumer Health Informatics: Bibliometric Study on the Web of Science from 1999 to 2019

BACKGROUND

Consumer health informatics (CHI) originated in the 1990s. With the rapid development of computer and information technology for health decision making, an increasing number of consumers have obtained health-related information through the internet, and CHI has also attracted the attention of an increasing number of scholars.

OBJECTIVE

The aim of this study was to analyze the research themes and evolution characteristics of different study periods and to discuss the dynamic evolution path and research theme rules in a time-series framework from the perspective of a strategy map and a data flow in CHI.

METHODS

The Web of Science core collection database of the Institute for Scientific Information was used as the data source to retrieve relevant articles in the field of CHI. SciMAT was used to preprocess the literature data and construct the overlapping map, evolution map, strategic diagram, and cluster network characterized by keywords. Besides, a bibliometric analysis of the general characteristics, the evolutionary characteristics of the theme, and the evolutionary path of the theme was conducted.

RESULTS

A total of 986 articles were obtained after the retrieval, and 931 articles met the document-type requirement. In the past 21 years, the number of articles increased every year, with a remarkable growth after 2015. The research content in 4 different study periods formed the following 38 themes: patient education, medicine, needs, and bibliographic database in the 1999-2003 study period; world wide web, patient education, eHealth, patients, medication, terminology, behavior, technology, and disease in the 2004-2008 study period; websites, information seeking, physicians, attitudes, technology, risk, food labeling, patient, strategies, patient education, and eHealth in the 2009-2014 study period; and electronic medical records, health information seeking, attitudes, health communication, breast cancer, health literacy, technology, natural language processing, user-centered design, pharmacy, academic libraries, costs, internet utilization, and online health information in the 2015-2019 study period. Besides, these themes formed 10 evolution paths in 3 research directions: patient education and intervention, consumer demand attitude and behavior, and internet information technology application.

CONCLUSIONS

Averaging 93 publications every year since 2015, CHI research is in a rapid growth period. The research themes mainly focus on patient education, health information needs, health information search behavior, health behavior intervention, health literacy, health information technology, eHealth, and other aspects. Patient education and intervention research, consumer demand, attitude, and behavior research comprise the main theme evolution path, whose evolution process has been relatively stable. This evolution path will continue to become the research hotspot in this field. Research on the internet and information technology application is a secondary theme evolution path with development potential.

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP. Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n = 2469) and chest radiographs and computer tomography (CT) studies (n = 2255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score. After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to > 0.90. CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.

Constructing High-Fidelity Phenotype Knowledge Graphs for Infectious Diseases With a Fine-Grained Semantic Information Model: Development and Usability Study

BACKGROUND

Phenotypes characterize the clinical manifestations of diseases and provide important information for diagnosis. Therefore, the construction of phenotype knowledge graphs for diseases is valuable to the development of artificial intelligence in medicine. However, phenotype knowledge graphs in current knowledge bases such as WikiData and DBpedia are coarse-grained knowledge graphs because they only consider the core concepts of phenotypes while neglecting the details (attributes) associated with these phenotypes.

OBJECTIVE

To characterize the details of disease phenotypes for clinical guidelines, we proposed a fine-grained semantic information model named PhenoSSU (semantic structured unit of phenotypes).

METHODS

PhenoSSU is an "entity-attribute-value" model by its very nature, and it aims to capture the full semantic information underlying phenotype descriptions with a series of attributes and values. A total of 193 clinical guidelines for infectious diseases from Wikipedia were selected as the study corpus, and 12 attributes from SNOMED-CT were introduced into the PhenoSSU model based on the co-occurrences of phenotype concepts and attribute values. The expressive power of the PhenoSSU model was evaluated by analyzing whether PhenoSSU instances could capture the full semantics underlying the descriptions of the corresponding phenotypes. To automatically construct fine-grained phenotype knowledge graphs, a hybrid strategy that first recognized phenotype concepts with the MetaMap tool and then predicted the attribute values of phenotypes with machine learning classifiers was developed.

RESULTS

Fine-grained phenotype knowledge graphs of 193 infectious diseases were manually constructed with the BRAT annotation tool. A total of 4020 PhenoSSU instances were annotated in these knowledge graphs, and 3757 of them (89.5%) were found to be able to capture the full semantics underlying the descriptions of the corresponding phenotypes listed in clinical guidelines. By comparison, other information models, such as the clinical element model and the HL7 fast health care interoperability resource model, could only capture the full semantics underlying 48.4% (2034/4020) and 21.8% (914/4020) of the descriptions of phenotypes listed in clinical guidelines, respectively. The hybrid strategy achieved an F1-score of 0.732 for the subtask of phenotype concept recognition and an average weighted accuracy of 0.776 for the subtask of attribute value prediction.

CONCLUSIONS

PhenoSSU is an effective information model for the precise representation of phenotype knowledge for clinical guidelines, and machine learning can be used to improve the efficiency of constructing PhenoSSU-based knowledge graphs. Our work will potentially shift the focus of medical knowledge engineering from a coarse-grained level to a more fine-grained level.

Machine Learning for Mental Health in Social Media: Bibliometric Study

Background

Social media platforms provide an easily accessible and time-saving communication approach for individuals with mental disorders compared to face-to-face meetings with medical providers. Recently, machine learning (ML)-based mental health exploration using large-scale social media data has attracted significant attention.

Objective

We aimed to provide a bibliometric analysis and discussion on research trends of ML for mental health in social media.

Methods

Publications addressing social media and ML in the field of mental health were retrieved from the Scopus and Web of Science databases. We analyzed the publication distribution to measure productivity on sources, countries, institutions, authors, and research subjects, and visualized the trends in this field using a keyword co-occurrence network. The research methodologies of previous studies with high citations are also thoroughly described.

Results

We obtained a total of 565 relevant papers published from 2015 to 2020. In the last 5 years, the number of publications has demonstrated continuous growth with Lecture Notes in Computer Science and Journal of Medical Internet Research as the two most productive sources based on Scopus and Web of Science records. In addition, notable methodological approaches with data resources presented in high-ranking publications were investigated.

Conclusions

The results of this study highlight continuous growth in this research area. Moreover, we retrieved three main discussion points from a comprehensive overview of highly cited publications that provide new in-depth directions for both researchers and practitioners.