Development and Validation of a Deep Learning Model for Detection of Allergic Reactions Using Safety Event Reports Across Hospitals

Neural transition system abstraction for neural network dynamical system models and its application to Computational Tree Logic verification

This paper proposes an explainable abstraction-based verification method that prioritizes user interaction and enhances interpretability. By partitioning the system's state space using a data-driven process, we can abstract the dynamics into words consisting of state labels. When given a trained neural network model, a set-valued reachability analysis method is introduced to estimate the relationship between each subsystem. We construct the neural transition system abstraction with the neural network model and the relationships between partitions. Then, the abstracted model can be verified through Computational Tree Logic (CTL), enabling formal verification of the system's behavior. This approach greatly enhances the interpretability and verification of data-driven models, as well as the ability to validate against the specification. Finally, examples of the Maglev model and handwritten model abstractions are given to illustrate our proposed model verification framework, which demonstrates that the proposed framework has advantages in enhancing model interpretability and verifying user-specified properties based on CTL.

Opportunities and challenges with artificial intelligence in allergy and immunology: a bibliometric study

Introduction

The fields of allergy and immunology are increasingly recognizing the transformative potential of artificial intelligence (AI). Its adoption is reshaping research directions, clinical practices, and healthcare systems. However, a systematic overview identifying current statuses, emerging trends, and future research hotspots is lacking.

Methods

This study applied bibliometric analysis methods to systematically evaluate the global research landscape of AI applications in allergy and immunology. Data from 3,883 articles published by 21,552 authors across 1,247 journals were collected and analyzed to identify leading contributors, prevalent research themes, and collaboration patterns.

Results

Analysis revealed that the USA and China are currently leading in research output and scientific impact in this domain. AI methodologies, especially machine learning (ML) and deep learning (DL), are predominantly applied in drug discovery and development, disease classification and prediction, immune response modeling, clinical decision support, diagnostics, healthcare system digitalization, and medical education. Emerging trends indicate significant movement toward personalized medical systems integration.

Discussion

The findings demonstrate the dynamic evolution of AI in allergy and immunology, highlighting the broadening scope from basic diagnostics to comprehensive personalized healthcare systems. Despite advancements, critical challenges persist, including technological limitations, ethical concerns, and regulatory frameworks that could potentially hinder further implementation and integration.

Conclusion

AI holds considerable promise for advancing allergy and immunology globally by enhancing healthcare precision, efficiency, and accessibility. Addressing existing technological, ethical, and regulatory challenges will be crucial to fully realizing its potential, ultimately improving global health outcomes and patient well-being.

Identification of an ANCA-associated vasculitis cohort using deep learning and electronic health records

BACKGROUND

ANCA-associated vasculitis (AAV) is a rare but serious disease. Traditional case-identification methods using claims data can be time-intensive and may miss important subgroups. We hypothesized that a deep learning model analyzing electronic health records (EHR) can more accurately identify AAV cases.

METHODS

We examined the Mass General Brigham (MGB) repository of clinical documentation from 12/1/1979 to 5/11/2021, using expert-curated keywords and ICD codes to identify a large cohort of potential AAV cases. Three labeled datasets (I, II, III) were created, each containing note sections. We trained and evaluated a range of machine learning and deep learning algorithms for note-level classification, using metrics like positive predictive value (PPV), sensitivity, F-score, area under the receiver operating characteristic curve (AUROC), and area under the precision and recall curve (AUPRC). The hierarchical attention network (HAN) was further evaluated for its ability to classify AAV cases at the patient-level, compared with rule-based algorithms in 2000 randomly chosen samples.

RESULTS

Datasets I, II, and III comprised 6000, 3008, and 7500 note sections, respectively. HAN achieved the highest AUROC in all three datasets, with scores of 0.983, 0.991, and 0.991. The deep learning approach also had among the highest PPVs across the three datasets (0.941, 0.954, and 0.800, respectively). In a test cohort of 2000 cases, the HAN model achieved a PPV of 0.262 and an estimated sensitivity of 0.975. Compared to the best rule-based algorithm, HAN identified six additional AAV cases, representing 13% of the total.

CONCLUSION

The deep learning model effectively classifies clinical note sections for AAV diagnosis. Its application to EHR notes can potentially uncover additional cases missed by traditional rule-based methods.

Leveraging Generative AI for Drug Safety and Pharmacovigilance

Predictions are made by artificial intelligence, especially through machine learning, which uses algorithms and past knowledge. Notably, there has been an increase in interest in using artificial intelligence, particularly generative AI, in the pharmacovigilance of pharmaceuticals under development, as well as those already in the market. This review was conducted to understand how generative AI can play an important role in pharmacovigilance and improving drug safety monitoring. Data from previously published articles and news items were reviewed in order to obtain information. We used PubMed and Google Scholar as our search engines, and keywords (pharmacovigilance, artificial intelligence, machine learning, drug safety, and patient safety) were used. In toto, we reviewed 109 articles published till 31st January 2024, and the obtained information was interpreted, compiled, evaluated, and conclusions were reached. Generative AI has transformative potential in pharmacovigilance, showcasing benefits, such as enhanced adverse event detection, data-driven risk prediction, and optimized drug development. By making it easier to process and analyze big datasets, generative artificial intelligence has applications across a variety of disease states. Machine learning and automation in this field can streamline pharmacovigilance procedures and provide a more efficient way to assess safety-related data. Nevertheless, more investigation is required to determine how this optimization affects the caliber of safety analyses. In the near future, the increased utilization of artificial intelligence is anticipated, especially in predicting side effects and Adverse Drug Reactions (ADRs).

Probabilistic medical predictions of large language models

Large Language Models (LLMs) have shown promise in clinical applications through prompt engineering, allowing flexible clinical predictions. However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making. While explicit prompts can lead LLMs to generate probability estimates, their numerical reasoning limitations raise concerns about reliability. We compared explicit probabilities from text generation to implicit probabilities derived from the likelihood of predicting the correct label token. Across six advanced open-source LLMs and five medical datasets, explicit probabilities consistently underperformed implicit probabilities in discrimination, precision, and recall. This discrepancy is more pronounced with smaller LLMs and imbalanced datasets, highlighting the need for cautious interpretation, improved probability estimation methods, and further research for clinical use of LLMs.

Enhancing early detection of cognitive decline in the elderly: a comparative study utilizing large language models in clinical notes

BACKGROUND

Large language models (LLMs) have shown promising performance in various healthcare domains, but their effectiveness in identifying specific clinical conditions in real medical records is less explored. This study evaluates LLMs for detecting signs of cognitive decline in real electronic health record (EHR) clinical notes, comparing their error profiles with traditional models. The insights gained will inform strategies for performance enhancement.

METHODS

This study, conducted at Mass General Brigham in Boston, MA, analysed clinical notes from the four years prior to a 2019 diagnosis of mild cognitive impairment in patients aged 50 and older. We developed prompts for two LLMs, Llama 2 and GPT-4, on Health Insurance Portability and Accountability Act (HIPAA)-compliant cloud-computing platforms using multiple approaches (e.g., hard prompting, retrieval augmented generation, and error analysis-based instructions) to select the optimal LLM-based method. Baseline models included a hierarchical attention-based neural network and XGBoost. Subsequently, we constructed an ensemble of the three models using a majority vote approach. Confusion-matrix-based scores were used for model evaluation.

FINDINGS

We used a randomly annotated sample of 4949 note sections from 1969 patients (women: 1046 [53.1%]; age: mean, 76.0 [SD, 13.3] years), filtered with keywords related to cognitive functions, for model development. For testing, a random annotated sample of 1996 note sections from 1161 patients (women: 619 [53.3%]; age: mean, 76.5 [SD, 10.2] years) without keyword filtering was utilised. GPT-4 demonstrated superior accuracy and efficiency compared to Llama 2, but did not outperform traditional models. The ensemble model outperformed the individual models in terms of all evaluation metrics with statistical significance (p < 0.01), achieving a precision of 90.2% [95% CI: 81.9%-96.8%], a recall of 94.2% [95% CI: 87.9%-98.7%], and an F1-score of 92.1% [95% CI: 86.8%-96.4%]. Notably, the ensemble model showed a significant improvement in precision, increasing from a range of 70%-79% to above 90%, compared to the best-performing single model. Error analysis revealed that 63 samples were incorrectly predicted by at least one model; however, only 2 cases (3.2%) were mutual errors across all models, indicating diverse error profiles among them.

INTERPRETATION

LLMs and traditional machine learning models trained using local EHR data exhibited diverse error profiles. The ensemble of these models was found to be complementary, enhancing diagnostic performance. Future research should investigate integrating LLMs with smaller, localised models and incorporating medical data and domain knowledge to enhance performance on specific tasks.

FUNDING

This research was supported by the National Institute on Aging grants (R44AG081006, R01AG080429) and National Library of Medicine grant (R01LM014239).

An explainable long short-term memory network for surgical site infection identification

BACKGROUND

Currently, surgical site infection surveillance relies on labor-intensive manual chart review. Recently suggested solutions involve machine learning to identify surgical site infections directly from the medical record. Deep learning is a form of machine learning that has historically performed better than traditional methods while being harder to interpret. We propose a deep learning model, a long short-term memory network, for the identification of surgical site infection from the medical record with an attention layer for explainability.

METHODS

We retrieved structured data and clinical notes from the University of Utah Health System's electronic health care record for operative events randomly selected for manual chart review from January 2016 to June 2021. Surgical site infection occurring within 30 days of surgery was determined according to the National Surgical Quality Improvement Program definition. We trained the long short-term memory model along with traditional machine learning models for comparison. We calculated several performance metrics from a holdout test set and performed additional analyses to understand the performance of the long short-term memory, including an explainability analysis.

RESULTS

Surgical site infection was present in 4.7% of the total 9,185 operative events. The area under the receiver operating characteristic curve and sensitivity of the long short-term memory was higher (area under the receiver operating characteristic curve: 0.954, sensitivity: 0.920) compared to the top traditional model (area under the receiver operating characteristic curve: 0.937, sensitivity: 0.736). The top 5 features of the long short-term memory included 2 procedure codes and 3 laboratory values.

CONCLUSION

Surgical site infection surveillance is vital for the reduction of surgical site infection rates. Our explainable long short-term memory achieved a comparable area under the receiver operating characteristic curve and greater sensitivity when compared to traditional machine learning methods. With explainable deep learning, automated surgical site infection surveillance could replace burdensome manual chart review processes.

Identification of an ANCA-Associated Vasculitis Cohort Using Deep Learning and Electronic Health Records

Background

ANCA-associated vasculitis (AAV) is a rare but serious disease. Traditional case-identification methods using claims data can be time-intensive and may miss important subgroups. We hypothesized that a deep learning model analyzing electronic health records (EHR) can more accurately identify AAV cases.

Methods

We examined the Mass General Brigham (MGB) repository of clinical documentation from 12/1/1979 to 5/11/2021, using expert-curated keywords and ICD codes to identify a large cohort of potential AAV cases. Three labeled datasets (I, II, III) were created, each containing note sections. We trained and evaluated a range of machine learning and deep learning algorithms for note-level classification, using metrics like positive predictive value (PPV), sensitivity, F-score, area under the receiver operating characteristic curve (AUROC), and area under the precision and recall curve (AUPRC). The deep learning model was further evaluated for its ability to classify AAV cases at the patient-level, compared with rule-based algorithms in 2,000 randomly chosen samples.

Results

Datasets I, II, and III comprised 6,000, 3,008, and 7,500 note sections, respectively. Deep learning achieved the highest AUROC in all three datasets, with scores of 0.983, 0.991, and 0.991. The deep learning approach also had among the highest PPVs across the three datasets (0.941, 0.954, and 0.800, respectively). In a test cohort of 2,000 cases, the deep learning model achieved a PPV of 0.262 and an estimated sensitivity of 0.975. Compared to the best rule-based algorithm, the deep learning model identified six additional AAV cases, representing 13% of the total.

Conclusion

The deep learning model effectively classifies clinical note sections for AAV diagnosis. Its application to EHR notes can potentially uncover additional cases missed by traditional rule-based methods.

Enhancing Early Detection of Cognitive Decline in the Elderly: A Comparative Study Utilizing Large Language Models in Clinical Notes

Background

Large language models (LLMs) have shown promising performance in various healthcare domains, but their effectiveness in identifying specific clinical conditions in real medical records is less explored. This study evaluates LLMs for detecting signs of cognitive decline in real electronic health record (EHR) clinical notes, comparing their error profiles with traditional models. The insights gained will inform strategies for performance enhancement.

Methods

This study, conducted at Mass General Brigham in Boston, MA, analyzed clinical notes from the four years prior to a 2019 diagnosis of mild cognitive impairment in patients aged 50 and older. We used a randomly annotated sample of 4,949 note sections, filtered with keywords related to cognitive functions, for model development. For testing, a random annotated sample of 1,996 note sections without keyword filtering was utilized. We developed prompts for two LLMs, Llama 2 and GPT-4, on HIPAA-compliant cloud-computing platforms using multiple approaches (e.g., both hard and soft prompting and error analysis-based instructions) to select the optimal LLM-based method. Baseline models included a hierarchical attention-based neural network and XGBoost. Subsequently, we constructed an ensemble of the three models using a majority vote approach.

Results

GPT-4 demonstrated superior accuracy and efficiency compared to Llama 2, but did not outperform traditional models. The ensemble model outperformed the individual models, achieving a precision of 90.3%, a recall of 94.2%, and an F1-score of 92.2%. Notably, the ensemble model showed a significant improvement in precision, increasing from a range of 70%-79% to above 90%, compared to the best-performing single model. Error analysis revealed that 63 samples were incorrectly predicted by at least one model; however, only 2 cases (3.2%) were mutual errors across all models, indicating diverse error profiles among them.

Conclusions

LLMs and traditional machine learning models trained using local EHR data exhibited diverse error profiles. The ensemble of these models was found to be complementary, enhancing diagnostic performance. Future research should investigate integrating LLMs with smaller, localized models and incorporating medical data and domain knowledge to enhance performance on specific tasks.

Risk Management and Patient Safety in the Artificial Intelligence Era: A Systematic Review

BACKGROUND

Healthcare systems represent complex organizations within which multiple factors (physical environment, human factor, technological devices, quality of care) interconnect to form a dense network whose imbalance is potentially able to compromise patient safety. In this scenario, the need for hospitals to expand reactive and proactive clinical risk management programs is easily understood, and artificial intelligence fits well in this context. This systematic review aims to investigate the state of the art regarding the impact of AI on clinical risk management processes. To simplify the analysis of the review outcomes and to motivate future standardized comparisons with any subsequent studies, the findings of the present review will be grouped according to the possibility of applying AI in the prevention of the different incident type groups as defined by the ICPS.

MATERIALS AND METHODS

On 3 November 2023, a systematic review of the literature according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was carried out using the SCOPUS and Medline (via PubMed) databases. A total of 297 articles were identified. After the selection process, 36 articles were included in the present systematic review.

RESULTS AND DISCUSSION

The studies included in this review allowed for the identification of three main "incident type" domains: clinical process, healthcare-associated infection, and medication. Another relevant application of AI in clinical risk management concerns the topic of incident reporting.

CONCLUSIONS

This review highlighted that AI can be applied transversely in various clinical contexts to enhance patient safety and facilitate the identification of errors. It appears to be a promising tool to improve clinical risk management, although its use requires human supervision and cannot completely replace human skills. To facilitate the analysis of the present review outcome and to enable comparison with future systematic reviews, it was deemed useful to refer to a pre-existing taxonomy for the identification of adverse events. However, the results of the present study highlighted the usefulness of AI not only for risk prevention in clinical practice, but also in improving the use of an essential risk identification tool, which is incident reporting. For this reason, the taxonomy of the areas of application of AI to clinical risk processes should include an additional class relating to risk identification and analysis tools. For this purpose, it was considered convenient to use ICPS classification.

Identifying Functional Status Impairment in People Living With Dementia Through Natural Language Processing of Clinical Documents: Cross-Sectional Study

BACKGROUND

Assessment of activities of daily living (ADLs) and instrumental ADLs (iADLs) is key to determining the severity of dementia and care needs among older adults. However, such information is often only documented in free-text clinical notes within the electronic health record and can be challenging to find.

OBJECTIVE

This study aims to develop and validate machine learning models to determine the status of ADL and iADL impairments based on clinical notes.

METHODS

This cross-sectional study leveraged electronic health record clinical notes from Mass General Brigham's Research Patient Data Repository linked with Medicare fee-for-service claims data from 2007 to 2017 to identify individuals aged 65 years or older with at least 1 diagnosis of dementia. Notes for encounters both 180 days before and after the first date of dementia diagnosis were randomly sampled. Models were trained and validated using note sentences filtered by expert-curated keywords (filtered cohort) and further evaluated using unfiltered sentences (unfiltered cohort). The model's performance was compared using area under the receiver operating characteristic curve and area under the precision-recall curve (AUPRC).

RESULTS

The study included 10,000 key-term-filtered sentences representing 441 people (n=283, 64.2% women; mean age 82.7, SD 7.9 years) and 1000 unfiltered sentences representing 80 people (n=56, 70% women; mean age 82.8, SD 7.5 years). Area under the receiver operating characteristic curve was high for the best-performing ADL and iADL models on both cohorts (>0.97). For ADL impairment identification, the random forest model achieved the best AUPRC (0.89, 95% CI 0.86-0.91) on the filtered cohort; the support vector machine model achieved the highest AUPRC (0.82, 95% CI 0.75-0.89) for the unfiltered cohort. For iADL impairment, the Bio+Clinical bidirectional encoder representations from transformers (BERT) model had the highest AUPRC (filtered: 0.76, 95% CI 0.68-0.82; unfiltered: 0.58, 95% CI 0.001-1.0). Compared with a keyword-search approach on the unfiltered cohort, machine learning reduced false-positive rates from 4.5% to 0.2% for ADL and 1.8% to 0.1% for iADL.

CONCLUSIONS

In this study, we demonstrated the ability of machine learning models to accurately identify ADL and iADL impairment based on free-text clinical notes, which could be useful in determining the severity of dementia.

IFN-γ ELISpot-enabled machine learning for culprit drug identification in nonimmediate drug hypersensitivity

BACKGROUND

Diagnosing drug-induced allergy, especially nonimmediate phenotypes, is challenging. Incorrect classifications have unwanted consequences.

OBJECTIVE

We sought to evaluate the diagnostic utility of IFN-γ ELISpot and clinical parameters in predicting drug-induced nonimmediate hypersensitivity using machine learning.

METHODS

The study recruited 393 patients. A positive patch test or drug provocation test (DPT) was used to define positive drug hypersensitivity. Various clinical factors were considered in developing random forest (RF) and logistic regression (LR) models. Performances were compared against the IFN-γ ELISpot-only model.

RESULTS

Among the 102 patients who had 164 DPTs, most patients had severe cutaneous adverse reactions (35/102, 34.3%) and maculopapular exanthems (33/102, 32.4%). Common suspected drugs were antituberculosis drugs (46/164, 28.1%) and β-lactams (42/164, 25.6%). Mean (SD) age of patients with DPT was 52.7 (20.8) years. IFN-γ ELISpot, fixed drug eruption, Naranjo categories, and nonsteroidal anti-inflammatory drugs were the most important features in all developed models. The RF and LR models had higher discriminating abilities. An IFN-γ ELISpot cutoff value of 16.0 spot-forming cells/106 PBMCs achieved 94.8% specificity and 57.1% sensitivity. Depending on clinical needs, optimal cutoff values for RF and LR models can be chosen to achieve either high specificity (0.41 for 96.1% specificity and 0.52 for 97.4% specificity, respectively) or high sensitivity (0.26 for 78.6% sensitivity and 0.37 for 71.4% sensitivity, respectively).

CONCLUSIONS

IFN-γ ELISpot assay was valuable in identifying culprit drugs, whether used individually or incorporated in a prediction model. Performances of RF and LR models were comparable. Additional test datasets with DPT would be helpful to validate the model further.

A Novel Informatics Tool to Detect Periprocedural Antibiotic Allergy Adverse Events for Near Real-time Surveillance to Support Audit and Feedback

Importance

Standardized processes for identifying when allergic-type reactions occur and linking reactions to drug exposures are limited.

Objective

To develop an informatics tool to improve detection of antibiotic allergic-type events.

Design, Setting, and Participants

This retrospective cohort study was conducted from October 1, 2015, to September 30, 2019, with data analyzed between July 1, 2021, and January 31, 2022. The study was conducted across Veteran Affairs hospitals among patients who underwent cardiovascular implantable electronic device (CIED) procedures and received periprocedural antibiotic prophylaxis. The cohort was split into training and test cohorts, and cases were manually reviewed to determine presence of allergic-type reaction and its severity. Variables potentially indicative of allergic-type reactions were selected a priori and included allergies entered in the Veteran Affair's Allergy Reaction Tracking (ART) system (either historical [reported] or observed), allergy diagnosis codes, medications administered to treat allergic reactions, and text searches of clinical notes for keywords and phrases indicative of a potential allergic-type reaction. A model to detect allergic-type reaction events was iteratively developed on the training cohort and then applied to the test cohort. Algorithm test characteristics were assessed.

Exposure

Preprocedural and postprocedural prophylactic antibiotic administration.

Main Outcomes and Measures

Antibiotic allergic-type reactions.

Results

The cohort of 36 344 patients included 34 703 CIED procedures with antibiotic exposures (mean [SD] age, 72 [10] years; 34 008 [98%] male patients); median duration of postprocedural prophylaxis was 4 days (IQR, 2-7 days; maximum, 45 days). The final algorithm included 7 variables: entries in the Veteran Affair's hospitals ART, either historic (odds ratio [OR], 42.37; 95% CI, 11.33-158.43) or observed (OR, 175.10; 95% CI, 44.84-683.76); PheCodes for "symptoms affecting skin" (OR, 8.49; 95% CI, 1.90-37.82), "urticaria" (OR, 7.01; 95% CI, 1.76-27.89), and "allergy or adverse event to an antibiotic" (OR, 11.84, 95% CI, 2.88-48.69); keyword detection in clinical notes (OR, 3.21; 95% CI, 1.27-8.08); and antihistamine administration alone or in combination (OR, 6.51; 95% CI, 1.90-22.30). In the final model, antibiotic allergic-type reactions were identified with an estimated probability of 30% or more; positive predictive value was 61% (95% CI, 45%-76%); and sensitivity was 87% (95% CI, 70%-96%).

Conclusions and Relevance

In this retrospective cohort study of patients receiving periprocedural antibiotic prophylaxis, an algorithm with a high sensitivity to detect incident antibiotic allergic-type reactions that can be used to provide clinician feedback about antibiotic harms from unnecessarily prolonged antibiotic exposures was developed.

Allergy Safety Events in Health Care: Development and Application of a Classification Schema Based on Retrospective Review

BACKGROUND

Allergy safety requires understanding the operational processes that expose patients to their known allergens, including how and when such processes fail.

OBJECTIVE

To improve health care safety for patients with allergies, we developed and assessed an allergy safety event classification schema to describe failures resulting in allergy-related safety events.

METHODS

Using keyword searches followed by expert manual review of 299,031 voluntarily-filed safety event reports at 2 large academic medical centers, we identified and classified allergy-related safety events from 5 years of safety reports. We used driver diagrams to elucidate root causes for commonly observed allergy safety events in health care settings.

RESULTS

From 299,031 safety reports, 1922 (0.6%) were extracted with keywords and 744 (0.2%) were manually confirmed as allergy-related safety events. Safety failures were due to incomplete/inaccurate electronic health record documentation (n = 375, 50.4%), human factors (n = 175, 23.5%), allergy alert limitation and/or malfunction (n = 127, 17.1%), data exchange and interoperability failures (n = 92, 12.4%), and electronic health record system default options (n = 30, 4.0%). Safety failures resulted in known allergen exposures to drugs (n = 537), including heparin (n = 27) and topical anesthetics such as lidocaine (n = 8); latex (n = 114); food allergens (n = 73); and adhesive (n = 23).

CONCLUSIONS

We identified 744 allergy-related safety events to inform a novel safety failure classification schema as an important step toward a safer health care environment for patients with allergies. Improved systems are required to address safety issues with certain food and drug allergens.

Novel Analysis Methods to Mine Immune-Mediated Phenotypes and Find Genetic Variation Within the Electronic Health Record (Roadmap for Phenotype to Genotype: Immunogenomics)

The field of immunogenomics has the opportunity for accelerated genetic discovery aided by the maturation of electronic health records (EHRs) linked to DNA biobanks. Novel analysis methods in deep phenotyping of EHR data allow the full realization of the paired and increasingly dense genetic/phenotypic information available. This enables researchers to uncover genetic risk factors for the prevention and optimal treatment of immune-mediated diseases and immune-mediated adverse drug reactions. This article reviews the background of EHRs linked to DNA biobanks, potential applications to immunogenomic discovery, and current and emerging techniques in EHR-based deep phenotyping.

Data Science Trends Relevant to Nursing Practice: A Rapid Review of the 2020 Literature

BACKGROUND

The term "data science" encompasses several methods, many of which are considered cutting edge and are being used to influence care processes across the world. Nursing is an applied science and a key discipline in health care systems in both clinical and administrative areas, making the profession increasingly influenced by the latest advances in data science. The greater informatics community should be aware of current trends regarding the intersection of nursing and data science, as developments in nursing practice have cross-professional implications.

OBJECTIVES

This study aimed to summarize the latest (calendar year 2020) research and applications of nursing-relevant patient outcomes and clinical processes in the data science literature.

METHODS

We conducted a rapid review of the literature to identify relevant research published during the year 2020. We explored the following 16 topics: (1) artificial intelligence/machine learning credibility and acceptance, (2) burnout, (3) complex care (outpatient), (4) emergency department visits, (5) falls, (6) health care-acquired infections, (7) health care utilization and costs, (8) hospitalization, (9) in-hospital mortality, (10) length of stay, (11) pain, (12) patient safety, (13) pressure injuries, (14) readmissions, (15) staffing, and (16) unit culture.

RESULTS

Of 16,589 articles, 244 were included in the review. All topics were represented by literature published in 2020, ranging from 1 article to 59 articles. Numerous contemporary data science methods were represented in the literature including the use of machine learning, neural networks, and natural language processing.

CONCLUSION

This review provides an overview of the data science trends that were relevant to nursing practice in 2020. Examinations of such literature are important to monitor the status of data science's influence in nursing practice.

PASCLex: A comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon derived from electronic health record clinical notes

OBJECTIVE

To develop a comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon (PASCLex) from clinical notes to support PASC symptom identification and research.

METHODS

We identified 26,117 COVID-19 positive patients from the Mass General Brigham's electronic health records (EHR) and extracted 328,879 clinical notes from their post-acute infection period (day 51-110 from first positive COVID-19 test). PASCLex incorporated Unified Medical Language System® (UMLS) Metathesaurus concepts and synonyms based on selected semantic types. The MTERMS natural language processing (NLP) tool was used to automatically extract symptoms from a development dataset. The lexicon was iteratively revised with manual chart review, keyword search, concept consolidation, and evaluation of NLP output. We assessed the comprehensiveness of PASCLex and the NLP performance using a validation dataset and reported the symptom prevalence across the entire corpus.

RESULTS

PASCLex included 355 symptoms consolidated from 1520 UMLS concepts of 16,466 synonyms. NLP achieved an averaged precision of 0.94 and an estimated recall of 0.84. Symptoms with the highest frequency included pain (43.1%), anxiety (25.8%), depression (24.0%), fatigue (23.4%), joint pain (21.0%), shortness of breath (20.8%), headache (20.0%), nausea and/or vomiting (19.9%), myalgia (19.0%), and gastroesophageal reflux (18.6%).

DISCUSSION AND CONCLUSION

PASC symptoms are diverse. A comprehensive lexicon of PASC symptoms can be derived using an ontology-driven, EHR-guided and NLP-assisted approach. By using unstructured data, this approach may improve identification and analysis of patient symptoms in the EHR, and inform prospective study design, preventative care strategies, and therapeutic interventions for patient care.

Development and Validation of a Deep Learning Model for Earlier Detection of Cognitive Decline From Clinical Notes in Electronic Health Records

IMPORTANCE

Detecting cognitive decline earlier among older adults can facilitate enrollment in clinical trials and early interventions. Clinical notes in longitudinal electronic health records (EHRs) provide opportunities to detect cognitive decline earlier than it is noted in structured EHR fields as formal diagnoses.

OBJECTIVE

To develop and validate a deep learning model to detect evidence of cognitive decline from clinical notes in the EHR.

DESIGN, SETTING, AND PARTICIPANTS

Notes documented 4 years preceding the initial mild cognitive impairment (MCI) diagnosis were extracted from Mass General Brigham's Enterprise Data Warehouse for patients aged 50 years or older and with initial MCI diagnosis during 2019. The study was conducted from March 1, 2020, to June 30, 2021. Sections of notes for cognitive decline were labeled manually and 2 reference data sets were created. Data set I contained a random sample of 4950 note sections filtered by a list of keywords related to cognitive functions and was used for model training and testing. Data set II contained 2000 randomly selected sections without keyword filtering for assessing whether the model performance was dependent on specific keywords.

MAIN OUTCOMES AND MEASURES

A deep learning model and 4 baseline models were developed and their performance was compared using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC).

RESULTS

Data set I represented 1969 patients (1046 [53.1%] women; mean [SD] age, 76.0 [13.3] years). Data set II comprised 1161 patients (619 [53.3%] women; mean [SD] age, 76.5 [10.2] years). With some overlap of patients deleted, the unique population was 2166. Cognitive decline was noted in 1453 sections (29.4%) in data set I and 69 sections (3.45%) in data set II. Compared with the 4 baseline models, the deep learning model achieved the best performance in both data sets, with AUROC of 0.971 (95% CI, 0.967-0.976) and AUPRC of 0.933 (95% CI, 0.921-0.944) for data set I and AUROC of 0.997 (95% CI, 0.994-0.999) and AUPRC of 0.929 (95% CI, 0.870-0.969) for data set II.

CONCLUSIONS AND RELEVANCE

In this diagnostic study, a deep learning model accurately detected cognitive decline from clinical notes preceding MCI diagnosis and had better performance than keyword-based search and other machine learning models. These results suggest that a deep learning model could be used for earlier detection of cognitive decline in the EHRs.

Anaphylaxis and digital medicine

PURPOSE OF REVIEW

Digital medicine (mHealth) aims to help patients and healthcare providers (HCPs) improve and facilitate the provision of patient care. It encompasses equipment/connected medical devices, mHealth services and mHealth apps (apps). An updated review on digital health in anaphylaxis is proposed.

RECENT FINDINGS

In anaphylaxis, mHealth is used in electronic health records and registries.It will greatly benefit from the new International Classification of Diseases-11 rules and artificial intelligence. Telehealth has been revolutionised by the coronavirus disease 2019 pandemic, and lessons learnt should be extended to shared decision making in anaphylaxis. Very few nonvalidated apps exist and there is an urgent need to develop and validate such tools.

SUMMARY

Although digital health appears to be of great importance in anaphylaxis, it is still insufficiently used.

The development of a machine learning algorithm to identify occupational injuries in agriculture using pre-hospital care reports

Purpose

Current injury surveillance efforts in agriculture are considerably hampered by the limited quantity of occupation or industry data in current health records. This has impeded efforts to develop more accurate injury burden estimates and has negatively impacted the prioritization of workplace health and safety in state and federal public health efforts. This paper describes the development of a Naïve Bayes machine learning algorithm to identify occupational injuries in agriculture using existing administrative data, specifically in pre-hospital care reports (PCR).

Methods

A Naïve Bayes machine learning algorithm was trained on PCR datasets from 2008–2010 from Maine and New Hampshire and tested on newer data from those states between 2011 and 2016. Further analyses were devoted to establishing the generalizability of the model across various states and various years. Dual visual inspection was used to verify the records subset by the algorithm.

Results

The Naïve Bayes machine learning algorithm reduced the volume of cases that required visual inspection by 69.5 percent over a keyword search strategy alone. Coders identified 341 true agricultural injury records (Case class = 1) (Maine 2011–2016, New Hampshire 2011–2015). In addition, there were 581 (Case class = 2 or 3) that were suspected to be agricultural acute/traumatic events, but lacked the necessary detail to make a certain distinction.

Conclusions

The application of the trained algorithm on newer data reduced the volume of records requiring visual inspection by two thirds over the previous keyword search strategy, making it a sustainable and cost-effective way to understand injury trends in agriculture.

Conducting Safety Research Safely: A Policy-Based Approach for Conducting Research with Peer Review Protected Material

A multidisciplinary team developed a policy-based approach that provides guidance for using peer review protected information for safety research while maintaining peer review privilege. The approach includes project approval by an ad hoc review committee, signed confidentiality agreements by investigators and study staff, early removal of case identification numbers, standards for maintaining data security, and publication of aggregate data without data set sharing. By describing this procedure and embedding into an institutional policy on Data for Performance Improvement, the team encourages other institutions to develop similar policies consistent with their state regulations.