Application of a Natural Language Processing Algorithm to Asthma Ascertainment. An Automated Chart Review

Comparative Analysis of the Accuracy of Microsoft Excel Macros in Retrospective Chart Review Studies

INTRODUCTION

While retrospective chart review is a useful methodology for clinical research, challenges still exist when abstracting data from the electronic health record. When collected manually, unstructured "free text" data are particularly tedious and can be susceptible to errors and biases. We aimed to evaluate the accuracy of Microsoft Excel macros to facilitate the data abstraction process.

METHODS

One hundred pathology reports following surgery for thyroid cancer were retrospectively evaluated. Twenty variables of interest (tumor characteristics, invasive features, and lymph node counts) were manually abstracted by a physician reviewer. A macro ("ThyMAC") was developed to extract the same variables. Abstraction error rates and speed were measured between manual and macro-assisted methods using a paired t-test. Accuracy, classification rates, and interrater reliability of ThyMAC were then analyzed. After identifying correctable errors, an ad hoc analysis of the optimized macro was then performed.

RESULTS

Abstraction errors by physician reviewer were slightly lower relative to ThyMAC (3.8 versus 5.3% error rate, P = 0.03). By contrast, data collection time was 270 times faster via macro-assistance (65 versus 0.24 s per pathology report, P < 0.001). Overall, ThyMAC performed with high rates of accuracy (87-100%) for all abstracted variables, with moderate-to-perfect agreement for 14 of 20 variables. Addressing correctable errors significantly decreased macro error rates compared to the physician abstractor (3.6 versus 0.5%, P < 0.001).

CONCLUSIONS

Compared to a trained physician abstractor, macros can extract unstructured data in retrospective chart review studies with high accuracy at speeds superior to a manual approach. Macro errors are typically preventable, and the program can be modified to improve data extraction accuracy. Macros can serve as an efficient and versatile tool to assist researchers with chart review data collection, especially when large datasets are involved.

Association of delayed asthma diagnosis with asthma exacerbations in children

Background

There is a significant delay between symptom onset and diagnosis of childhood asthma, but the impact of this delay on asthma outcomes has not been well understood.

Objectives

We sought to study the association of delayed diagnosis of asthma with asthma exacerbations (AEs) in children.

Methods

Using the Mayo Clinic birth cohort, we identified children with a diagnosis of asthma from electronic health records. We defined onset date as the date when subjects first met predetermined asthma criteria ascertained by an electronic health records-based natural language processing algorithm. Delay in diagnosis (DD) was defined as first diagnosis >30 days from onset date (vs timely diagnosis [TD] within 30 days). The primary outcome was AE after the index date (for DD: first diagnosis date vs for TD: clinic visit at similar delay from diagnosis as matched DD counterpart). A Cox proportional hazard model was used to test the association between delayed diagnosis status and risk of AE, adjusting for sociodemographics, care quality, and asthma severity.

Results

Among 537 matched pairs of DD and TD (median age at index date: 4.1 years), a total of 344 and 253 children in DD and TD, respectively, had ≥1 AE during median follow-up period of 9.3 years. Children in the DD group had a significantly increased risk of AE compared to TD (adjusted hazard ratio: 1.53; 95% CI: 1.28, 1.80; P < .001).

Conclusions

DD of asthma in children is associated with an increased risk of AE compared to TD. TD of asthma should be an important priority in childhood asthma management.

AI model for predicting asthma prognosis in children

Background

Childhood asthma often continues into adulthood, but some children experience remission. Utilizing electronic health records (EHRs) to predict asthma prognosis can aid health care providers and patients in developing effective prioritized care plans.

Objective

We aimed to develop artificial intelligence (AI) models using various clinical variables extracted from EHRs to predict childhood asthma prognosis (remission vs no remission) in different age groups.

Methods

We developed AI models utilizing patients' EHRs during the first 6, 9, or 12 years of their lives to predict their asthma prognosis status at ages 6 to 9, 9 to 12, or 12 to 15 years, respectively. We first developed the models based on a manually annotated birth cohort (n = 900). We then leveraged a larger birth cohort (n = 29,594) labeled automatically (with weak labels) by a previously validated natural language processing algorithm for asthma prognosis. Different models (logistic regression, random forest, and XGBoost [eXtreme Gradient Boosting]) were tested with diverse clinical variables from structured and unstructured EHRs.

Results

The best AI models of each age group produced a prediction performance with areas under the receiver operating characteristic curve ranging from 0.85 to 0.93. The prediction model at age 12 showed the highest performance. Most of the AI models with weak labels showed enhanced performance, and models using the top 10 variables performed similarly to those using all of the variables.

Conclusions

The AI models effectively predicted asthma prognosis for children by using EHRs with a relatively small number of variables. This approach demonstrates the potential to enhance prioritized care plans and patient education, improving disease management and quality of life for asthmatic patients.

Opportunities and challenges with artificial intelligence in allergy and immunology: a bibliometric study

Introduction

The fields of allergy and immunology are increasingly recognizing the transformative potential of artificial intelligence (AI). Its adoption is reshaping research directions, clinical practices, and healthcare systems. However, a systematic overview identifying current statuses, emerging trends, and future research hotspots is lacking.

Methods

This study applied bibliometric analysis methods to systematically evaluate the global research landscape of AI applications in allergy and immunology. Data from 3,883 articles published by 21,552 authors across 1,247 journals were collected and analyzed to identify leading contributors, prevalent research themes, and collaboration patterns.

Results

Analysis revealed that the USA and China are currently leading in research output and scientific impact in this domain. AI methodologies, especially machine learning (ML) and deep learning (DL), are predominantly applied in drug discovery and development, disease classification and prediction, immune response modeling, clinical decision support, diagnostics, healthcare system digitalization, and medical education. Emerging trends indicate significant movement toward personalized medical systems integration.

Discussion

The findings demonstrate the dynamic evolution of AI in allergy and immunology, highlighting the broadening scope from basic diagnostics to comprehensive personalized healthcare systems. Despite advancements, critical challenges persist, including technological limitations, ethical concerns, and regulatory frameworks that could potentially hinder further implementation and integration.

Conclusion

AI holds considerable promise for advancing allergy and immunology globally by enhancing healthcare precision, efficiency, and accessibility. Addressing existing technological, ethical, and regulatory challenges will be crucial to fully realizing its potential, ultimately improving global health outcomes and patient well-being.

Using Electronic Health Records to Identify Asthma-Related Acute Care Encounters

OBJECTIVE

Leveraging "big data" to improve care requires that clinical concepts be operationalized using available data. Electronic health record (EHR) data can be used to evaluate asthma care, but relying solely on diagnosis codes may misclassify asthma-related encounters. We created streamlined, feasible and transparent prototype algorithms for EHR data to classify emergency department (ED) encounters and hospitalizations as "asthma-related."

METHODS

As part of an asthma program evaluation, expert clinicians conducted a multi-phase iterative chart review to evaluate 467 pediatric ED encounters and 136 hospitalizations with asthma diagnosis codes from calendar years 2017 and 2019, rating the likelihood that each encounter was actually asthma-related. Using this as a reference standard, we developed rule-based algorithms for EHR data to classify visits. Accuracy was evaluated using sensitivity, specificity, and positive and negative predictive values (PPV, NPV).

RESULTS

Clinicians categorized 38% of ED encounters as "definitely" or "probably" asthma-related; 13% as "possibly" asthma-related; and 49% as "probably not" or "definitely not" related to asthma. Based on this reference standard, we created two rule-based algorithms to identify "definitely" or "probably" asthma-related encounters, one using text and non-text EHR fields and another using non-text fields only. Sensitivity, specificity, PPV, and NPV were >95% for the algorithm using text and non-text fields and >87% for the algorithm using only non-text fields compared to the reference standard. We created a two-rule algorithm to identify asthma-related hospitalizations using only non-text fields.

CONCLUSIONS

Diagnostic codes alone are insufficient to identify asthma-related visits, but EHR-based prototype algorithms that include additional methods of identification can predict clinician-identified visits with sufficient accuracy.

Sex differences in sociodemographic, clinical, and laboratory variables in childhood asthma: A birth cohort study

BACKGROUND

There are marked sex differences in the prevalence and severity of asthma, both during childhood and adulthood. There is a relative lack of comprehensive studies exploring sexdifferences in pediatric asthma cohorts.

OBJECTIVE

To identify the most relevant sex differences in sociodemographic, clinical, and laboratory variables in a well-characterized large pediatric asthma cohort.

METHODS

We performed a cross-sectional analysis of the Mayo Clinic Olmsted County Birth Cohort. In the full birth cohort, we used a natural language-processing algorithm based on the Predetermined Asthma Criteria for asthma ascertainment. In a stratified random sample of 300 children, we obtained additional pulmonary function tests and laboratory data. We identified the significant sex differences among available sociodemographic, clinical, and laboratory variables.

RESULTS

Boys were more frequently diagnosed with having asthma than girls and were younger at the time of asthma diagnosis. There were no sex differences in relation to socioeconomic status. We identified a male predominance in the presence of a tympanostomy tube and a female predominance in the history of pneumonia. A higher percentage of boys had a forced expiratory volume in 1 second/forced vital capacity ratio less than 0.85. Blood eosinophilia and atopic sensitization were also more common in boys. Finally, boys had higher levels of serum periostin than girls.

CONCLUSION

This study described significant sex differences in a large pediatric asthma cohort. Overall, boys had earlier and more severe asthma than girls. Differences in blood eosinophilia and serum periostin provide insights into possible mechanisms of the sex bias in childhood asthma.

Current technological advancement in asthma care

INTRODUCTION

Asthma is a common chronic respiratory disease affecting 262 million people globally, causing half a million deaths each year. Poor asthma outcomes are frequently due to non-adherence to medication, poor engagement with asthma services, and a lack of objective diagnostic tests. In recent years, technologies have been developed to improve diagnosis, monitoring, and care.

AREAS COVERED

Technology has impacted asthma care with the potential to improve patient outcomes, reduce healthcare costs, and provide personalized management. We focus on current evidence on home diagnostics and monitoring, remote asthma reviews, and digital smart inhalers. PubMed, Ovid/Embase, Cochrane Library, Scopus and Google Scholar were searched in November 2023 with no limit by year of publication.

EXPERT OPINION

Advanced diagnostic technologies have enabled early asthma detection and personalized treatment plans. Mobile applications and digital therapeutics empower patients to manage their condition and improve adherence to treatments. Telemedicine platforms and remote monitoring devices have the potential to streamline asthma care. AI algorithms can analyze patient data and predict exacerbations in proof-of-concept studies. Technology can potentially provide precision medicine to a wider patient group in the future, but further development is essential for implementation into routine care which in itself will be a major challenge.

Heterologous Production of Antimicrobial Peptides: Notes to Consider

Heavy and irresponsible use of antibiotics in the last century has put selection pressure on the microbes to evolve even faster and develop more resilient strains. In the confrontation with such sometimes called "superbugs", the search for new sources of biochemical antibiotics seems to have reached the limit. In the last two decades, bioactive antimicrobial peptides (AMPs), which are polypeptide chains with less than 100 amino acids, have attracted the attention of many in the control of microbial pathogens, more than the other types of antibiotics. AMPs are groups of components involved in the immune response of many living organisms, and have come to light as new frontiers in fighting with microbes. AMPs are generally produced in minute amounts within organisms; therefore, to address the market, they have to be either produced on a large scale through recombinant DNA technology or to be synthesized via chemical methods. Here, heterologous expression of AMPs within bacterial, fungal, yeast, plants, and insect cells, and points that need to be considered towards their industrialization will be reviewed.

Study on Predicting Clinical Stage of Patients with Bronchial Asthma Based on CT Radiomics

Objective

To explore the value of a new model based on CT radiomics in predicting the staging of patients with bronchial asthma (BA).

Methods

Patients with BA from 2018 to 2021 were retrospectively analyzed and underwent plain chest CT before treatment. According to the guidelines for the prevention and treatment of BA (2016 edition), they were divided into two groups: acute attack and non-acute attack. The images were processed as follows: using Lung Kit software for image standardization and segmentation, using AK software for image feature extraction, and using R language for data analysis and model construction (training set: test set = 7: 3). The efficacy and clinical effects of the constructed model were evaluated with ROC curve, sensitivity, specificity, calibration curve and decision curve.

Results

A total of 112 patients with BA were enrolled, including 80 patients with acute attack (range: 2-86 years old, mean: 53.89±17.306 years old, males of 33) and 32 patients with non-acute attack (range: 4-79 years old, mean: 57.38±19.223 years old, males of 18). A total of 10 imaging features are finally retained and used to construct model using multi-factor logical regression method. In the training group, the AUC, sensitivity and specificity of the model was 0.881 (95% CI:0.808-0.955), 0.804 and 0.818, separately; while in the test group, it was 0.792 (95% CI:0.608-0.976), 0.792 and 0.80, respectively.

Conclusion

The model constructed based on radiomics has a good effect on predicting the staging of patients with BA, which provides a new method for clinical diagnosis of staging in BA patients.

Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: A systematic review

OBJECTIVE

Natural language processing (NLP) combined with machine learning (ML) techniques are increasingly used to process unstructured/free-text patient-reported outcome (PRO) data available in electronic health records (EHRs). This systematic review summarizes the literature reporting NLP/ML systems/toolkits for analyzing PROs in clinical narratives of EHRs and discusses the future directions for the application of this modality in clinical care.

METHODS

We searched PubMed, Scopus, and Web of Science for studies written in English between 1/1/2000 and 12/31/2020. Seventy-nine studies meeting the eligibility criteria were included. We abstracted and summarized information related to the study purpose, patient population, type/source/amount of unstructured PRO data, linguistic features, and NLP systems/toolkits for processing unstructured PROs in EHRs.

RESULTS

Most of the studies used NLP/ML techniques to extract PROs from clinical narratives (n = 74) and mapped the extracted PROs into specific PRO domains for phenotyping or clustering purposes (n = 26). Some studies used NLP/ML to process PROs for predicting disease progression or onset of adverse events (n = 22) or developing/validating NLP/ML pipelines for analyzing unstructured PROs (n = 19). Studies used different linguistic features, including lexical, syntactic, semantic, and contextual features, to process unstructured PROs. Among the 25 NLP systems/toolkits we identified, 15 used rule-based NLP, 6 used hybrid NLP, and 4 used non-neural ML algorithms embedded in NLP.

CONCLUSIONS

This study supports the potential utility of different NLP/ML techniques in processing unstructured PROs available in EHRs for clinical care. Though using annotation rules for NLP/ML to analyze unstructured PROs is dominant, deploying novel neural ML-based methods is warranted.

Ascertaining asthma status in epidemiologic studies: a comparison between administrative health data and self-report

BACKGROUND

Studies have suggested that agreement between administrative health data and self-report for asthma status ranges from fair to good, but few studies benefited from administrative health data over a long period. We aimed to (1) evaluate agreement between asthma status ascertained in administrative health data covering a period of 30 years and from self-report, and (2) identify determinants of agreement between the two sources.

METHODS

We used administrative health data (1983-2012) from the Quebec Birth Cohort on Immunity and Health, which included 81,496 individuals born in the province of Quebec, Canada, in 1974. Additional information, including self-reported asthma, was collected by telephone interview with 1643 participants in 2012. By design, half of them had childhood asthma based on health services utilization. Results were weighted according to the inverse of the sampling probabilities. Five algorithms were applied to administrative health data (having ≥ 2 physician claims over a 1-, 2-, 3-, 5-, or 30-year interval or ≥ 1 hospitalization), to enable comparisons with previous studies. We estimated the proportion of overall agreement and Kappa, between asthma status derived from algorithms and self-reports. We used logistic regression to identify factors associated with agreement.

RESULTS

Applying the five algorithms, the prevalence of asthma ranged from 49 to 55% among the 1643 participants. At interview (mean age = 37 years), 49% and 47% of participants respectively reported ever having asthma and asthma diagnosed by a physician. Proportions of agreement between administrative health data and self-report ranged from 88 to 91%, with Kappas ranging from 0.57 (95% CI: 0.52-0.63) to 0.67 (95% CI: 0.62-0.72); the highest values were obtained with the [≥ 2 physician claims over a 30-year interval or ≥ 1 hospitalization] algorithm. Having sought health services for allergic diseases other than asthma was related to lower agreement (Odds ratio = 0.41; 95% CI: 0.25-0.65 comparing ≥ 1 health services to none).

CONCLUSIONS

These findings indicate good agreement between asthma status defined from administrative health data and self-report. Agreement was higher than previously observed, which may be due to the 30-year lookback window in administrative data. Our findings support using both administrative health data and self-report in population-based epidemiological studies.

The Digital Hospital: A Scoping Review of How Technology Is Transforming Cardiopulmonary Care

BACKGROUND

Innovative models of health care that involve advanced technology in the form of a digital hospital are emerging globally. Models include technology such as machine learning and smart wearables, that can be used to integrate patient data and improve continuity of care. This model may have benefits in situations where patient deterioration must be detected quickly so that a rapid response can occur such as cardiopulmonary settings.

AIM

The purpose of this scoping review was to examine the evidence for a digital hospital model of care, in the context of cardiac and pulmonary settings.

DESIGN

Scoping review.

DATA SOURCES

Databases searched were using PsycInfo, Ovid MEDLINE, and CINAHL. Studies written in English and containing key terms related to digital hospital and cardiopulmonary care were included. The Joanna Briggs Institute methodology for systematic reviews was used to assess the risk of bias.

RESULTS

Thirteen (13) studies fulfilled the inclusion criteria. For cardiac conditions, a deep-learning-based rapid response system warning system for predicting patient deterioration leading to cardiac arrest had up to 257% higher sensitivity than conventional methods. There was also a reduction in the number of patients who needed to be examined by a physician. Using continuous telemonitoring with a wireless real-time electrocardiogram compared with non-monitoring, there was improved initial resuscitation and 24-hour post-event survival for high-risk patients. However, there were no benefits for survival to discharge. For pulmonary conditions, a natural language processing algorithm reduced the time to asthma diagnosis, demonstrating high predictive values. Virtual inhaler education was found to be as effective as in-person education, and prescription error was reduced following the implementation of computer-based physician order entry electronic medical records and a clinical decision support tool.

CONCLUSIONS

While we currently have only a brief glimpse at the impact of technology care delivery for cardiac and respiratory conditions, technology presents an opportunity to improve quality and safety in care, but only with the support of adequate infrastructure and processes.

PROTOCOL REGISTRATION

Open Science Framework (OSF: DOI 10.17605/OSF.IO/PS6ZU).

Machine learning model for classification of predominantly allergic and non-allergic asthma among preschool children with asthma hospitalization

OBJECTIVE

Asthma is the most frequent chronic airway illness in preschool children and is difficult to diagnose due to the disease's heterogeneity. This study aimed to investigate different machine learning models and suggested the most effective one to classify two forms of asthma in preschool children (predominantly allergic asthma and non-allergic asthma) using a minimum number of features.

METHODS

After pre-processing, 127 patients (70 with non-allergic asthma and 57 with predominantly allergic asthma) were chosen for final analysis from the Frankfurt dataset, which had asthma-related information on 205 patients. The Random Forest algorithm and Chi-square were used to select the key features from a total of 63 features. Six machine learning models: random forest, extreme gradient boosting, support vector machines, adaptive boosting, extra tree classifier, and logistic regression were then trained and tested using 10-fold stratified cross-validation.

RESULTS

Among all features, age, weight, C-reactive protein, eosinophilic granulocytes, oxygen saturation, pre-medication inhaled corticosteroid + long-acting beta2-agonist (PM-ICS + LABA), PM-other (other pre-medication), H-Pulmicort/celestamine (Pulmicort/celestamine during hospitalization), and H-azithromycin (azithromycin during hospitalization) were found to be highly important. The support vector machine approach with a linear kernel was able to diffrentiate between predominantly allergic asthma and non-allergic asthma with higher accuracy (77.8%), precision (0.81), with a true positive rate of 0.73 and a true negative rate of 0.81, a F1 score of 0.81, and a ROC-AUC score of 0.79. Logistic regression was found to be the second-best classifier with an overall accuracy of 76.2%.

CONCLUSION

Predominantly allergic and non-allergic asthma can be classified using machine learning approaches based on nine features.

Supplemental data for this article is available online at at www.tandfonline.com/ijas .

Using a gradient boosted model for case ascertainment from free-text veterinary records

Case ascertainment for prevalence and incidence studies from veterinary clinical data poses a major challenge because medical notes are not consistently structured or complete. Using natural language processing (NLP) and machine learning, this study aimed to obtain accurate case recognition for feline upper respiratory tract infections (primarily caused by viruses such as feline herpes virus (FHV-1) and feline calici virus (FCV), and bacteria such as Chlamydophila felis, Mycoplasma felis and Bordetella bronchiseptica using retrospective electronic veterinary records from the Royal Society for Prevention of Cruelty to Animals, Queensland (RSPCA Qld). Data cleaning and NLP on eight years of free-text veterinary records from RSPCA Queensland was carried out to derive text-based predictors. The NLP steps included sorting records by length of stay, vectorising, tokenising and spell checking against a bespoke veterinary database. A gradient boosted model (GBM) was trained to predict the probability of each animal having a diagnosis of upper respiratory infection. A manually annotated dataset was used for training the algorithm to learn dominant patterns between predictors (frequencies of n-grams) and responses (manual binary case classification). The GBM's performance was tested against an out of sample validation dataset, and model agnostics were used to interrogate the model's learning process. The GBM used patient-level frequencies of 1250 unique n-grams as predictor variables and was able to predict the probability of cases in the validation dataset with an accuracy of 0.95 (95% CI 0.92, 0.97) and F1 score of 0.96. Predictors that exerted the highest influence on the model included frequencies of "doxycycline", "flu", "sneezing", "doxybrom" and "ocular". The trained GBM was deployed on the full dataset spanning eight years, comprising 60,258 clinical entries. The prevalence in the full dataset was predicted to be 23.59%, which is in line with domain expertise from practicing veterinarians at the shelter. Case ascertainment is a crucial step for further epidemiological study of cat flu. Ultimately, this tool can be extended to other clinical procedures, conditions, and diseases such as intensive care treatment due to snake bites and tick paralysis, physical injuries such as orthopaedic fractures or chest injuries and labour-intensive infectious diseases like parvovirus, canine cough, and ringworm, all of which require prolonged quarantine and care.

Biological gene extraction path based on knowledge graph and natural language processing

The continuous progress of society and the vigorous development of science and technology have brought people the dawn of maintaining health and preventing and controlling diseases. At the same time, with the update and iteration of bioinformatics technology, the current biological gene research has also undergone revolutionary changes. However, a long-standing problem in genetic research has always plagued researchers, that is, how to find the most needed sample genes from a large number of sample genes, so as to reduce unnecessary research and reduce research costs. By studying the extraction path of biological genes, it can help researchers to extract the most valuable research genes and avoid wasting time and energy. In order to solve the above problems, this paper used the Bhattacharyya distance index and the Gini index to screen the sample genes when extracting the characteristic genes of breast cancer. In the selected 49 public genes, 6 principal components were extracted by principal component analysis (PCA), and finally the experimental results were tested. It was found that when the optimal number of characteristic genes was selected as 5, the recognition rate of genes reached the highest 90.31%, which met the experimental requirements. In addition, the experiment also proved that the characteristic gene extraction method designed in this paper had a removal rate of 99.75% of redundant genes, which can greatly reduce the time and money cost of research.

Paired Indoor and Outdoor Nitrogen Dioxide Associated With Childhood Asthma Outcomes in a Mixed Rural-Urban Setting: A Feasibility Study

INTRODUCTION

Nitrogen dioxide (NO2) is known to be a trigger for asthma exacerbation. However, little is known about the role of seasonal variation in indoor and outdoor NO2 levels in childhood asthma in a mixed rural-urban setting of North America.

METHODS

This prospective cohort study, as a feasibility study, included 62 families with children (5-17 years) that had diagnosed persistent asthma residing in Olmsted County, Minnesota. Indoor and outdoor NO2 concentrations were measured using passive air samples over 2 weeks in winter and 2 weeks in summer. We assessed seasonal variation in NO2 levels in urban and rural residential areas and the association with asthma control status collected from participants' asthma diaries during the study period.

RESULTS

Outdoor NO2 levels were lower (median: 2.4 parts per billion (ppb) in summer, 3.9 ppb in winter) than the Environmental Protection Agency (EPA) annual standard (53 ppb). In winter, a higher level of outdoor NO2 was significantly associated with urban residential living area (P = .014) and lower socioeconomic status (SES) (P = .027). For both seasons, indoor NO2 was significantly higher (P < .05) in rural versus urban areas and in homes with gas versus electric stoves (P < .05). Asthma control status was not associated with level of indoor or outdoor NO2 in this cohort.

CONCLUSIONS

NO2 levels were low in this mixed rural-urban community and not associated with asthma control status in this small feasibility study. Further research with a larger sample size is warranted for defining a lower threshold of NO2 concentration with health effect on asthma in mixed rural-urban settings.

Multicenter Validation of Natural Language Processing Algorithms for the Detection of Common Data Elements in Operative Notes for Total Hip Arthroplasty: Algorithm Development and Validation

BACKGROUND

Natural language processing (NLP) methods are powerful tools for extracting and analyzing critical information from free-text data. MedTaggerIE, an open-source NLP pipeline for information extraction based on text patterns, has been widely used in the annotation of clinical notes. A rule-based system, MedTagger-total hip arthroplasty (THA), developed based on MedTaggerIE, was previously shown to correctly identify the surgical approach, fixation, and bearing surface from the THA operative notes at Mayo Clinic.

OBJECTIVE

This study aimed to assess the implementability, usability, and portability of MedTagger-THA at two external institutions, Michigan Medicine and the University of Iowa, and provide lessons learned for best practices.

METHODS

We conducted iterative test-apply-refinement processes with three involved sites-the development site (Mayo Clinic) and two deployment sites (Michigan Medicine and the University of Iowa). Mayo Clinic was the primary NLP development site, with the THA registry as the gold standard. The activities at the two deployment sites included the extraction of the operative notes, gold standard development (Michigan: registry data; Iowa: manual chart review), the refinement of NLP algorithms on training data, and the evaluation of test data. Error analyses were conducted to understand language variations across sites. To further assess the model specificity for approach and fixation, we applied the refined MedTagger-THA to arthroscopic hip procedures and periacetabular osteotomy cases, as neither of these operative notes should contain any approach or fixation keywords.

RESULTS

MedTagger-THA algorithms were implemented and refined independently for both sites. At Michigan, the study comprised THA-related notes for 2569 patient-date pairs. Before model refinement, MedTagger-THA algorithms demonstrated excellent accuracy for approach (96.6%, 95% CI 94.6%-97.9%) and fixation (95.7%, 95% CI 92.4%-97.6%). These results were comparable with internal accuracy at the development site (99.2% for approach and 90.7% for fixation). Model refinement improved accuracies slightly for both approach (99%, 95% CI 97.6%-99.6%) and fixation (98%, 95% CI 95.3%-99.3%). The specificity of approach identification was 88.9% for arthroscopy cases, and the specificity of fixation identification was 100% for both periacetabular osteotomy and arthroscopy cases. At the Iowa site, the study comprised an overall data set of 100 operative notes (50 training notes and 50 test notes). MedTagger-THA algorithms achieved moderate-high performance on the training data. After model refinement, the model achieved high performance for approach (100%, 95% CI 91.3%-100%), fixation (98%, 95% CI 88.3%-100%), and bearing surface (92%, 95% CI 80.5%-97.3%).

CONCLUSIONS

High performance across centers was achieved for the MedTagger-THA algorithms, demonstrating that they were sufficiently implementable, usable, and portable to different deployment sites. This study provided important lessons learned during the model deployment and validation processes, and it can serve as a reference for transferring rule-based electronic health record models.

Assessing socioeconomic bias in machine learning algorithms in health care: a case study of the HOUSES index

OBJECTIVE

Artificial intelligence (AI) models may propagate harmful biases in performance and hence negatively affect the underserved. We aimed to assess the degree to which data quality of electronic health records (EHRs) affected by inequities related to low socioeconomic status (SES), results in differential performance of AI models across SES.

MATERIALS AND METHODS

This study utilized existing machine learning models for predicting asthma exacerbation in children with asthma. We compared balanced error rate (BER) against different SES levels measured by HOUsing-based SocioEconomic Status measure (HOUSES) index. As a possible mechanism for differential performance, we also compared incompleteness of EHR information relevant to asthma care by SES.

RESULTS

Asthmatic children with lower SES had larger BER than those with higher SES (eg, ratio = 1.35 for HOUSES Q1 vs Q2-Q4) and had a higher proportion of missing information relevant to asthma care (eg, 41% vs 24% for missing asthma severity and 12% vs 9.8% for undiagnosed asthma despite meeting asthma criteria).

DISCUSSION

Our study suggests that lower SES is associated with worse predictive model performance. It also highlights the potential role of incomplete EHR data in this differential performance and suggests a way to mitigate this bias.

CONCLUSION

The HOUSES index allows AI researchers to assess bias in predictive model performance by SES. Although our case study was based on a small sample size and a single-site study, the study results highlight a potential strategy for identifying bias by using an innovative SES measure.

Comparison and validation of algorithms for asthma diagnosis in an electronic medical record system

BACKGROUND

Asthma is one of the most common chronic health conditions, and to leverage the wealth of data in the electronic medical record (EMR), it is important to be able to accurately identify asthma diagnosis.

OBJECTIVE

We aimed to determine the rule-based algorithm with the most balanced performance for sensitivity and positive predictive value of asthma diagnosis.

METHODS

We performed a diagnostic accuracy study of multiple rule-based algorithms intended to identify asthma diagnosis in the EMR. Algorithm performance was validated by manual chart review of 795 charts of patients seen in a multisite, tertiary-level, pulmonary specialty clinic using explicit diagnostic criteria to distinguish asthma cases from controls.

RESULTS

An asthma diagnosis anywhere in the medical record had a 97% sensitivity and a 77% specificity for asthma (F-score 80) when tested on a validation set of asthma cases and nonasthma respiratory disease from a pulmonary specialty clinic. The most balanced performance was seen with asthma diagnosis restricted to an encounter, hospital problem, or problem list diagnosis with a sensitivity of 94% and specificity of 85% (F-score 84). High sensitivity was achieved with the modified Health Plan Employer Data and Information Set criteria and high specificity was achieved with the NUgene algorithm, an algorithm developed for identifying asthma cases by EMR for genome-wide association studies.

CONCLUSION

Asthma diagnosis can be accurately identified for research purposes by restricting to encounter, hospital problem, or problem list diagnosis in a pulmonary specialty clinic. Additional rules lead to steep drop-offs in algorithm sensitivity in our population.

A Framework for Augmented Intelligence in Allergy and Immunology Practice and Research—A Work Group Report of the AAAAI Health Informatics, Technology, and Education Committee

Artificial and augmented intelligence (AI) and machine learning (ML) methods are expanding into the health care space. Big data are increasingly used in patient care applications, diagnostics, and treatment decisions in allergy and immunology. How these technologies will be evaluated, approved, and assessed for their impact is an important consideration for researchers and practitioners alike. With the potential of ML, deep learning, natural language processing, and other assistive methods to redefine health care usage, a scaffold for the impact of AI technology on research and patient care in allergy and immunology is needed. An American Academy of Asthma Allergy and Immunology Health Information Technology and Education subcommittee workgroup was convened to perform a scoping review of AI within health care as well as the specialty of allergy and immunology to address impacts on allergy and immunology practice and research as well as potential challenges including education, AI governance, ethical and equity considerations, and potential opportunities for the specialty. There are numerous potential clinical applications of AI in allergy and immunology that range from disease diagnosis to multidimensional data reduction in electronic health records or immunologic datasets. For appropriate application and interpretation of AI, specialists should be involved in the design, validation, and implementation of AI in allergy and immunology. Challenges include incorporation of data science and bioinformatics into training of future allergists-immunologists.

Artificial Intelligence Assesses Clinicians' Adherence to Asthma Guidelines Using Electronic Health Records

BACKGROUND

Clinicians' asthma guideline adherence in asthma care is suboptimal. The effort to improve adherence can be enhanced by assessing and monitoring clinicians' adherence to guidelines reflected in electronic health records (EHRs), which require costly manual chart review because many care elements cannot be identified by structured data.

OBJECTIVE

This study was designed to demonstrate the feasibility of an artificial intelligence tool using natural language processing (NLP) leveraging the free text EHRs of pediatric patients to extract key components of the 2007 National Asthma Education and Prevention Program guidelines.

METHODS

This is a retrospective cross-sectional study using a birth cohort with a diagnosis of asthma at Mayo Clinic between 2003 and 2016. We used 1,039 clinical notes with an asthma diagnosis from a random sample of 300 patients. Rule-based NLP algorithms were developed to identify asthma guideline-congruent elements by examining care description in EHR free text.

RESULTS

Natural language processing algorithms demonstrated a sensitivity (0.82-1.0), specificity (0.95-1.0), positive predictive value (0.86-1.0), and negative predictive value (0.92-1.0) against manual chart review for asthma guideline-congruent elements. Assessing medication compliance and inhaler technique assessment were the most challenging elements to assess because of the complexity and wide variety of descriptions.

CONCLUSIONS

Natural language processing technologies may enable the automated assessment of clinicians' documentation in EHRs regarding adherence to asthma guidelines and can be a useful population management and research tool to assess and monitor asthma care quality. Multisite studies with a larger sample size are needed to assess the generalizability of these NLP algorithms.

Risk of pneumonia in asthmatic children using inhaled corticosteroids: a nested case-control study in a birth cohort

BACKGROUND

Inhaled corticosteroids (ICSs) are important in asthma management, but there are concerns regarding associated risk of pneumonia. While studies in asthmatic adults have shown inconsistent results, this risk in asthmatic children is unclear.

OBJECTIVE

Our aim was to determine the association of ICS use with pneumonia risk in asthmatic children.

METHODS

A nested case-control study was performed in the Mayo Clinic Birth Cohort. Asthmatic children (<18 years) with a physician diagnosis of asthma were identified from electronic medical records of children born at Mayo Clinic from 1997 to 2016 and followed until 31 December 2017. Pneumonia cases defined by Infectious Disease Society of America were 1:1 matched with controls without pneumonia by age, sex and asthma index date. Exposure was defined as ICS prescription at least 90 days prior to pneumonia. Associations of ICS use, type and dose (low, medium and high) with pneumonia risk were analysed using conditional logistic regression.

RESULTS

Of the 2108 asthmatic children eligible for the study (70% mild intermittent and 30% persistent asthma), 312 children developed pneumonia during the study period. ICS use overall was not associated with risk of pneumonia (adjusted OR: 0.94, 95% CI: 0.62 to 1.41). Poorly controlled asthma was significantly associated with the risk of pneumonia (OR: 2.03, 95% CI: 1.35 to 3.05; p<0.001). No ICS type or dose was associated with risk of pneumonia.

CONCLUSION

ICS use in asthmatic children was not associated with risk of pneumonia but poorly controlled asthma was. Future asthma studies may need to include pneumonia as a potential outcome of asthma management.

Clinical Outcomes for Young Children Diagnosed With Asthma Versus Reactive Airway Disease

OBJECTIVE

Clinical diagnoses of asthma and reactive airway disease (RAD) in young children are subjective. We examined how often children were diagnosed with asthma versus RAD, and whether preventive care and 2-year clinical outcomes differed based on initial diagnosis.

METHODS

We conducted a retrospective cohort analysis of children (2-7 years) from a university-based general pediatrics practice who had been diagnosed with RAD or asthma. We performed adjusted comparisons between groups for time until subsequent asthma-related care. We also compared delivery of asthma-related healthcare services, corticosteroid and controller prescriptions, and action plans within 2 years of index diagnosis, using bivariate and regression analyses.

RESULTS

Four hundred three children were included (64% male, 67% Black, 25% Hispanic). RAD was diagnosed in 62% of index visits, and was more likely than asthma to be diagnosed in emergency settings. In the full sample, the time between index visit and subsequent asthma care did not differ between groups, after adjustment for index location. For subjects with complete 24-month follow-up (N = 300), no between-group differences were found in adjusted analyses. Most children with RAD received action plans and controller medications only after a subsequent asthma diagnosis, on average, 9 months after their index visit.

CONCLUSIONS

RAD diagnoses were linked to delayed delivery of preventive care measures, but within 2 years of initial diagnosis, clinical outcomes for those diagnosed with RAD and asthma did not differ. To facilitate clear communication and timely treatment, a prompt diagnosis of asthma, rather than RAD, should be considered for children with asthma symptoms.

Development and Validation of a Natural Language Processing Algorithm to Extract Descriptors of Microbial Keratitis From the Electronic Health Record

PURPOSE

The purpose of this article was to develop and validate a natural language processing (NLP) algorithm to extract qualitative descriptors of microbial keratitis (MK) from electronic health records.

METHODS

In this retrospective cohort study, patients with MK diagnoses from 2 academic centers were identified using electronic health records. An NLP algorithm was created to extract MK centrality, depth, and thinning. A random sample of patient with MK encounters were used to train the algorithm (400 encounters of 100 patients) and compared with expert chart review. The algorithm was evaluated in internal (n = 100) and external validation data sets (n = 59) in comparison with masked chart review. Outcomes were sensitivity and specificity of the NLP algorithm to extract qualitative MK features as compared with masked chart review performed by an ophthalmologist.

RESULTS

Across data sets, gold-standard chart review found centrality was documented in 64.0% to 79.3% of charts, depth in 15.0% to 20.3%, and thinning in 25.4% to 31.3%. Compared with chart review, the NLP algorithm had a sensitivity of 80.3%, 50.0%, and 66.7% for identifying central MK, 85.4%, 66.7%, and 100% for deep MK, and 100.0%, 95.2%, and 100% for thin MK, in the training, internal, and external validation samples, respectively. Specificity was 41.1%, 38.6%, and 46.2% for centrality, 100%, 83.3%, and 71.4% for depth, and 93.3%, 100%, and was not applicable (n = 0) to the external data for thinning, in the samples, respectively.

CONCLUSIONS

MK features are not documented consistently showing a lack of standardization in recording MK examination elements. NLP shows promise but will be limited if the available clinical data are missing from the chart.

dbAMP 2.0: updated resource for antimicrobial peptides with an enhanced scanning method for genomic and proteomic data

The last 18 months, or more, have seen a profound shift in our global experience, with many of us navigating a once-in-100-year pandemic. To date, COVID-19 remains a life-threatening pandemic with little to no targeted therapeutic recourse. The discovery of novel antiviral agents, such as vaccines and drugs, can provide therapeutic solutions to save human beings from severe infections; however, there is no specifically effective antiviral treatment confirmed for now. Thus, great attention has been paid to the use of natural or artificial antimicrobial peptides (AMPs) as these compounds are widely regarded as promising solutions for the treatment of harmful microorganisms. Given the biological significance of AMPs, it was obvious that there was a significant need for a single platform for identifying and engaging with AMP data. This led to the creation of the dbAMP platform that provides comprehensive information about AMPs and facilitates their investigation and analysis. To date, the dbAMP has accumulated 26 447 AMPs and 2262 antimicrobial proteins from 3044 organisms using both database integration and manual curation of >4579 articles. In addition, dbAMP facilitates the evaluation of AMP structures using I-TASSER for automated protein structure prediction and structure-based functional annotation, providing predictive structure information for clinical drug development. Next-generation sequencing (NGS) and third-generation sequencing have been applied to generate large-scale sequencing reads from various environments, enabling greatly improved analysis of genome structure. In this update, we launch an efficient online tool that can effectively identify AMPs from genome/metagenome and proteome data of all species in a short period. In conclusion, these improvements promote the dbAMP as one of the most abundant and comprehensively annotated resources for AMPs. The updated dbAMP is now freely accessible at http://awi.cuhk.edu.cn/dbAMP.

Identification of asthma control factor in clinical notes using a hybrid deep learning model

BACKGROUND

There are significant variabilities in guideline-concordant documentation in asthma care. However, assessing clinician's documentation is not feasible using only structured data but requires labor-intensive chart review of electronic health records (EHRs). A certain guideline element in asthma control factors, such as review inhaler techniques, requires context understanding to correctly capture from EHR free text.

METHODS

The study data consist of two sets: (1) manual chart reviewed data-1039 clinical notes of 300 patients with asthma diagnosis, and (2) weakly labeled data (distant supervision)-27,363 clinical notes from 800 patients with asthma diagnosis. A context-aware language model, Bidirectional Encoder Representations from Transformers (BERT) was developed to identify inhaler techniques in EHR free text. Both original BERT and clinical BioBERT (cBERT) were applied with a cost-sensitivity to deal with imbalanced data. The distant supervision using weak labels by rules was also incorporated to augment the training set and alleviate a costly manual labeling process in the development of a deep learning algorithm. A hybrid approach using post-hoc rules was also explored to fix BERT model errors. The performance of BERT with/without distant supervision, hybrid, and rule-based models were compared in precision, recall, F-score, and accuracy.

RESULTS

The BERT models on the original data performed similar to a rule-based model in F1-score (0.837, 0.845, and 0.838 for rules, BERT, and cBERT, respectively). The BERT models with distant supervision produced higher performance (0.853 and 0.880 for BERT and cBERT, respectively) than without distant supervision and a rule-based model. The hybrid models performed best in F1-score of 0.877 and 0.904 over the distant supervision on BERT and cBERT.

CONCLUSIONS

The proposed BERT models with distant supervision demonstrated its capability to identify inhaler techniques in EHR free text, and outperformed both the rule-based model and BERT models trained on the original data. With a distant supervision approach, we may alleviate costly manual chart review to generate the large training data required in most deep learning-based models. A hybrid model was able to fix BERT model errors and further improve the performance.

Establishing an expert consensus for the operational definitions of asthma-associated infectious and inflammatory multimorbidities for computational algorithms through a modified Delphi technique

BACKGROUND

A subgroup of patients with asthma has been reported to have an increased risk for asthma-associated infectious and inflammatory multimorbidities (AIMs). To systematically investigate the association of asthma with AIMs using a large patient cohort, it is desired to leverage a broad range of electronic health record (EHR) data sources to automatically identify AIMs accurately and efficiently.

METHODS

We established an expert consensus for an operational definition for each AIM from EHR through a modified Delphi technique. A series of questions about the operational definition of 19 AIMS (11 infectious diseases and 8 inflammatory diseases) was generated by a core team of experts who considered feasibility, balance between sensitivity and specificity, and generalizability. Eight internal and 5 external expert panelists were invited to individually complete a series of online questionnaires and provide judgement and feedback throughout three sequential internal rounds and two external rounds. Panelists' responses were collected, descriptive statistics tabulated, and results reported back to the entire group. Following each round the core team of experts made iterative edits to the operational definitions until a moderate (≥ 60%) or strong (≥ 80%) level of consensus among the panel was achieved.

RESULTS

Response rates for each Delphi round were 100% in all 5 rounds with the achievement of the following consensus levels: (1) Internal panel consensus: 100% for 8 definitions, 88% for 10 definitions, and 75% for 1 definition, (2) External panel consensus: 100% for 12 definitions and 80% for 7 definitions.

CONCLUSIONS

The final operational definitions of AIMs established through a modified Delphi technique can serve as a foundation for developing computational algorithms to automatically identify AIMs from EHRs to enable large scale research studies on patient's multimorbidities associated with asthma.

Natural Language Processing and Machine Learning Methods to Characterize Unstructured Patient-Reported Outcomes: Validation Study

BACKGROUND

Assessing patient-reported outcomes (PROs) through interviews or conversations during clinical encounters provides insightful information about survivorship.

OBJECTIVE

This study aims to test the validity of natural language processing (NLP) and machine learning (ML) algorithms in identifying different attributes of pain interference and fatigue symptoms experienced by child and adolescent survivors of cancer versus the judgment by PRO content experts as the gold standard to validate NLP/ML algorithms.

METHODS

This cross-sectional study focused on child and adolescent survivors of cancer, aged 8 to 17 years, and caregivers, from whom 391 meaning units in the pain interference domain and 423 in the fatigue domain were generated for analyses. Data were collected from the After Completion of Therapy Clinic at St. Jude Children's Research Hospital. Experienced pain interference and fatigue symptoms were reported through in-depth interviews. After verbatim transcription, analyzable sentences (ie, meaning units) were semantically labeled by 2 content experts for each attribute (physical, cognitive, social, or unclassified). Two NLP/ML methods were used to extract and validate the semantic features: bidirectional encoder representations from transformers (BERT) and Word2vec plus one of the ML methods, the support vector machine or extreme gradient boosting. Receiver operating characteristic and precision-recall curves were used to evaluate the accuracy and validity of the NLP/ML methods.

RESULTS

Compared with Word2vec/support vector machine and Word2vec/extreme gradient boosting, BERT demonstrated higher accuracy in both symptom domains, with 0.931 (95% CI 0.905-0.957) and 0.916 (95% CI 0.887-0.941) for problems with cognitive and social attributes on pain interference, respectively, and 0.929 (95% CI 0.903-0.953) and 0.917 (95% CI 0.891-0.943) for problems with cognitive and social attributes on fatigue, respectively. In addition, BERT yielded superior areas under the receiver operating characteristic curve for cognitive attributes on pain interference and fatigue domains (0.923, 95% CI 0.879-0.997; 0.948, 95% CI 0.922-0.979) and superior areas under the precision-recall curve for cognitive attributes on pain interference and fatigue domains (0.818, 95% CI 0.735-0.917; 0.855, 95% CI 0.791-0.930).

CONCLUSIONS

The BERT method performed better than the other methods. As an alternative to using standard PRO surveys, collecting unstructured PROs via interviews or conversations during clinical encounters and applying NLP/ML methods can facilitate PRO assessment in child and adolescent cancer survivors.

Stroke Outcome Measurements From Electronic Medical Records: Cross-sectional Study on the Effectiveness of Neural and Nonneural Classifiers

BACKGROUND

With the rapid adoption of electronic medical records (EMRs), there is an ever-increasing opportunity to collect data and extract knowledge from EMRs to support patient-centered stroke management.

OBJECTIVE

This study aims to compare the effectiveness of state-of-the-art automatic text classification methods in classifying data to support the prediction of clinical patient outcomes and the extraction of patient characteristics from EMRs.

METHODS

Our study addressed the computational problems of information extraction and automatic text classification. We identified essential tasks to be considered in an ischemic stroke value-based program. The 30 selected tasks were classified (manually labeled by specialists) according to the following value agenda: tier 1 (achieved health care status), tier 2 (recovery process), care related (clinical management and risk scores), and baseline characteristics. The analyzed data set was retrospectively extracted from the EMRs of patients with stroke from a private Brazilian hospital between 2018 and 2019. A total of 44,206 sentences from free-text medical records in Portuguese were used to train and develop 10 supervised computational machine learning methods, including state-of-the-art neural and nonneural methods, along with ontological rules. As an experimental protocol, we used a 5-fold cross-validation procedure repeated 6 times, along with subject-wise sampling. A heatmap was used to display comparative result analyses according to the best algorithmic effectiveness (F1 score), supported by statistical significance tests. A feature importance analysis was conducted to provide insights into the results.

RESULTS

The top-performing models were support vector machines trained with lexical and semantic textual features, showing the importance of dealing with noise in EMR textual representations. The support vector machine models produced statistically superior results in 71% (17/24) of tasks, with an F1 score >80% regarding care-related tasks (patient treatment location, fall risk, thrombolytic therapy, and pressure ulcer risk), the process of recovery (ability to feed orally or ambulate and communicate), health care status achieved (mortality), and baseline characteristics (diabetes, obesity, dyslipidemia, and smoking status). Neural methods were largely outperformed by more traditional nonneural methods, given the characteristics of the data set. Ontological rules were also effective in tasks such as baseline characteristics (alcoholism, atrial fibrillation, and coronary artery disease) and the Rankin scale. The complementarity in effectiveness among models suggests that a combination of models could enhance the results and cover more tasks in the future.

CONCLUSIONS

Advances in information technology capacity are essential for scalability and agility in measuring health status outcomes. This study allowed us to measure effectiveness and identify opportunities for automating the classification of outcomes of specific tasks related to clinical conditions of stroke victims, and thus ultimately assess the possibility of proactively using these machine learning techniques in real-world situations.

Epidemiology of adult chronic cough: disease burden, regional issues, and recent findings

Chronic cough is a common medical condition that has a significant impact on patients' quality of life. Although it was previously considered a symptom of other disorders, it is now regarded as a pathologic state that is characterized by a deviation from the intrinsic protective functions of the cough reflex, especially in adults. There are several factors that may underlie the cough reflex hypersensitivity and its persistence, such as age, sex, comorbidities, viral infection, exposure to irritants or environmental pollutants, and their interactions may determine the epidemiology of chronic cough in different countries. With a deeper understanding of disease pathophysiology and advanced research methodology, there are more attempts to investigate cough epidemiology using a large cohort of healthcare population data. This is a narrative overview of recent findings on the disease burden, risk factors, Asia-Pacific issues, and longitudinal outcomes in adults with chronic cough. This paper also discusses the approaches utilizing routinely collected data in cough research.

Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial

RATIONALE

Clinical decision support (CDS) tools leveraging electronic health records (EHRs) have been an approach for addressing challenges in asthma care but remain under-studied through clinical trials.

OBJECTIVES

To assess the effectiveness and efficiency of Asthma-Guidance and Prediction System (A-GPS), an Artificial Intelligence (AI)-assisted CDS tool, in optimizing asthma management through a randomized clinical trial (RCT).

METHODS

This was a single-center pragmatic RCT with a stratified randomization design conducted for one year in the primary care pediatric practice of the Mayo Clinic, MN. Children (<18 years) diagnosed with asthma receiving care at the study site were enrolled along with their 42 primary care providers. Study subjects were stratified into three strata (based on asthma severity, asthma care status, and asthma diagnosis) and were blinded to the assigned groups.

MEASUREMENTS

Intervention was a quarterly A-GPS report to clinicians including relevant clinical information for asthma management from EHRs and machine learning-based prediction for risk of asthma exacerbation (AE). Primary endpoint was the occurrence of AE within 1 year and secondary outcomes included time required for clinicians to review EHRs for asthma management.

MAIN RESULTS

Out of 555 participants invited to the study, 184 consented for the study and were randomized (90 in intervention and 94 in control group). Median age of 184 participants was 8.5 years. While the proportion of children with AE in both groups decreased from the baseline (P = 0.042), there was no difference in AE frequency between the two groups (12% for the intervention group vs. 15% for the control group, Odds Ratio: 0.82; 95%CI 0.374-1.96; P = 0.626) during the study period. For the secondary end points, A-GPS intervention, however, significantly reduced time for reviewing EHRs for asthma management of each participant (median: 3.5 min, IQR: 2-5), compared to usual care without A-GPS (median: 11.3 min, IQR: 6.3-15); p<0.001). Mean health care costs with 95%CI of children during the trial (compared to before the trial) in the intervention group were lower than those in the control group (-$1,036 [-$2177, $44] for the intervention group vs. +$80 [-$841, $1000] for the control group), though there was no significant difference (p = 0.12). Among those who experienced the first AE during the study period (n = 25), those in the intervention group had timelier follow up by the clinical care team compared to those in the control group but no significant difference was found (HR = 1.93; 95% CI: 0.82-1.45, P = 0.10). There was no difference in the proportion of duration when patients had well-controlled asthma during the study period between the intervention and the control groups.

CONCLUSIONS

While A-GPS-based intervention showed similar reduction in AE events to usual care, it might reduce clinicians' burden for EHRs review resulting in efficient asthma management. A larger RCT is needed for further studying the findings.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02865967.

Artificial intelligence in the diagnosis of pediatric allergic diseases

Artificial intelligence (AI) is a field of data science pertaining to advanced computing machines capable of learning from data and interacting with the human world. Early diagnosis and diagnostics, self-care, prevention and wellness, clinical decision support, care delivery, and chronic care management have been identified within the healthcare areas that could benefit from introducing AI. In pediatric allergy research, the recent developments in AI approach provided new perspectives for characterizing the heterogeneity of allergic diseases among patients. Moreover, the increasing use of electronic health records and personal healthcare records highlighted the relevance of AI in improving data quality and processing and setting-up advanced algorithms to interpret the data. This review aimed to summarize current knowledge about AI and discuss its impact on the diagnostic framework of pediatric allergic diseases such as eczema, food allergy, and respiratory allergy, along with the future opportunities that AI research can offer in this medical area.

Cohort profile: St. Michael's Hospital Tuberculosis Database (SMH-TB), a retrospective cohort of electronic health record data and variables extracted using natural language processing

Background

Tuberculosis (TB) is a major cause of death worldwide. TB research draws heavily on clinical cohorts which can be generated using electronic health records (EHR), but granular information extracted from unstructured EHR data is limited. The St. Michael’s Hospital TB database (SMH-TB) was established to address gaps in EHR-derived TB clinical cohorts and provide researchers and clinicians with detailed, granular data related to TB management and treatment.

Methods

We collected and validated multiple layers of EHR data from the TB outpatient clinic at St. Michael’s Hospital, Toronto, Ontario, Canada to generate the SMH-TB database. SMH-TB contains structured data directly from the EHR, and variables generated using natural language processing (NLP) by extracting relevant information from free-text within clinic, radiology, and other notes. NLP performance was assessed using recall, precision and F₁ score averaged across variable labels. We present characteristics of the cohort population using binomial proportions and 95% confidence intervals (CI), with and without adjusting for NLP misclassification errors.

Results

SMH-TB currently contains retrospective patient data spanning 2011 to 2018, for a total of 3298 patients (N = 3237 with at least 1 associated dictation). Performance of TB diagnosis and medication NLP rulesets surpasses 93% in recall, precision and F₁ metrics, indicating good generalizability. We estimated 20% (95% CI: 18.4–21.2%) were diagnosed with active TB and 46% (95% CI: 43.8–47.2%) were diagnosed with latent TB. After adjusting for potential misclassification, the proportion of patients diagnosed with active and latent TB was 18% (95% CI: 16.8–19.7%) and 40% (95% CI: 37.8–41.6%) respectively

Conclusion

SMH-TB is a unique database that includes a breadth of structured data derived from structured and unstructured EHR data by using NLP rulesets. The data are available for a variety of research applications, such as clinical epidemiology, quality improvement and mathematical modeling studies.

Electronic Medical Record-Based Case Phenotyping for the Charlson Conditions: Scoping Review

BACKGROUND

Electronic medical records (EMRs) contain large amounts of rich clinical information. Developing EMR-based case definitions, also known as EMR phenotyping, is an active area of research that has implications for epidemiology, clinical care, and health services research.

OBJECTIVE

This review aims to describe and assess the present landscape of EMR-based case phenotyping for the Charlson conditions.

METHODS

A scoping review of EMR-based algorithms for defining the Charlson comorbidity index conditions was completed. This study covered articles published between January 2000 and April 2020, both inclusive. Embase (Excerpta Medica database) and MEDLINE (Medical Literature Analysis and Retrieval System Online) were searched using keywords developed in the following 3 domains: terms related to EMR, terms related to case finding, and disease-specific terms. The manuscript follows the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA) guidelines.

RESULTS

A total of 274 articles representing 299 algorithms were assessed and summarized. Most studies were undertaken in the United States (181/299, 60.5%), followed by the United Kingdom (42/299, 14.0%) and Canada (15/299, 5.0%). These algorithms were mostly developed either in primary care (103/299, 34.4%) or inpatient (168/299, 56.2%) settings. Diabetes, congestive heart failure, myocardial infarction, and rheumatology had the highest number of developed algorithms. Data-driven and clinical rule-based approaches have been identified. EMR-based phenotype and algorithm development reflect the data access allowed by respective health systems, and algorithms vary in their performance.

CONCLUSIONS

Recognizing similarities and differences in health systems, data collection strategies, extraction, data release protocols, and existing clinical pathways is critical to algorithm development strategies. Several strategies to assist with phenotype-based case definitions have been proposed.

Deep Learning Identification of Asthma Inhaler Techniques in Clinical Notes

There are significant variabilities in clinicians' guideline-concordant documentation in asthma care. However, assessing clinicians' documentation is not feasible using only structured data but requires labor intensive chart review of electronic health records. Although the national asthma guidelines are available it is still challenging to use them as a real-time tool for providing feedback on adhering documentation guidelines for asthma care improvement. A certain guideline element, such as teaching or reviewing inhaler techniques, is difficult to capture by handcrafted rules since it requires contextual understanding of clinical narratives. This study examined a deep learning based natural language model, Bidirectional Encoder Representations from Transformers (BERT) coupled with distant supervision to identify inhaler techniques from clinical narratives. The BERT model with distant supervision outperformed the rule-based approach and achieved performance gain compared with the BERT without distant supervision.

Artificial Intelligence and Machine Learning in Chronic Airway Diseases: Focus on Asthma and Chronic Obstructive Pulmonary Disease

Chronic airway diseases are characterized by airway inflammation, obstruction, and remodeling and show high prevalence, especially in developing countries. Among them, asthma and chronic obstructive pulmonary disease (COPD) show the highest morbidity and socioeconomic burden worldwide. Although there are extensive guidelines for the prevention, early diagnosis, and rational treatment of these lifelong diseases, their value in precision medicine is very limited. Artificial intelligence (AI) and machine learning (ML) techniques have emerged as effective methods for mining and integrating large-scale, heterogeneous medical data for clinical practice, and several AI and ML methods have recently been applied to asthma and COPD. However, very few methods have significantly contributed to clinical practice. Here, we review four aspects of AI and ML implementation in asthma and COPD to summarize existing knowledge and indicate future steps required for the safe and effective application of AI and ML tools by clinicians.

Development of an Automated Algorithm to Generate Guideline-based Recommendations for Follow-up Colonoscopy

BACKGROUND AND AIMS

Physician adherence to published colonoscopy surveillance guidelines varies. We aimed to develop and validate an automated clinical decision support algorithm that can extract procedure and pathology data from the electronic medical record (EMR) and generate surveillance intervals congruent with guidelines, which might increase physician adherence.

METHODS

We constructed a clinical decision support (CDS) algorithm based on guidelines from the United States Multi-Society Task Force on Colorectal Cancer. We used a randomly generated validation dataset of 300 outpatient colonoscopies performed at the Cleveland Clinic from 2012 through 2016 to evaluate the accuracy of extracting data from reports stored in the EMR using natural language processing (NLP). We compared colonoscopy follow-up recommendations from the CDS algorithm, endoscopists, and task force guidelines. Using a testing dataset of 2439 colonoscopies, we compared endoscopist recommendations with those of the algorithm.

RESULTS

Manual review of the validation dataset confirmed the NLP program accurately extracted procedure and pathology data for all cases. Recommendations made by endoscopists and the CDS algorithm were guideline-concordant in 62% and 99% of cases, respectively. Discrepant recommendations by endoscopists were earlier than recommended in 94% of the cases. In the testing dataset, 69% of endoscopist and NLP-CDS algorithm recommendations were concordant. Discrepant recommendations by endoscopists were earlier than guidelines in 91% of cases.

CONCLUSIONS

We constructed and tested an automated CDS algorithm that can use NLP-extracted data from the EMR to generate follow-up colonoscopy surveillance recommendations based on published guidelines.

Registries and collaborative studies for primary ciliary dyskinesia in Europe

Primary ciliary dyskinesia (PCD) is a rare inherited disease characterised by malfunctioning cilia leading to a heterogeneous clinical phenotype with many organ systems affected. There is a lack of data on clinical presentation, prognosis and effectiveness of treatments, making it mandatory to improve the scientific evidence base.

This article reviews the data resources that are available in Europe for clinical and epidemiological research in PCD, namely established national PCD registries and national cohort studies, plus two large collaborative efforts (the international PCD (iPCD) Cohort and the International PCD Registry), and discusses their strengths, limitations and perspectives.

Denmark, Cyprus, Norway and Switzerland have national population-based registries, while England and France conduct multicentre cohort studies. Based on the data contained in these registries, the prevalence of diagnosed PCD is 3–7 per 100 000 in children and 0.2–6 per 100 000 in adults. All registries, together with other studies from Europe and beyond, contribute to the iPCD Cohort, a collaborative study including data from over 4000 PCD patients, and to the International PCD Registry, which is part of the ERN (European Reference Network)-LUNG network.

This rich resource of readily available, standardised and contemporaneous data will allow obtaining fast answers to emerging clinical and research questions in PCD.

The growing collaborative network of national and international registries and cohort studies of patients with PCD provides an excellent resource for research on this rare diseasehttps://bit.ly/3dto75l

A diagnostic codes-based algorithm improves accuracy for identification of childhood asthma in archival data sets

OBJECTIVE

While a single but truncated ICD code (493) had been widely used for identifying asthma in asthma care and research, it significantly under-identifies asthma. We aimed to develop and validate a diagnostic codes-based algorithm for identifying asthmatics using Predetermined Asthma Criteria (PAC) as the reference.

METHODS

This is a retrospective cross-sectional study which utilized two different coding systems, the Hospital Adaptation of the International Classification of Diseases, Eighth Revision (H-ICDA) and the International Classification of Diseases, Ninth Revision (ICD-9). The algorithm was developed using two population-based asthma study cohorts, and validated in a validation cohort, a random sample of the 1976-2007 Olmsted County Birth Cohort. Performance of the diagnostic codes-based algorithm for ascertaining asthma status against manual chart review for PAC (gold standard) was assessed by determining both criterion and construct validity.

RESULTS

Among eligible 267 subjects of the validation cohort, 50% were male, 70% white, and the median age at last follow-up was 17 (interquartile range, 8.7-24.4) years. Asthma prevalence by PAC through manual chart review was 34%. Sensitivity and specificity of the codes-based algorithm for identifying asthma were 82% and 98% respectively. Associations of asthma-related risk factors with asthma status ascertained by the code-based algorithm were similar to those by the manual review.

CONCLUSIONS

The diagnostic codes-based algorithm for identifying asthmatics improves accuracy of identification of asthma and can be a useful tool for large scale studies in a setting without automated chart review capabilities.

dbAMP: an integrated resource for exploring antimicrobial peptides with functional activities and physicochemical properties on transcriptome and proteome data

Antimicrobial peptides (AMPs), naturally encoded from genes and generally contained 10–100 amino acids, are crucial components of the innate immune system and can protect the host from various pathogenic bacteria, as well as viruses. In recent years, the widespread use of antibiotics has inspired the rapid growth of antibiotic-resistant microorganisms that usually induce critical infection and pathogenesis. An increasing interest therefore was motivated to explore natural AMPs that enable the development of new antibiotics. With the potential of AMPs being as new drugs for multidrug-resistant pathogens, we were thus motivated to develop a database (dbAMP, http://csb.cse.yzu.edu.tw/dbAMP/) by accumulating comprehensive AMPs from public domain and manually curating literature. Currently in dbAMP there are 12 389 unique entries, including 4271 experimentally verified AMPs and 8118 putative AMPs along with their functional activities, supported by 1924 research articles. The advent of high-throughput biotechnologies, such as mass spectrometry and next-generation sequencing, has led us to further expand dbAMP as a database-assisted platform for providing comprehensively functional and physicochemical analyses for AMPs based on the large-scale transcriptome and proteome data. Significant improvements available in dbAMP include the information of AMP–protein interactions, antimicrobial potency analysis for ‘cryptic’ region detection, annotations of AMP target species, as well as AMP detection on transcriptome and proteome datasets. Additionally, a Docker container has been developed as a downloadable package for discovering known and novel AMPs on high-throughput omics data. The user-friendly visualization interfaces have been created to facilitate peptide searching, browsing, and sequence alignment against dbAMP entries. All the facilities integrated into dbAMP can promote the functional analyses of AMPs and the discovery of new antimicrobial drugs.

dbPTM in 2019: exploring disease association and cross-talk of post-translational modifications

The dbPTM (http://dbPTM.mbc.nctu.edu.tw/) has been maintained for over 10 years with the aim to provide functional and structural analyses for post-translational modifications (PTMs). In this update, dbPTM not only integrates more experimentally validated PTMs from available databases and through manual curation of literature but also provides PTM-disease associations based on non-synonymous single nucleotide polymorphisms (nsSNPs). The high-throughput deep sequencing technology has led to a surge in the data generated through analysis of association between SNPs and diseases, both in terms of growth amount and scope. This update thus integrated disease-associated nsSNPs from dbSNP based on genome-wide association studies. The PTM substrate sites located at a specified distance in terms of the amino acids encoded from nsSNPs were deemed to have an association with the involved diseases. In recent years, increasing evidence for crosstalk between PTMs has been reported. Although mass spectrometry-based proteomics has substantially improved our knowledge about substrate site specificity of single PTMs, the fact that the crosstalk of combinatorial PTMs may act in concert with the regulation of protein function and activity is neglected. Because of the relatively limited information about concurrent frequency and functional relevance of PTM crosstalk, in this update, the PTM sites neighboring other PTM sites in a specified window length were subjected to motif discovery and functional enrichment analysis. This update highlights the current challenges in PTM crosstalk investigation and breaks the bottleneck of how proteomics may contribute to understanding PTM codes, revealing the next level of data complexity and proteomic limitation in prospective PTM research.

Predicting Inpatient Falls Using Natural Language Processing of Nursing Records Obtained From Japanese Electronic Medical Records: Case-Control Study

BACKGROUND

Falls in hospitals are the most common risk factor that affects the safety of inpatients and can result in severe harm. Therefore, preventing falls is one of the most important areas of risk management for health care organizations. However, existing methods for predicting falls are laborious and costly.

OBJECTIVE

The objective of this study is to verify whether hospital inpatient falls can be predicted through the analysis of a single input-unstructured nursing records obtained from Japanese electronic medical records (EMRs)-using a natural language processing (NLP) algorithm and machine learning.

METHODS

The nursing records of 335 fallers and 408 nonfallers for a 12-month period were extracted from the EMRs of an acute care hospital and randomly divided into a learning data set and test data set. The former data set was subjected to NLP and machine learning to extract morphemes that contributed to separating fallers from nonfallers to construct a model for predicting falls. Then, the latter data set was used to determine the predictive value of the model using receiver operating characteristic (ROC) analysis.

RESULTS

The prediction of falls using the test data set showed high accuracy, with an area under the ROC curve, sensitivity, specificity, and odds ratio of mean 0.834 (SD 0.005), mean 0.769 (SD 0.013), mean 0.785 (SD 0.020), and mean 12.27 (SD 1.11) for five independent experiments, respectively. The morphemes incorporated into the final model included many words closely related to known risk factors for falls, such as the use of psychotropic drugs, state of consciousness, and mobility, thereby demonstrating that an NLP algorithm combined with machine learning can effectively extract risk factors for falls from nursing records.

CONCLUSIONS

We successfully established that falls among hospital inpatients can be predicted by analyzing nursing records using an NLP algorithm and machine learning. Therefore, it may be possible to develop a fall risk monitoring system that analyzes nursing records daily and alerts health care professionals when the fall risk of an inpatient is increased.

Expert artificial intelligence-based natural language processing characterises childhood asthma

INTRODUCTION

The lack of effective, consistent, reproducible and efficient asthma ascertainment methods results in inconsistent asthma cohorts and study results for clinical trials or other studies. We aimed to assess whether application of expert artificial intelligence (AI)-based natural language processing (NLP) algorithms for two existing asthma criteria to electronic health records of a paediatric population systematically identifies childhood asthma and its subgroups with distinctive characteristics.

METHODS

Using the 1997-2007 Olmsted County Birth Cohort, we applied validated NLP algorithms for Predetermined Asthma Criteria (NLP-PAC) as well as Asthma Predictive Index (NLP-API). We categorised subjects into four groups (both criteria positive (NLP-PAC⁺/NLP-API⁺); PAC positive only (NLP-PAC⁺ only); API positive only (NLP-API⁺ only); and both criteria negative (NLP-PAC^-/NLP-API^-)) and characterised them. Results were replicated in unsupervised cluster analysis for asthmatics and a random sample of 300 children using laboratory and pulmonary function tests (PFTs).

RESULTS

Of the 8196 subjects (51% male, 80% white), we identified 1614 (20%), NLP-PAC⁺/NLP-API⁺; 954 (12%), NLP-PAC⁺ only; 105 (1%), NLP-API⁺ only; and 5523 (67%), NLP-PAC^-/NLP-API^-. Asthmatic children classified as NLP-PAC⁺/NLP-API⁺ showed earlier onset asthma, more Th2-high profile, poorer lung function, higher asthma exacerbation and higher risk of asthma-associated comorbidities compared with other groups. These results were consistent with those based on unsupervised cluster analysis and lab and PFT data of a random sample of study subjects.

CONCLUSION

Expert AI-based NLP algorithms for two asthma criteria systematically identify childhood asthma with distinctive characteristics. This approach may improve precision, reproducibility, consistency and efficiency of large-scale clinical studies for asthma and enable population management.

Artificial intelligence approaches using natural language processing to advance EHR-based clinical research

The wide adoption of electronic health record systems in health care generates big real-world data that open new venues to conduct clinical research. As a large amount of valuable clinical information is locked in clinical narratives, natural language processing techniques as an artificial intelligence approach have been leveraged to extract information from clinical narratives in electronic health records. This capability of natural language processing potentially enables automated chart review for identifying patients with distinctive clinical characteristics in clinical care and reduces methodological heterogeneity in defining phenotype, obscuring biological heterogeneity in research concerning allergy, asthma, and immunology. This brief review discusses the current literature on the secondary use of electronic health record data for clinical research concerning allergy, asthma, and immunology and highlights the potential, challenges, and implications of natural language processing techniques.

Asthma and risk of glioma: a population-based case-control study

OBJECTIVES

Literature suggests an inconsistent, but largely inverse, association between asthma and risk of glioma, which is primarily due to methodological inconsistency in sampling frame and ascertainment of asthma. The objective of the study was to clarify the association between asthma and risk of glioma by minimising methodological biases (eg, recall and detection bias).

DESIGN

A population-based case-control study.

SETTING

General population in Olmsted County, Minnesota, USA.

PARTICIPANTS

All eligible biopsy-proven incident glioma cases (1995-2014) and two sets of controls among residents matched to age and sex (first set: community controls without glioma; second set: MRI-negative controls from the same community).

METHODS

The predetermined asthma criteria via medical record review were applied to ascertain asthma status of cases and controls. History of asthma prior to index date was compared between glioma cases and their matched controls using conditional logistic regression models. Propensity score for asthma status was adjusted for multivariate analysis.

RESULTS

We enrolled 135 glioma cases (median age at index date: 53 years) and 270 controls. Of the cases, 21 had a history of asthma (16%), compared with 36 of MRI controls (27%) (OR (95% CI) 0.48 (0.26 to 0.91), p=0.03). With MRI controls, an inverse association between asthma and risk of glioma persisted after adjusting for the propensity score for asthma status, but did not reach statistical significance probably due to the lack of statistical power (OR (95% CI) 0.48 (0.21 to 1.09); p=0.08). Based on comparison of characteristics of controls and cases, community controls seem to be more susceptible to a detection bias.

CONCLUSIONS

While differential detection might account for the association between asthma and risk of glioma, asthma may potentially pose a protective effect on risk of glioma. Our study results need to be replicated by a larger study.

Relationship between asthma status and antibody response pattern to 23-valent pneumococcal vaccination

Objective: Asthma poses an increased risk for serious pneumococcal disease, but little is known about the influence of asthma status on the 23-valent serotype-specific pneumococcal antibody response. We examined differences in antibody titers between pre- and post-vaccination with 23-valent pneumococcal polysaccharide vaccine (PPSV-23) in relation to asthma status. Methods: Asthma status was retrospectively ascertained by the Predetermined Asthma Criteria in an existing vaccine cohort through comprehensive medical record review. Twenty-three serotype-specific pneumococcal antibody titers measured at baseline and 4-6 weeks post-vaccination were analyzed. Vaccine responses to PPSV-23 were calculated from pre- to post-vaccine titers for each of the serotypes. Results: Of the 64 eligible and enrolled subjects, 18 (28%) had asthma. Controls (i.e., subjects without asthma) demonstrated a statistically significant fold change response compared to their baseline for all serotypes, while those with asthma did not mount a significant response to serotypes 7F, 22F, and 23F. The overall vaccine response as measured by fold change over baseline was lower in subjects with asthma than controls. Conclusions: Poorer humoral immune responses to PPSV-23 as measured by fold change were more likely to be observed in subjects with asthma compared to controls. We recommend the consideration of asthma status when interpreting vaccine response for immune competence workup through larger studies. Further studies are warranted to replicate these findings.

Early Identification of Childhood Asthma: The Role of Informatics in an Era of Electronic Health Records

Emerging literature suggests that delayed identification of childhood asthma results in an increased risk of long-term and various morbidities compared to those with timely diagnosis and intervention, and yet this risk is still overlooked. Even when children and adolescents have a history of recurrent asthma-like symptoms and risk factors embedded in their medical records, this information is sometimes overlooked by clinicians at the point of care. Given the rapid adoption of electronic health record (EHR) systems, early identification of childhood asthma can be achieved utilizing (1) asthma ascertainment criteria leveraging relevant clinical information embedded in EHR and (2) innovative informatics approaches such as natural language processing (NLP) algorithms for asthma ascertainment criteria to enable such a strategy. In this review, we discuss literature relevant to this topic and introduce recently published informatics algorithms (criteria-based NLP) as a potential solution to address the current challenge of early identification of childhood asthma.