Early Prediction of Functional Outcomes After Acute Ischemic Stroke Using Unstructured Clinical Text: Retrospective Cohort Study

StrokeClassifier: ischemic stroke etiology classification by ensemble consensus modeling using electronic health records

Determining acute ischemic stroke (AIS) etiology is fundamental to secondary stroke prevention efforts but can be diagnostically challenging. We trained and validated an automated classification tool, StrokeClassifier, using electronic health record (EHR) text from 2039 non-cryptogenic AIS patients at 2 academic hospitals to predict the 4-level outcome of stroke etiology adjudicated by agreement of at least 2 board-certified vascular neurologists' review of the EHR. StrokeClassifier is an ensemble consensus meta-model of 9 machine learning classifiers applied to features extracted from discharge summary texts by natural language processing. StrokeClassifier was externally validated in 406 discharge summaries from the MIMIC-III dataset reviewed by a vascular neurologist to ascertain stroke etiology. Compared with vascular neurologists' diagnoses, StrokeClassifier achieved the mean cross-validated accuracy of 0.74 and weighted F1 of 0.74 for multi-class classification. In MIMIC-III, its accuracy and weighted F1 were 0.70 and 0.71, respectively. In binary classification, the two metrics ranged from 0.77 to 0.96. The top 5 features contributing to stroke etiology prediction were atrial fibrillation, age, middle cerebral artery occlusion, internal carotid artery occlusion, and frontal stroke location. We designed a certainty heuristic to grade the confidence of StrokeClassifier's diagnosis as non-cryptogenic by the degree of consensus among the 9 classifiers and applied it to 788 cryptogenic patients, reducing cryptogenic diagnoses from 25.2% to 7.2%. StrokeClassifier is a validated artificial intelligence tool that rivals the performance of vascular neurologists in classifying ischemic stroke etiology. With further training, StrokeClassifier may have downstream applications including its use as a clinical decision support system.

Clinical narratives as a predictor for prognosticating functional outcomes after intracerebral hemorrhage

BACKGROUND

Intracerebral hemorrhage (ICH) is a devastating stroke type that causes high mortality rates and severe disability among survivors. Many prognostic models are available for prognosticating patients with ICH. This study aimed to investigate whether clinical narratives can improve the performance for predicting functional outcomes after ICH.

METHODS

This study used data from the hospital stroke registry and electronic health records. The study population (n = 1363) was randomly divided into a training set (75%, n = 1023) and a holdout test set (25%, n = 340). Five risk scores for ICH were used as baseline prognostic models. Using natural language processing (NLP), text-based markers were generated from the clinical narratives of the training set through machine learning (ML) and deep learning (DL) approaches. The primary outcome was a poor functional outcome (modified Rankin Scale score of 3 to 6) at hospital discharge. The predictive performance was compared between the baseline models and models enhanced by incorporating the text-based markers using the holdout test set.

RESULTS

The enhanced prognostic models outperformed the baseline models, regardless of whether ML or DL approaches were used. The areas under the receiver operating characteristic curve (AUCs) of the baseline models were between 0.760 and 0.892. Adding the text-based marker to the baseline models significantly increased the model discrimination, with AUCs ranging from 0.861 to 0.914. The net reclassification improvement and integrated discrimination improvement indices also showed significant improvements.

CONCLUSIONS

Using NLP to extract textual information from clinical narratives could improve the predictive performance of all baseline prognostic models for ICH.

Applications of Natural Language Processing for the Management of Stroke Disorders: Scoping Review

BACKGROUND

Recent advances in natural language processing (NLP) have heightened the interest of the medical community in its application to health care in general, in particular to stroke, a medical emergency of great impact. In this rapidly evolving context, it is necessary to learn and understand the experience already accumulated by the medical and scientific community.

OBJECTIVE

The aim of this scoping review was to explore the studies conducted in the last 10 years using NLP to assist the management of stroke emergencies so as to gain insight on the state of the art, its main contexts of application, and the software tools that are used.

METHODS

Data were extracted from Scopus and Medline through PubMed, using the keywords "natural language processing" and "stroke." Primary research questions were related to the phases, contexts, and types of textual data used in the studies. Secondary research questions were related to the numerical and statistical methods and the software used to process the data. The extracted data were structured in tables and their relative frequencies were calculated. The relationships between categories were analyzed through multiple correspondence analysis.

RESULTS

Twenty-nine papers were included in the review, with the majority being cohort studies of ischemic stroke published in the last 2 years. The majority of papers focused on the use of NLP to assist in the diagnostic phase, followed by the outcome prognosis, using text data from diagnostic reports and in many cases annotations on medical images. The most frequent approach was based on general machine learning techniques applied to the results of relatively simple NLP methods with the support of ontologies and standard vocabularies. Although smaller in number, there has been an increasing body of studies using deep learning techniques on numerical and vectorized representations of the texts obtained with more sophisticated NLP tools.

CONCLUSIONS

Studies focused on NLP applied to stroke show specific trends that can be compared to the more general application of artificial intelligence to stroke. The purpose of using NLP is often to improve processes in a clinical context rather than to assist in the rehabilitation process. The state of the art in NLP is represented by deep learning architectures, among which Bidirectional Encoder Representations from Transformers has been found to be especially widely used in the medical field in general, and for stroke in particular, with an increasing focus on the processing of annotations on medical images.

Identification of Preanesthetic History Elements by a Natural Language Processing Engine

BACKGROUND

Methods that can automate, support, and streamline the preanesthesia evaluation process may improve resource utilization and efficiency. Natural language processing (NLP) involves the extraction of relevant information from unstructured text data. We describe the utilization of a clinical NLP pipeline intended to identify elements relevant to preoperative medical history by analyzing clinical notes. We hypothesize that the NLP pipeline would identify a significant portion of pertinent history captured by a perioperative provider.

METHODS

For each patient, we collected all pertinent notes from the institution's electronic medical record that were available no later than 1 day before their preoperative anesthesia clinic appointment. Pertinent notes included free-text notes consisting of history and physical, consultation, outpatient, inpatient progress, and previous preanesthetic evaluation notes. The free-text notes were processed by a Named Entity Recognition pipeline, an NLP machine learning model trained to recognize and label spans of text that corresponded to medical concepts. These medical concepts were then mapped to a list of medical conditions that were of interest for a preanesthesia evaluation. For each condition, we calculated the percentage of time across all patients in which (1) the NLP pipeline and the anesthesiologist both captured the condition; (2) the NLP pipeline captured the condition but the anesthesiologist did not; and (3) the NLP pipeline did not capture the condition but the anesthesiologist did.

RESULTS

A total of 93 patients were included in the NLP pipeline input. Free-text notes were extracted from the electronic medical record of these patients for a total of 9765 notes. The NLP pipeline and anesthesiologist agreed in 81.24% of instances on the presence or absence of a specific condition. The NLP pipeline identified information that was not noted by the anesthesiologist in 16.57% of instances and did not identify a condition that was noted by the anesthesiologist's review in 2.19% of instances.

CONCLUSIONS

In this proof-of-concept study, we demonstrated that utilization of NLP produced an output that identified medical conditions relevant to preanesthetic evaluation from unstructured free-text input. Automation of risk stratification tools may provide clinical decision support or recommend additional preoperative testing or evaluation. Future studies are needed to integrate these tools into clinical workflows and validate its efficacy.

Application of machine learning and natural language processing for predicting stroke-associated pneumonia

Background

Identifying patients at high risk of stroke-associated pneumonia (SAP) may permit targeting potential interventions to reduce its incidence. We aimed to explore the functionality of machine learning (ML) and natural language processing techniques on structured data and unstructured clinical text to predict SAP by comparing it to conventional risk scores.

Methods

Linked data between a hospital stroke registry and a deidentified research-based database including electronic health records and administrative claims data was used. Natural language processing was applied to extract textual features from clinical notes. The random forest algorithm was used to build ML models. The predictive performance of ML models was compared with the A²DS², ISAN, PNA, and ACDD⁴ scores using the area under the receiver operating characteristic curve (AUC).

Results

Among 5,913 acute stroke patients hospitalized between Oct 2010 and Sep 2021, 450 (7.6%) developed SAP within the first 7 days after stroke onset. The ML model based on both textual features and structured variables had the highest AUC [0.840, 95% confidence interval (CI) 0.806-0.875], significantly higher than those of the ML model based on structured variables alone (0.828, 95% CI 0.793-0.863, P = 0.040), ACDD⁴ (0.807, 95% CI 0.766-0.849, P = 0.041), A²DS² (0.803, 95% CI 0.762-0.845, P = 0.013), ISAN (0.795, 95% CI 0.752-0.837, P = 0.009), and PNA (0.778, 95% CI 0.735-0.822, P < 0.001). All models demonstrated adequate calibration except for the A²DS² score.

Conclusions

The ML model based on both textural features and structured variables performed better than conventional risk scores in predicting SAP. The workflow used to generate ML prediction models can be disseminated for local adaptation by individual healthcare organizations.

Automated risk assessment of newly detected atrial fibrillation poststroke from electronic health record data using machine learning and natural language processing

Background

Timely detection of atrial fibrillation (AF) after stroke is highly clinically relevant, aiding decisions on the optimal strategies for secondary prevention of stroke. In the context of limited medical resources, it is crucial to set the right priorities of extended heart rhythm monitoring by stratifying patients into different risk groups likely to have newly detected AF (NDAF). This study aimed to develop an electronic health record (EHR)-based machine learning model to assess the risk of NDAF in an early stage after stroke.

Methods

Linked data between a hospital stroke registry and a deidentified research-based database including EHRs and administrative claims data was used. Demographic features, physiological measurements, routine laboratory results, and clinical free text were extracted from EHRs. The extreme gradient boosting algorithm was used to build the prediction model. The prediction performance was evaluated by the C-index and was compared to that of the AS5F and CHASE-LESS scores.

Results

The study population consisted of a training set of 4,064 and a temporal test set of 1,492 patients. During a median follow-up of 10.2 months, the incidence rate of NDAF was 87.0 per 1,000 person-year in the test set. On the test set, the model based on both structured and unstructured data achieved a C-index of 0.840, which was significantly higher than those of the AS5F (0.779, p = 0.023) and CHASE-LESS (0.768, p = 0.005) scores.

Conclusions

It is feasible to build a machine learning model to assess the risk of NDAF based on EHR data available at the time of hospital admission. Inclusion of information derived from clinical free text can significantly improve the model performance and may outperform risk scores developed using traditional statistical methods. Further studies are needed to assess the clinical usefulness of the prediction model.